2024-04-23T18:08:48.139 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-23T18:08:48.139 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --archive /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114 --owner scheduled_leonidus@teuthology --name leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi -- /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/orig.config.yaml 2024-04-23T18:08:48.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T18:08:48.288 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7670114' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5182 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 57e0caf76fdb7ab8b1358588f292d08519163844 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAdzgjUz4UKsk+xC7gwWrQTH52WMXwv0EtInqpZV35h/C2SQZcujHFi6AoM7hEcBBKUK91DbDitIJJ7hTtCVy+w= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFu92g+Tux/vRXy/r03xV1Fu2Nbs1a+keFd+ch+/1ANXVh+Eja4TlcJq1YNqkn39KjFOAK+h5wdvxNa4fy9XZXo= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbICBQ+WuqZTjhay4PcBhL2mlwnjuW0IQoPWBv/rwJU+LuTDd1CBalq5aHbgLBTxwYr5F4VWwc14EA0EtEcQaY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-23_17:44:29 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-23T18:08:48.288 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa; will attempt to use it 2024-04-23T18:08:48.289 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks 2024-04-23T18:08:48.289 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-23T18:08:48.291 INFO:teuthology.task.internal:Checking packages... 2024-04-23T18:08:48.315 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-23T18:08:48.315 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-23T18:08:48.315 INFO:teuthology.packaging:ref: None 2024-04-23T18:08:48.315 INFO:teuthology.packaging:tag: None 2024-04-23T18:08:48.315 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-23T18:08:48.315 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:08:48.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-23T18:08:48.482 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-23T18:08:48.483 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-23T18:08:48.493 INFO:teuthology.task.internal:no buildpackages task found 2024-04-23T18:08:48.493 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-23T18:08:48.531 INFO:teuthology.task.internal:Saving configuration 2024-04-23T18:08:48.557 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-23T18:08:48.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T18:08:48.625 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.331214', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAdzgjUz4UKsk+xC7gwWrQTH52WMXwv0EtInqpZV35h/C2SQZcujHFi6AoM7hEcBBKUK91DbDitIJJ7hTtCVy+w='} 2024-04-23T18:08:48.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.333189', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFu92g+Tux/vRXy/r03xV1Fu2Nbs1a+keFd+ch+/1ANXVh+Eja4TlcJq1YNqkn39KjFOAK+h5wdvxNa4fy9XZXo='} 2024-04-23T18:08:48.666 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.329517', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbICBQ+WuqZTjhay4PcBhL2mlwnjuW0IQoPWBv/rwJU+LuTDd1CBalq5aHbgLBTxwYr5F4VWwc14EA0EtEcQaY='} 2024-04-23T18:08:48.667 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-23T18:08:48.675 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-23T18:08:48.675 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-23T18:08:48.675 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-23T18:08:48.676 INFO:teuthology.run_tasks:Running task console_log... 2024-04-23T18:08:48.792 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f71e82b60d0>, signals=[15]) 2024-04-23T18:08:48.792 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-23T18:08:48.800 INFO:teuthology.task.internal:Opening connections... 2024-04-23T18:08:48.800 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:08:48.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:08:48.885 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-23T18:08:48.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:08:48.967 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-23T18:08:48.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:08:49.040 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-23T18:08:49.047 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-23T18:08:49.066 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-23T18:08:49.066 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:08:49.124 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:08:49.125 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:08:49.126 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-23T18:08:49.148 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-23T18:08:49.166 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-23T18:08:49.166 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-23T18:08:49.223 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:08:49.224 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:08:49.225 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-23T18:08:49.239 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-23T18:08:49.257 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-23T18:08:49.257 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-23T18:08:49.314 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-23T18:08:49.314 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-23T18:08:49.314 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-23T18:08:49.314 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-23T18:08:49.314 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-23T18:08:49.315 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-23T18:08:49.315 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-23T18:08:49.339 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-23T18:08:49.351 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-23T18:08:49.369 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-23T18:08:49.369 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:08:49.372 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:08:49.374 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-23T18:08:49.390 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-23T18:08:49.397 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-23T18:08:49.397 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:08:49.429 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:08:49.432 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-23T18:08:49.470 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-23T18:08:49.556 INFO:teuthology.run_tasks:Running task kernel... 2024-04-23T18:08:49.572 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-23T18:08:49.572 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-23T18:08:49.572 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-23T18:08:49.573 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:08:49.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:08:49.590 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-23T18:08:49.649 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-437.el9.x86_64 2024-04-23T18:08:49.649 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-437.el9.x86_64 2024-04-23T18:08:49.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-04-23T18:08:52.913 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 5.0 MB/s | 8.0 MB 00:01 2024-04-23T18:08:54.689 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-23T18:08:59.222 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-04-23T18:09:01.584 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 16 kB 00:01 2024-04-23T18:09:06.978 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 4.0 MB/s | 21 MB 00:05 2024-04-23T18:09:12.232 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-23T18:09:14.164 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-23T18:09:14.164 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-23T18:09:14.260 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-23T18:09:14.261 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-23T18:09:14.261 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-23T18:09:14.356 DEBUG:teuthology.orchestra.run.smithi033:> echo no | sudo yum reinstall kernel || true 2024-04-23T18:09:14.878 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:02 ago on Tue 23 Apr 2024 06:09:12 PM UTC. 2024-04-23T18:09:15.007 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-23T18:09:15.016 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.017 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-23T18:09:15.018 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-23T18:09:15.018 INFO:teuthology.orchestra.run.smithi033.stderr:Operation aborted. 2024-04-23T18:09:15.070 INFO:teuthology.orchestra.run.smithi033.stdout:Is this ok [y/N]: 2024-04-23T18:09:15.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum reinstall -y kernel || true 2024-04-23T18:09:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:03 ago on Tue 23 Apr 2024 06:09:12 PM UTC. 2024-04-23T18:09:15.707 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-23T18:09:15.707 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.707 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-23T18:09:15.708 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-23T18:09:16.009 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-23T18:09:16.009 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:09:16.009 INFO:teuthology.orchestra.run.smithi033.stdout:Total 846 kB/s | 255 kB 00:00 2024-04-23T18:09:16.030 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-23T18:09:16.229 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-23T18:09:16.229 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-23T18:09:16.234 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-23T18:09:16.234 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-23T18:09:16.372 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-23T18:09:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:09:16.444 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:09:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:09:17.086 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-23T18:09:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-23T18:09:17.796 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:17.796 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalled: 2024-04-23T18:09:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-23T18:09:17.797 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:09:17.797 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-23T18:09:17.958 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-04-23T18:09:18.007 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-23T18:09:18.007 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-23T18:09:18.007 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-23T18:09:18.007 DEBUG:teuthology.parallel:result is None 2024-04-23T18:09:18.008 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-23T18:09:18.015 INFO:teuthology.task.internal:Creating test directory... 2024-04-23T18:09:18.015 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:09:18.018 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:09:18.020 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-23T18:09:18.038 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-23T18:09:18.057 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-23T18:09:18.081 INFO:teuthology.task.internal:Creating archive directory... 2024-04-23T18:09:18.081 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:09:18.083 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:09:18.086 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-23T18:09:18.182 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-23T18:09:18.190 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-23T18:09:18.190 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:09:18.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:09:18.207 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-23T18:09:18.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:09:18.223 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:09:18.226 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:09:18.249 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-23T18:09:18.279 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.286 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.293 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.304 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.326 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.330 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-23T18:09:18.332 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-23T18:09:18.339 INFO:teuthology.task.internal:Configuring sudo... 2024-04-23T18:09:18.339 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:09:18.342 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:09:18.371 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-23T18:09:18.397 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-23T18:09:18.408 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-23T18:09:18.408 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:09:18.410 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:09:18.438 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-23T18:09:18.454 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:18.507 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:18.560 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:18.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:18.681 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:09:18.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:09:18.746 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:18.787 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:18.849 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:18.934 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:18.978 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:09:18.978 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:09:19.045 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:19.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-23T18:09:19.148 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:19.232 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-23T18:09:19.277 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:09:19.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-23T18:09:19.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-23T18:09:19.346 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-23T18:09:19.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-23T18:09:19.387 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:09:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:09:19.426 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T18:09:19.775 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-23T18:09:19.783 INFO:teuthology.task.internal:Starting timer... 2024-04-23T18:09:19.783 INFO:teuthology.run_tasks:Running task pcp... 2024-04-23T18:09:19.817 INFO:teuthology.run_tasks:Running task selinux... 2024-04-23T18:09:19.842 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-23T18:09:19.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-04-23T18:09:19.897 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-04-23T18:09:19.898 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-23T18:09:19.958 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-23T18:09:19.960 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-04-23T18:09:20.010 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-04-23T18:09:20.012 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-23T18:09:20.013 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-04-23T18:09:20.043 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-04-23T18:09:20.043 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-23T18:09:20.081 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-23T18:09:20.081 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-04-23T18:09:20.102 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-04-23T18:09:20.103 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-04-23T18:09:20.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:09:20.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:09:20.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:09:20.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:09:20.158 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T18:09:20.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:09:20.188 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-23T18:09:20.188 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:09:20.221 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:09:20.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-04-23T18:09:20.277 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-23T18:09:20.287 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-23T18:09:20.293 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-23T18:09:20.426 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-23T18:09:20.442 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-23T18:09:20.443 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-23T18:13:25.426 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-04-23T18:13:25.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-23T18:13:25.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:13:25.506 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-23T18:13:25.586 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-23T18:13:25.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-23T18:13:25.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:13:25.665 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-23T18:13:25.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-23T18:13:25.738 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-23T18:13:25.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-23T18:13:25.811 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-23T18:13:25.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-23T18:13:25.900 INFO:teuthology.run_tasks:Running task clock... 2024-04-23T18:13:25.914 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-23T18:13:25.914 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:13:25.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:13:25.917 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:13:25.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:13:25.919 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-23T18:13:25.920 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T18:13:25.946 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:13:25.947 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:13:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:13:25.962 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:13:25.970 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-23T18:13:25.981 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-23T18:13:25.984 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-23T18:13:25.987 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-04-23T18:13:25.992 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-23T18:13:25.998 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-04-23T18:13:26.004 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:13:26.008 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-04-23T18:13:26.013 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:13:26.017 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:13:26.019 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-04-23T18:13:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:13:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-23T18:13:26.045 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-23T18:13:26.078 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.082 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.088 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.101 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.105 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T18:13:26.106 INFO:teuthology.run_tasks:Running task install... 2024-04-23T18:13:26.114 DEBUG:teuthology.task.install:project ceph 2024-04-23T18:13:26.115 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-23T18:13:26.115 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-23T18:13:26.115 INFO:teuthology.task.install:Using flavor: default 2024-04-23T18:13:26.121 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-23T18:13:26.121 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-23T18:13:26.121 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:13:26.122 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:13:26.124 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:13:26.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:13:26.125 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-23T18:13:26.125 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:13:26.275 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:13:26.276 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:13:26.343 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:13:26.343 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:13:26.347 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-23T18:13:26.347 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-23T18:13:26.447 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:13:26.448 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:13:26.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:13:26.455 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:13:26.455 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:13:26.455 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:13:26.474 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:13:26.474 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:13:26.481 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:13:26.481 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:13:26.517 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-23T18:13:26.518 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:13:26.518 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-23T18:13:26.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:13:26.543 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-23T18:13:26.544 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-23T18:13:26.547 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:13:26.609 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-23T18:13:26.614 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:13:26.621 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:13:26.679 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-04-23T18:13:26.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-04-23T18:13:26.684 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-23T18:13:26.685 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-04-23T18:13:26.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-04-23T18:13:26.747 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-23T18:13:26.749 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-23T18:13:26.910 INFO:teuthology.orchestra.run.smithi122.stdout:48 files removed 2024-04-23T18:13:26.915 INFO:teuthology.orchestra.run.smithi033.stdout:48 files removed 2024-04-23T18:13:26.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:13:26.942 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:13:26.974 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-23T18:13:27.002 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-23T18:13:27.532 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 247 kB/s | 83 kB 00:00 2024-04-23T18:13:27.668 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 184 kB/s | 83 kB 00:00 2024-04-23T18:13:27.727 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 180 kB/s | 83 kB 00:00 2024-04-23T18:13:27.950 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-23T18:13:28.052 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-23T18:13:28.070 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 39 kB/s | 12 kB 00:00 2024-04-23T18:13:28.325 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-23T18:13:28.427 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-23T18:13:28.487 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-23T18:13:28.994 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-23T18:13:29.084 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-23T18:13:29.095 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-23T18:13:30.922 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-23T18:13:32.102 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 9.7 MB/s | 19 MB 00:01 2024-04-23T18:13:36.523 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:02 2024-04-23T18:13:36.661 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-23T18:13:38.333 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-04-23T18:13:38.663 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 16 kB 00:00 2024-04-23T18:13:39.958 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-23T18:13:43.614 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 21 MB 00:04 2024-04-23T18:13:44.933 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-23T18:13:47.051 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:13:47.051 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:13:47.062 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:13:47.063 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:13:47.064 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:13:47.065 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:13:47.066 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:13:47.068 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:13:47.069 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:13:47.129 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-23T18:13:47.144 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-23T18:13:47.144 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-23T18:13:47.144 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:13:47.145 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:13:47.146 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:13:47.147 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:13:47.148 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:13:47.149 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:13:47.150 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:13:47.151 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:13:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:13:47.153 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:13:47.154 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:13:47.155 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:13:47.156 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 4 Packages 2024-04-23T18:13:47.157 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:13:47.158 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 222 M 2024-04-23T18:13:47.158 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-23T18:13:48.301 INFO:teuthology.orchestra.run.smithi106.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-23T18:13:48.501 INFO:teuthology.orchestra.run.smithi106.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 890 kB 00:00 2024-04-23T18:13:48.552 INFO:teuthology.orchestra.run.smithi106.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-23T18:13:48.647 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-23T18:13:48.818 INFO:teuthology.orchestra.run.smithi106.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 8.9 MB/s | 2.4 MB 00:00 2024-04-23T18:13:49.036 INFO:teuthology.orchestra.run.smithi106.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-23T18:13:49.102 INFO:teuthology.orchestra.run.smithi106.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.5 MB/s | 5.4 MB 00:00 2024-04-23T18:13:49.544 INFO:teuthology.orchestra.run.smithi106.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 9.2 MB/s | 4.7 MB 00:00 2024-04-23T18:13:50.528 INFO:teuthology.orchestra.run.smithi106.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 11 MB/s | 11 MB 00:00 2024-04-23T18:13:50.722 INFO:teuthology.orchestra.run.smithi106.stdout:(9/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 8.1 MB/s | 21 MB 00:02 2024-04-23T18:13:50.739 INFO:teuthology.orchestra.run.smithi106.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 119 kB/s | 25 kB 00:00 2024-04-23T18:13:50.774 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:13:50.774 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:13:50.785 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:13:50.785 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:13:50.786 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:13:50.787 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:13:50.788 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:13:50.790 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:13:50.791 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:13:50.797 INFO:teuthology.orchestra.run.smithi106.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 225 kB 00:00 2024-04-23T18:13:50.850 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-23T18:13:50.856 INFO:teuthology.orchestra.run.smithi106.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 465 kB/s | 27 kB 00:00 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:13:50.877 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:13:50.878 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:13:50.879 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:13:50.880 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:13:50.881 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:13:50.882 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:13:50.883 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:13:50.884 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:13:50.885 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:13:50.886 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:13:50.887 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:13:50.888 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-04-23T18:13:50.889 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:======================================================================================= 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 4 Packages 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 222 M 2024-04-23T18:13:50.890 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-23T18:13:50.914 INFO:teuthology.orchestra.run.smithi106.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 543 kB/s | 32 kB 00:00 2024-04-23T18:13:51.023 INFO:teuthology.orchestra.run.smithi106.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 6.7 MB/s | 747 kB 00:00 2024-04-23T18:13:51.098 INFO:teuthology.orchestra.run.smithi106.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 163 kB 00:00 2024-04-23T18:13:51.165 INFO:teuthology.orchestra.run.smithi106.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-23T18:13:51.240 INFO:teuthology.orchestra.run.smithi106.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 6.5 MB/s | 500 kB 00:00 2024-04-23T18:13:51.685 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 903 kB/s | 19 MB 00:21 2024-04-23T18:13:51.766 INFO:teuthology.orchestra.run.smithi122.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-23T18:13:51.782 INFO:teuthology.orchestra.run.smithi106.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 9.9 MB/s | 5.4 MB 00:00 2024-04-23T18:13:51.958 INFO:teuthology.orchestra.run.smithi106.stdout:(19/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 5.9 MB/s | 17 MB 00:02 2024-04-23T18:13:51.983 INFO:teuthology.orchestra.run.smithi106.stdout:(20/152): python3-ceph-argparse-19.0.0-3156.gd6 225 kB/s | 45 kB 00:00 2024-04-23T18:13:52.033 INFO:teuthology.orchestra.run.smithi106.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-23T18:13:52.059 INFO:teuthology.orchestra.run.smithi106.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-23T18:13:52.124 INFO:teuthology.orchestra.run.smithi122.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 890 kB 00:00 2024-04-23T18:13:52.134 INFO:teuthology.orchestra.run.smithi106.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.9 MB/s | 298 kB 00:00 2024-04-23T18:13:52.192 INFO:teuthology.orchestra.run.smithi106.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-23T18:13:52.216 INFO:teuthology.orchestra.run.smithi122.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.5 MB/s | 145 kB 00:00 2024-04-23T18:13:52.217 INFO:teuthology.orchestra.run.smithi106.stdout:(25/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 320 kB 00:00 2024-04-23T18:13:52.252 INFO:teuthology.orchestra.run.smithi106.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-23T18:13:52.360 INFO:teuthology.orchestra.run.smithi106.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-23T18:13:52.452 INFO:teuthology.orchestra.run.smithi106.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 371 kB/s | 34 kB 00:00 2024-04-23T18:13:52.561 INFO:teuthology.orchestra.run.smithi106.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.3 MB/s | 142 kB 00:00 2024-04-23T18:13:53.028 INFO:teuthology.orchestra.run.smithi106.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 3.1 MB 00:00 2024-04-23T18:13:53.320 INFO:teuthology.orchestra.run.smithi106.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.7 MB/s | 3.5 MB 00:00 2024-04-23T18:13:53.378 INFO:teuthology.orchestra.run.smithi106.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 4.2 MB/s | 249 kB 00:00 2024-04-23T18:13:53.428 INFO:teuthology.orchestra.run.smithi106.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 989 kB/s | 49 kB 00:00 2024-04-23T18:13:53.496 INFO:teuthology.orchestra.run.smithi106.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 246 kB/s | 16 kB 00:00 2024-04-23T18:13:53.554 INFO:teuthology.orchestra.run.smithi106.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.6 MB/s | 273 kB 00:00 2024-04-23T18:13:53.591 INFO:teuthology.orchestra.run.smithi122.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-23T18:13:53.633 INFO:teuthology.orchestra.run.smithi122.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 2.4 MB 00:01 2024-04-23T18:13:53.713 INFO:teuthology.orchestra.run.smithi106.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.7 MB/s | 762 kB 00:00 2024-04-23T18:13:53.780 INFO:teuthology.orchestra.run.smithi106.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-23T18:13:54.000 INFO:teuthology.orchestra.run.smithi122.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-23T18:13:54.297 INFO:teuthology.orchestra.run.smithi106.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 155 kB/s | 80 kB 00:00 2024-04-23T18:13:54.581 INFO:teuthology.orchestra.run.smithi106.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.8 MB/s | 7.4 MB 00:01 2024-04-23T18:13:54.975 INFO:teuthology.orchestra.run.smithi122.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 4.7 MB 00:01 2024-04-23T18:13:55.156 INFO:teuthology.orchestra.run.smithi106.stdout:(40/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 11 MB/s | 49 MB 00:04 2024-04-23T18:13:55.414 INFO:teuthology.orchestra.run.smithi106.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 234 kB/s | 195 kB 00:00 2024-04-23T18:13:55.517 INFO:teuthology.orchestra.run.smithi122.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 21 MB 00:04 2024-04-23T18:13:55.559 INFO:teuthology.orchestra.run.smithi122.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 610 kB/s | 25 kB 00:00 2024-04-23T18:13:55.973 INFO:teuthology.orchestra.run.smithi106.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 310 kB/s | 253 kB 00:00 2024-04-23T18:13:56.256 INFO:teuthology.orchestra.run.smithi106.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 410 kB/s | 803 kB 00:01 2024-04-23T18:13:56.301 INFO:teuthology.orchestra.run.smithi122.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 7.3 MB/s | 17 MB 00:02 2024-04-23T18:13:56.376 INFO:teuthology.orchestra.run.smithi122.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 225 kB 00:00 2024-04-23T18:13:56.398 INFO:teuthology.orchestra.run.smithi106.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 250 kB/s | 106 kB 00:00 2024-04-23T18:13:56.443 INFO:teuthology.orchestra.run.smithi122.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 406 kB/s | 27 kB 00:00 2024-04-23T18:13:56.457 INFO:teuthology.orchestra.run.smithi106.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-04-23T18:13:56.510 INFO:teuthology.orchestra.run.smithi122.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 476 kB/s | 32 kB 00:00 2024-04-23T18:13:56.682 INFO:teuthology.orchestra.run.smithi106.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 318 kB/s | 135 kB 00:00 2024-04-23T18:13:56.686 INFO:teuthology.orchestra.run.smithi122.stdout:(14/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 6.2 MB/s | 11 MB 00:01 2024-04-23T18:13:56.719 INFO:teuthology.orchestra.run.smithi122.stdout:(15/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.5 MB/s | 747 kB 00:00 2024-04-23T18:13:56.745 INFO:teuthology.orchestra.run.smithi122.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-23T18:13:56.787 INFO:teuthology.orchestra.run.smithi122.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-23T18:13:56.870 INFO:teuthology.orchestra.run.smithi122.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 3.9 MB/s | 500 kB 00:00 2024-04-23T18:13:56.891 INFO:teuthology.orchestra.run.smithi106.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-04-23T18:13:56.901 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-23T18:13:56.916 INFO:teuthology.orchestra.run.smithi106.stdout:(48/152): python3-requests-2.25.1-8.el9.noarch. 241 kB/s | 125 kB 00:00 2024-04-23T18:13:56.920 INFO:teuthology.orchestra.run.smithi122.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 904 kB/s | 45 kB 00:00 2024-04-23T18:13:56.966 INFO:teuthology.orchestra.run.smithi106.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-23T18:13:56.987 INFO:teuthology.orchestra.run.smithi122.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.0 MB/s | 135 kB 00:00 2024-04-23T18:13:56.991 INFO:teuthology.orchestra.run.smithi106.stdout:(50/152): python3-urllib3-1.26.5-5.el9.noarch.r 402 kB/s | 215 kB 00:00 2024-04-23T18:13:57.017 INFO:teuthology.orchestra.run.smithi106.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 428 kB/s | 21 kB 00:00 2024-04-23T18:13:57.046 INFO:teuthology.orchestra.run.smithi122.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-23T18:13:57.050 INFO:teuthology.orchestra.run.smithi106.stdout:(52/152): boost-program-options-1.75.0-8.el9.x8 797 kB/s | 107 kB 00:00 2024-04-23T18:13:57.076 INFO:teuthology.orchestra.run.smithi106.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-23T18:13:57.126 INFO:teuthology.orchestra.run.smithi106.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-23T18:13:57.130 INFO:teuthology.orchestra.run.smithi122.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.8 MB/s | 320 kB 00:00 2024-04-23T18:13:57.176 INFO:teuthology.orchestra.run.smithi106.stdout:(55/152): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-23T18:13:57.202 INFO:teuthology.orchestra.run.smithi106.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-23T18:13:57.213 INFO:teuthology.orchestra.run.smithi122.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-23T18:13:57.227 INFO:teuthology.orchestra.run.smithi106.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-23T18:13:57.252 INFO:teuthology.orchestra.run.smithi106.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-23T18:13:57.272 INFO:teuthology.orchestra.run.smithi122.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-23T18:13:57.286 INFO:teuthology.orchestra.run.smithi106.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-23T18:13:57.331 INFO:teuthology.orchestra.run.smithi122.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-23T18:13:57.336 INFO:teuthology.orchestra.run.smithi106.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-04-23T18:13:57.361 INFO:teuthology.orchestra.run.smithi106.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 339 kB/s | 25 kB 00:00 2024-04-23T18:13:57.386 INFO:teuthology.orchestra.run.smithi106.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-23T18:13:57.420 INFO:teuthology.orchestra.run.smithi106.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-23T18:13:57.445 INFO:teuthology.orchestra.run.smithi106.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-23T18:13:57.471 INFO:teuthology.orchestra.run.smithi106.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-23T18:13:57.496 INFO:teuthology.orchestra.run.smithi106.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-23T18:13:57.540 INFO:teuthology.orchestra.run.smithi122.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.1 MB/s | 5.4 MB 00:00 2024-04-23T18:13:57.621 INFO:teuthology.orchestra.run.smithi106.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-04-23T18:13:57.624 INFO:teuthology.orchestra.run.smithi122.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.0 MB/s | 170 kB 00:00 2024-04-23T18:13:57.688 INFO:teuthology.orchestra.run.smithi106.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-23T18:13:57.690 INFO:teuthology.orchestra.run.smithi122.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 513 kB/s | 34 kB 00:00 2024-04-23T18:13:57.722 INFO:teuthology.orchestra.run.smithi106.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-23T18:13:57.747 INFO:teuthology.orchestra.run.smithi106.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-23T18:13:57.766 INFO:teuthology.orchestra.run.smithi122.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.9 MB/s | 142 kB 00:00 2024-04-23T18:13:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:(71/152): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-23T18:13:57.823 INFO:teuthology.orchestra.run.smithi106.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-23T18:13:57.848 INFO:teuthology.orchestra.run.smithi106.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-23T18:13:57.875 INFO:teuthology.orchestra.run.smithi122.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 5.7 MB/s | 3.1 MB 00:00 2024-04-23T18:13:57.923 INFO:teuthology.orchestra.run.smithi106.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-23T18:13:58.007 INFO:teuthology.orchestra.run.smithi106.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-04-23T18:13:58.032 INFO:teuthology.orchestra.run.smithi106.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-23T18:13:58.049 INFO:teuthology.orchestra.run.smithi106.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-04-23T18:13:58.116 INFO:teuthology.orchestra.run.smithi106.stdout:(78/152): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-04-23T18:13:58.133 INFO:teuthology.orchestra.run.smithi106.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 190 kB/s | 19 kB 00:00 2024-04-23T18:13:58.158 INFO:teuthology.orchestra.run.smithi106.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-04-23T18:13:58.175 INFO:teuthology.orchestra.run.smithi106.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-04-23T18:13:58.217 INFO:teuthology.orchestra.run.smithi106.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-23T18:13:58.242 INFO:teuthology.orchestra.run.smithi106.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-04-23T18:13:58.267 INFO:teuthology.orchestra.run.smithi106.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-23T18:13:58.275 INFO:teuthology.orchestra.run.smithi122.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.0 MB/s | 3.5 MB 00:00 2024-04-23T18:13:58.292 INFO:teuthology.orchestra.run.smithi106.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-23T18:13:58.326 INFO:teuthology.orchestra.run.smithi106.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-23T18:13:58.344 INFO:teuthology.orchestra.run.smithi122.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.5 MB/s | 249 kB 00:00 2024-04-23T18:13:58.351 INFO:teuthology.orchestra.run.smithi106.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-23T18:13:58.377 INFO:teuthology.orchestra.run.smithi106.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T18:13:58.402 INFO:teuthology.orchestra.run.smithi106.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-23T18:13:58.411 INFO:teuthology.orchestra.run.smithi122.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 741 kB/s | 49 kB 00:00 2024-04-23T18:13:58.427 INFO:teuthology.orchestra.run.smithi106.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T18:13:58.478 INFO:teuthology.orchestra.run.smithi122.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 247 kB/s | 16 kB 00:00 2024-04-23T18:13:58.486 INFO:teuthology.orchestra.run.smithi106.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-23T18:13:58.511 INFO:teuthology.orchestra.run.smithi106.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 498 kB/s | 42 kB 00:00 2024-04-23T18:13:58.536 INFO:teuthology.orchestra.run.smithi106.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-04-23T18:13:58.562 INFO:teuthology.orchestra.run.smithi106.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-23T18:13:58.570 INFO:teuthology.orchestra.run.smithi122.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 273 kB 00:00 2024-04-23T18:13:58.587 INFO:teuthology.orchestra.run.smithi106.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-23T18:13:58.612 INFO:teuthology.orchestra.run.smithi106.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-23T18:13:58.638 INFO:teuthology.orchestra.run.smithi106.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T18:13:58.663 INFO:teuthology.orchestra.run.smithi106.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T18:13:58.689 INFO:teuthology.orchestra.run.smithi106.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T18:13:58.712 INFO:teuthology.orchestra.run.smithi122.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.3 MB/s | 762 kB 00:00 2024-04-23T18:13:58.714 INFO:teuthology.orchestra.run.smithi106.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T18:13:58.747 INFO:teuthology.orchestra.run.smithi106.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-04-23T18:13:58.779 INFO:teuthology.orchestra.run.smithi122.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-23T18:13:58.823 INFO:teuthology.orchestra.run.smithi106.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-04-23T18:13:58.904 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-23T18:13:58.923 INFO:teuthology.orchestra.run.smithi106.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-04-23T18:13:58.929 INFO:teuthology.orchestra.run.smithi122.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-04-23T18:13:59.107 INFO:teuthology.orchestra.run.smithi106.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-04-23T18:13:59.232 INFO:teuthology.orchestra.run.smithi106.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-04-23T18:13:59.324 INFO:teuthology.orchestra.run.smithi106.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 771 kB/s | 309 kB 00:00 2024-04-23T18:13:59.382 INFO:teuthology.orchestra.run.smithi106.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-04-23T18:13:59.433 INFO:teuthology.orchestra.run.smithi106.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 899 kB/s | 548 kB 00:00 2024-04-23T18:13:59.458 INFO:teuthology.orchestra.run.smithi106.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-04-23T18:13:59.516 INFO:teuthology.orchestra.run.smithi106.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-04-23T18:13:59.542 INFO:teuthology.orchestra.run.smithi106.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-23T18:13:59.563 INFO:teuthology.orchestra.run.smithi122.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 102 kB/s | 80 kB 00:00 2024-04-23T18:13:59.634 INFO:teuthology.orchestra.run.smithi106.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-23T18:13:59.667 INFO:teuthology.orchestra.run.smithi106.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-23T18:13:59.693 INFO:teuthology.orchestra.run.smithi106.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-04-23T18:13:59.718 INFO:teuthology.orchestra.run.smithi106.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-04-23T18:13:59.743 INFO:teuthology.orchestra.run.smithi106.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-23T18:13:59.918 INFO:teuthology.orchestra.run.smithi106.stdout:(117/152): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-04-23T18:13:59.944 INFO:teuthology.orchestra.run.smithi106.stdout:(118/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.9 MB/s | 838 kB 00:00 2024-04-23T18:13:59.969 INFO:teuthology.orchestra.run.smithi106.stdout:(119/152): python3-cheroot-8.6.0-4.el9.noarch.r 685 kB/s | 172 kB 00:00 2024-04-23T18:13:59.994 INFO:teuthology.orchestra.run.smithi106.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-23T18:14:00.036 INFO:teuthology.orchestra.run.smithi106.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 382 kB/s | 35 kB 00:00 2024-04-23T18:14:00.061 INFO:teuthology.orchestra.run.smithi106.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-23T18:14:00.120 INFO:teuthology.orchestra.run.smithi106.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2024-04-23T18:14:00.137 INFO:teuthology.orchestra.run.smithi106.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-23T18:14:00.162 INFO:teuthology.orchestra.run.smithi106.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-23T18:14:00.179 INFO:teuthology.orchestra.run.smithi106.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-04-23T18:14:00.197 INFO:teuthology.orchestra.run.smithi122.stdout:(40/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 11 MB/s | 49 MB 00:04 2024-04-23T18:14:00.205 INFO:teuthology.orchestra.run.smithi106.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 343 kB/s | 23 kB 00:00 2024-04-23T18:14:00.230 INFO:teuthology.orchestra.run.smithi106.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 287 kB/s | 19 kB 00:00 2024-04-23T18:14:00.255 INFO:teuthology.orchestra.run.smithi106.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-04-23T18:14:00.289 INFO:teuthology.orchestra.run.smithi106.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-04-23T18:14:00.314 INFO:teuthology.orchestra.run.smithi106.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-23T18:14:00.398 INFO:teuthology.orchestra.run.smithi106.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 554 kB/s | 79 kB 00:00 2024-04-23T18:14:00.448 INFO:teuthology.orchestra.run.smithi106.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 363 kB/s | 58 kB 00:00 2024-04-23T18:14:00.474 INFO:teuthology.orchestra.run.smithi106.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 219 kB/s | 16 kB 00:00 2024-04-23T18:14:00.499 INFO:teuthology.orchestra.run.smithi106.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-23T18:14:00.730 INFO:teuthology.orchestra.run.smithi122.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 446 kB/s | 803 kB 00:01 2024-04-23T18:14:00.864 INFO:teuthology.orchestra.run.smithi122.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 150 kB/s | 195 kB 00:01 2024-04-23T18:14:01.147 INFO:teuthology.orchestra.run.smithi122.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 274 kB/s | 253 kB 00:00 2024-04-23T18:14:01.453 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 21 MB 00:02 2024-04-23T18:14:01.614 INFO:teuthology.orchestra.run.smithi122.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 142 kB/s | 106 kB 00:00 2024-04-23T18:14:01.931 INFO:teuthology.orchestra.run.smithi122.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 172 kB/s | 135 kB 00:00 2024-04-23T18:14:01.999 INFO:teuthology.orchestra.run.smithi106.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 58 kB/s | 90 kB 00:01 2024-04-23T18:14:02.024 INFO:teuthology.orchestra.run.smithi106.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 18 kB/s | 28 kB 00:01 2024-04-23T18:14:02.049 INFO:teuthology.orchestra.run.smithi106.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 20 kB/s | 31 kB 00:01 2024-04-23T18:14:02.083 INFO:teuthology.orchestra.run.smithi106.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-23T18:14:02.133 INFO:teuthology.orchestra.run.smithi106.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-04-23T18:14:02.456 INFO:teuthology.orchestra.run.smithi122.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 148 kB/s | 125 kB 00:00 2024-04-23T18:14:02.657 INFO:teuthology.orchestra.run.smithi122.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-23T18:14:02.758 INFO:teuthology.orchestra.run.smithi122.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-04-23T18:14:02.807 INFO:teuthology.orchestra.run.smithi122.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-23T18:14:02.849 INFO:teuthology.orchestra.run.smithi122.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 517 kB/s | 21 kB 00:00 2024-04-23T18:14:02.874 INFO:teuthology.orchestra.run.smithi122.stdout:(51/152): python3-urllib3-1.26.5-5.el9.noarch.r 228 kB/s | 215 kB 00:00 2024-04-23T18:14:02.917 INFO:teuthology.orchestra.run.smithi122.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-23T18:14:02.992 INFO:teuthology.orchestra.run.smithi122.stdout:(53/152): python3-cryptography-36.0.1-4.el9.x86 555 kB/s | 1.2 MB 00:02 2024-04-23T18:14:03.017 INFO:teuthology.orchestra.run.smithi122.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-04-23T18:14:03.059 INFO:teuthology.orchestra.run.smithi122.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-23T18:14:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 181 kB/s | 18 kB 00:00 2024-04-23T18:14:03.151 INFO:teuthology.orchestra.run.smithi122.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-23T18:14:03.193 INFO:teuthology.orchestra.run.smithi122.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-04-23T18:14:03.244 INFO:teuthology.orchestra.run.smithi122.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-04-23T18:14:03.294 INFO:teuthology.orchestra.run.smithi122.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-04-23T18:14:03.328 INFO:teuthology.orchestra.run.smithi122.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.2 MB/s | 582 kB 00:00 2024-04-23T18:14:03.353 INFO:teuthology.orchestra.run.smithi122.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-23T18:14:03.379 INFO:teuthology.orchestra.run.smithi122.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T18:14:03.429 INFO:teuthology.orchestra.run.smithi122.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 704 kB/s | 35 kB 00:00 2024-04-23T18:14:03.454 INFO:teuthology.orchestra.run.smithi122.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-23T18:14:03.505 INFO:teuthology.orchestra.run.smithi122.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.1 MB/s | 3.0 MB 00:00 2024-04-23T18:14:03.672 INFO:teuthology.orchestra.run.smithi122.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-23T18:14:03.811 INFO:teuthology.orchestra.run.smithi122.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.4 MB/s | 911 kB 00:00 2024-04-23T18:14:03.856 INFO:teuthology.orchestra.run.smithi122.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.0 MB/s | 133 kB 00:00 2024-04-23T18:14:03.989 INFO:teuthology.orchestra.run.smithi122.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.5 MB/s | 4.7 MB 00:00 2024-04-23T18:14:04.081 INFO:teuthology.orchestra.run.smithi122.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.1 MB/s | 4.4 MB 00:00 2024-04-23T18:14:04.131 INFO:teuthology.orchestra.run.smithi122.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-23T18:14:04.298 INFO:teuthology.orchestra.run.smithi122.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-04-23T18:14:04.349 INFO:teuthology.orchestra.run.smithi122.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-23T18:14:04.399 INFO:teuthology.orchestra.run.smithi122.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-23T18:14:04.441 INFO:teuthology.orchestra.run.smithi122.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 620 kB/s | 26 kB 00:00 2024-04-23T18:14:04.483 INFO:teuthology.orchestra.run.smithi122.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 459 kB/s | 19 kB 00:00 2024-04-23T18:14:04.533 INFO:teuthology.orchestra.run.smithi122.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-23T18:14:04.625 INFO:teuthology.orchestra.run.smithi122.stdout:(79/152): postgresql-server-13.14-1.el9.x86_64. 7.5 MB/s | 5.8 MB 00:00 2024-04-23T18:14:04.642 INFO:teuthology.orchestra.run.smithi122.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-04-23T18:14:04.717 INFO:teuthology.orchestra.run.smithi122.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-23T18:14:04.743 INFO:teuthology.orchestra.run.smithi122.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-23T18:14:04.776 INFO:teuthology.orchestra.run.smithi122.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-23T18:14:04.827 INFO:teuthology.orchestra.run.smithi122.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 849 kB/s | 42 kB 00:00 2024-04-23T18:14:04.877 INFO:teuthology.orchestra.run.smithi122.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-23T18:14:04.903 INFO:teuthology.orchestra.run.smithi122.stdout:(86/152): python3-packaging-20.9-5.el9.noarch.r 484 kB/s | 77 kB 00:00 2024-04-23T18:14:04.919 INFO:teuthology.orchestra.run.smithi122.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-23T18:14:04.953 INFO:teuthology.orchestra.run.smithi122.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-23T18:14:04.978 INFO:teuthology.orchestra.run.smithi122.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-04-23T18:14:05.004 INFO:teuthology.orchestra.run.smithi122.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-23T18:14:05.054 INFO:teuthology.orchestra.run.smithi122.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-04-23T18:14:05.113 INFO:teuthology.orchestra.run.smithi122.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-23T18:14:05.205 INFO:teuthology.orchestra.run.smithi122.stdout:(93/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.8 MB/s | 5.1 MB 00:00 2024-04-23T18:14:05.230 INFO:teuthology.orchestra.run.smithi122.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-04-23T18:14:05.255 INFO:teuthology.orchestra.run.smithi122.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-23T18:14:05.280 INFO:teuthology.orchestra.run.smithi122.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-23T18:14:05.306 INFO:teuthology.orchestra.run.smithi122.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T18:14:05.331 INFO:teuthology.orchestra.run.smithi122.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T18:14:05.365 INFO:teuthology.orchestra.run.smithi122.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-23T18:14:05.390 INFO:teuthology.orchestra.run.smithi122.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-23T18:14:05.415 INFO:teuthology.orchestra.run.smithi122.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-23T18:14:05.524 INFO:teuthology.orchestra.run.smithi122.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-04-23T18:14:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-04-23T18:14:06.142 INFO:teuthology.orchestra.run.smithi122.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 772 kB/s | 309 kB 00:00 2024-04-23T18:14:06.242 INFO:teuthology.orchestra.run.smithi122.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-23T18:14:06.275 INFO:teuthology.orchestra.run.smithi122.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 730 kB/s | 548 kB 00:00 2024-04-23T18:14:06.359 INFO:teuthology.orchestra.run.smithi122.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-23T18:14:06.434 INFO:teuthology.orchestra.run.smithi122.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-04-23T18:14:06.543 INFO:teuthology.orchestra.run.smithi122.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 622 kB/s | 67 kB 00:00 2024-04-23T18:14:06.626 INFO:teuthology.orchestra.run.smithi122.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-23T18:14:06.695 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-23T18:14:06.852 INFO:teuthology.orchestra.run.smithi122.stdout:(111/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.7 MB/s | 16 MB 00:01 2024-04-23T18:14:07.003 INFO:teuthology.orchestra.run.smithi122.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-04-23T18:14:07.061 INFO:teuthology.orchestra.run.smithi122.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 762 kB/s | 43 kB 00:00 2024-04-23T18:14:07.120 INFO:teuthology.orchestra.run.smithi122.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 552 kB/s | 32 kB 00:00 2024-04-23T18:14:07.170 INFO:teuthology.orchestra.run.smithi122.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 301 kB/s | 15 kB 00:00 2024-04-23T18:14:07.229 INFO:teuthology.orchestra.run.smithi122.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-23T18:14:07.254 INFO:teuthology.orchestra.run.smithi122.stdout:(117/152): python3-asyncssh-2.13.2-5.el9.noarch 1.3 MB/s | 548 kB 00:00 2024-04-23T18:14:07.330 INFO:teuthology.orchestra.run.smithi122.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-23T18:14:07.363 INFO:teuthology.orchestra.run.smithi122.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-23T18:14:07.380 INFO:teuthology.orchestra.run.smithi122.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-04-23T18:14:07.422 INFO:teuthology.orchestra.run.smithi122.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-23T18:14:07.481 INFO:teuthology.orchestra.run.smithi122.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-23T18:14:07.698 INFO:teuthology.orchestra.run.smithi122.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 49 kB/s | 11 kB 00:00 2024-04-23T18:14:07.782 INFO:teuthology.orchestra.run.smithi122.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-23T18:14:07.840 INFO:teuthology.orchestra.run.smithi122.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.4 MB/s | 2.0 MB 00:00 2024-04-23T18:14:07.865 INFO:teuthology.orchestra.run.smithi122.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-23T18:14:07.899 INFO:teuthology.orchestra.run.smithi122.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-23T18:14:07.925 INFO:teuthology.orchestra.run.smithi122.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-23T18:14:07.991 INFO:teuthology.orchestra.run.smithi122.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 695 kB/s | 46 kB 00:00 2024-04-23T18:14:08.050 INFO:teuthology.orchestra.run.smithi122.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-23T18:14:08.083 INFO:teuthology.orchestra.run.smithi106.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 5.9 kB/s | 36 kB 00:06 2024-04-23T18:14:08.083 INFO:teuthology.orchestra.run.smithi122.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-04-23T18:14:08.108 INFO:teuthology.orchestra.run.smithi106.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 8.5 kB/s | 51 kB 00:06 2024-04-23T18:14:08.109 INFO:teuthology.orchestra.run.smithi122.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-04-23T18:14:08.151 INFO:teuthology.orchestra.run.smithi122.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-23T18:14:08.175 INFO:teuthology.orchestra.run.smithi106.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 38 kB/s | 230 kB 00:06 2024-04-23T18:14:08.176 INFO:teuthology.orchestra.run.smithi122.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-04-23T18:14:08.210 INFO:teuthology.orchestra.run.smithi122.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T18:14:08.236 INFO:teuthology.orchestra.run.smithi122.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 480 kB/s | 28 kB 00:00 2024-04-23T18:14:08.269 INFO:teuthology.orchestra.run.smithi122.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 524 kB/s | 31 kB 00:00 2024-04-23T18:14:08.295 INFO:teuthology.orchestra.run.smithi122.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-23T18:14:08.328 INFO:teuthology.orchestra.run.smithi122.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-23T18:14:08.354 INFO:teuthology.orchestra.run.smithi122.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-04-23T18:14:08.387 INFO:teuthology.orchestra.run.smithi122.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 873 kB/s | 51 kB 00:00 2024-04-23T18:14:08.412 INFO:teuthology.orchestra.run.smithi122.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-23T18:14:08.446 INFO:teuthology.orchestra.run.smithi122.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-23T18:14:08.505 INFO:teuthology.orchestra.run.smithi122.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-23T18:14:08.530 INFO:teuthology.orchestra.run.smithi122.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-23T18:14:08.564 INFO:teuthology.orchestra.run.smithi122.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-23T18:14:08.772 INFO:teuthology.orchestra.run.smithi122.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 6.5 MB/s | 1.6 MB 00:00 2024-04-23T18:14:08.898 INFO:teuthology.orchestra.run.smithi122.stdout:(148/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 4.4 MB 00:02 2024-04-23T18:14:08.906 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:14:08.907 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-23T18:14:08.918 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-23T18:14:08.919 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-23T18:14:08.920 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-23T18:14:08.921 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-23T18:14:08.922 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-23T18:14:08.925 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-23T18:14:08.926 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-23T18:14:08.992 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-23T18:14:09.001 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-23T18:14:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-23T18:14:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-23T18:14:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-23T18:14:09.005 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-23T18:14:09.006 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-23T18:14:09.007 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-23T18:14:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-23T18:14:09.009 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-23T18:14:09.010 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-23T18:14:09.011 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-23T18:14:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-23T18:14:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-23T18:14:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-23T18:14:09.015 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-23T18:14:09.016 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-23T18:14:09.017 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-23T18:14:09.018 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-23T18:14:09.019 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-23T18:14:09.020 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:14:09.021 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-23T18:14:09.022 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-04-23T18:14:09.022 INFO:teuthology.orchestra.run.smithi033.stdout:Install 148 Packages 2024-04-23T18:14:09.022 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 4 Packages 2024-04-23T18:14:09.022 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:14:09.023 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 222 M 2024-04-23T18:14:09.023 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-23T18:14:09.049 INFO:teuthology.orchestra.run.smithi122.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.0 MB/s | 3.4 MB 00:00 2024-04-23T18:14:09.323 INFO:teuthology.orchestra.run.smithi122.stdout:(150/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.7 MB/s | 3.1 MB 00:00 2024-04-23T18:14:09.947 INFO:teuthology.orchestra.run.smithi033.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-23T18:14:10.117 INFO:teuthology.orchestra.run.smithi106.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 10 kB/s | 20 kB 00:01 2024-04-23T18:14:10.134 INFO:teuthology.orchestra.run.smithi106.stdout:(145/152): python3-websocket-client-1.2.3-2.el9 44 kB/s | 90 kB 00:02 2024-04-23T18:14:10.168 INFO:teuthology.orchestra.run.smithi106.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 207 kB/s | 427 kB 00:02 2024-04-23T18:14:10.297 INFO:teuthology.orchestra.run.smithi033.stdout:(2/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 9.0 MB/s | 5.4 MB 00:00 2024-04-23T18:14:10.339 INFO:teuthology.orchestra.run.smithi033.stdout:(3/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 890 kB 00:00 2024-04-23T18:14:10.364 INFO:teuthology.orchestra.run.smithi033.stdout:(4/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-23T18:14:10.451 INFO:teuthology.orchestra.run.smithi106.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-04-23T18:14:10.486 INFO:teuthology.orchestra.run.smithi106.stdout:(148/152): re2-20211101-3.el9.x86_64.rpm 522 kB/s | 192 kB 00:00 2024-04-23T18:14:10.547 INFO:teuthology.orchestra.run.smithi033.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 8.2 MB/s | 1.5 MB 00:00 2024-04-23T18:14:10.693 INFO:teuthology.orchestra.run.smithi106.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.5 MB/s | 3.4 MB 00:00 2024-04-23T18:14:10.989 INFO:teuthology.orchestra.run.smithi033.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 4.7 MB 00:00 2024-04-23T18:14:11.148 INFO:teuthology.orchestra.run.smithi033.stdout:(7/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.9 MB/s | 2.4 MB 00:00 2024-04-23T18:14:11.198 INFO:teuthology.orchestra.run.smithi122.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 601 kB/s | 1.3 MB 00:02 2024-04-23T18:14:11.227 INFO:teuthology.orchestra.run.smithi106.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 1.3 MB 00:00 2024-04-23T18:14:11.677 INFO:teuthology.orchestra.run.smithi106.stdout:(151/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 3.1 MB 00:01 2024-04-23T18:14:12.035 INFO:teuthology.orchestra.run.smithi106.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.8 MB/s | 2.4 MB 00:01 2024-04-23T18:14:12.039 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:14:12.039 INFO:teuthology.orchestra.run.smithi106.stdout:Total 8.9 MB/s | 222 MB 00:24 2024-04-23T18:14:12.215 INFO:teuthology.orchestra.run.smithi033.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 8.4 MB/s | 21 MB 00:02 2024-04-23T18:14:12.399 INFO:teuthology.orchestra.run.smithi033.stdout:(9/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 12 MB/s | 17 MB 00:01 2024-04-23T18:14:12.415 INFO:teuthology.orchestra.run.smithi033.stdout:(10/152): ceph-selinux-19.0.0-3156.gd69f13d8.el 126 kB/s | 25 kB 00:00 2024-04-23T18:14:12.499 INFO:teuthology.orchestra.run.smithi033.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.6 MB/s | 225 kB 00:00 2024-04-23T18:14:12.582 INFO:teuthology.orchestra.run.smithi033.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 325 kB/s | 27 kB 00:00 2024-04-23T18:14:12.650 INFO:teuthology.orchestra.run.smithi033.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 472 kB/s | 32 kB 00:00 2024-04-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi033.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 6.3 MB/s | 747 kB 00:00 2024-04-23T18:14:12.850 INFO:teuthology.orchestra.run.smithi033.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.9 MB/s | 163 kB 00:00 2024-04-23T18:14:12.865 INFO:teuthology.orchestra.run.smithi122.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 642 kB/s | 2.4 MB 00:03 2024-04-23T18:14:12.871 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:14:12.871 INFO:teuthology.orchestra.run.smithi122.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-23T18:14:12.934 INFO:teuthology.orchestra.run.smithi033.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-23T18:14:13.034 INFO:teuthology.orchestra.run.smithi033.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.9 MB/s | 500 kB 00:00 2024-04-23T18:14:13.100 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-23T18:14:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-23T18:14:13.243 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-23T18:14:13.685 INFO:teuthology.orchestra.run.smithi033.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.2 MB/s | 5.4 MB 00:00 2024-04-23T18:14:13.768 INFO:teuthology.orchestra.run.smithi033.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 543 kB/s | 45 kB 00:00 2024-04-23T18:14:13.893 INFO:teuthology.orchestra.run.smithi033.stdout:(20/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.9 MB/s | 11 MB 00:02 2024-04-23T18:14:13.910 INFO:teuthology.orchestra.run.smithi033.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 950 kB/s | 135 kB 00:00 2024-04-23T18:14:13.969 INFO:teuthology.orchestra.run.smithi033.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.1 MB/s | 161 kB 00:00 2024-04-23T18:14:13.981 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-23T18:14:13.994 INFO:teuthology.orchestra.run.smithi033.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.7 MB/s | 320 kB 00:00 2024-04-23T18:14:14.053 INFO:teuthology.orchestra.run.smithi033.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 298 kB 00:00 2024-04-23T18:14:14.086 INFO:teuthology.orchestra.run.smithi033.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-23T18:14:14.127 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-23T18:14:14.127 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-23T18:14:14.128 INFO:teuthology.orchestra.run.smithi033.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-23T18:14:14.229 INFO:teuthology.orchestra.run.smithi033.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-23T18:14:14.339 INFO:teuthology.orchestra.run.smithi033.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 310 kB/s | 34 kB 00:00 2024-04-23T18:14:14.431 INFO:teuthology.orchestra.run.smithi033.stdout:(29/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 9.0 MB/s | 3.1 MB 00:00 2024-04-23T18:14:14.456 INFO:teuthology.orchestra.run.smithi033.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.2 MB/s | 142 kB 00:00 2024-04-23T18:14:14.840 INFO:teuthology.orchestra.run.smithi033.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 8.7 MB/s | 3.5 MB 00:00 2024-04-23T18:14:14.949 INFO:teuthology.orchestra.run.smithi033.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.2 MB/s | 249 kB 00:00 2024-04-23T18:14:15.018 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-23T18:14:15.018 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-23T18:14:15.394 INFO:teuthology.orchestra.run.smithi033.stdout:(33/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 16 MB/s | 49 MB 00:02 2024-04-23T18:14:15.408 INFO:teuthology.orchestra.run.smithi033.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 107 kB/s | 49 kB 00:00 2024-04-23T18:14:15.425 INFO:teuthology.orchestra.run.smithi033.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 495 kB/s | 16 kB 00:00 2024-04-23T18:14:15.475 INFO:teuthology.orchestra.run.smithi033.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 4.0 MB/s | 273 kB 00:00 2024-04-23T18:14:15.525 INFO:teuthology.orchestra.run.smithi033.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 7.4 MB/s | 762 kB 00:00 2024-04-23T18:14:15.542 INFO:teuthology.orchestra.run.smithi033.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-23T18:14:15.892 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-23T18:14:15.892 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-23T18:14:15.968 INFO:teuthology.orchestra.run.smithi033.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 4.9 MB/s | 7.4 MB 00:01 2024-04-23T18:14:16.276 INFO:teuthology.orchestra.run.smithi033.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 107 kB/s | 80 kB 00:00 2024-04-23T18:14:16.826 INFO:teuthology.orchestra.run.smithi033.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 228 kB/s | 195 kB 00:00 2024-04-23T18:14:17.168 INFO:teuthology.orchestra.run.smithi033.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 284 kB/s | 253 kB 00:00 2024-04-23T18:14:17.243 INFO:teuthology.orchestra.run.smithi033.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 472 kB/s | 803 kB 00:01 2024-04-23T18:14:17.894 INFO:teuthology.orchestra.run.smithi033.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 147 kB/s | 106 kB 00:00 2024-04-23T18:14:18.010 INFO:teuthology.orchestra.run.smithi033.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 176 kB/s | 135 kB 00:00 2024-04-23T18:14:18.661 INFO:teuthology.orchestra.run.smithi033.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-04-23T18:14:18.819 INFO:teuthology.orchestra.run.smithi033.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-04-23T18:14:18.886 INFO:teuthology.orchestra.run.smithi033.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-23T18:14:18.911 INFO:teuthology.orchestra.run.smithi033.stdout:(49/152): python3-urllib3-1.26.5-5.el9.noarch.r 238 kB/s | 215 kB 00:00 2024-04-23T18:14:18.936 INFO:teuthology.orchestra.run.smithi033.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-23T18:14:18.979 INFO:teuthology.orchestra.run.smithi033.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-23T18:14:19.004 INFO:teuthology.orchestra.run.smithi033.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-23T18:14:19.037 INFO:teuthology.orchestra.run.smithi033.stdout:(53/152): python3-cryptography-36.0.1-4.el9.x86 568 kB/s | 1.2 MB 00:02 2024-04-23T18:14:19.062 INFO:teuthology.orchestra.run.smithi033.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 391 kB/s | 33 kB 00:00 2024-04-23T18:14:19.105 INFO:teuthology.orchestra.run.smithi033.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-04-23T18:14:19.146 INFO:teuthology.orchestra.run.smithi033.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-23T18:14:19.172 INFO:teuthology.orchestra.run.smithi033.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-23T18:14:19.214 INFO:teuthology.orchestra.run.smithi033.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 674 kB/s | 45 kB 00:00 2024-04-23T18:14:19.248 INFO:teuthology.orchestra.run.smithi033.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-04-23T18:14:19.298 INFO:teuthology.orchestra.run.smithi033.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-04-23T18:14:19.324 INFO:teuthology.orchestra.run.smithi033.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-23T18:14:19.349 INFO:teuthology.orchestra.run.smithi033.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-23T18:14:19.374 INFO:teuthology.orchestra.run.smithi033.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-23T18:14:19.407 INFO:teuthology.orchestra.run.smithi033.stdout:(64/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-23T18:14:19.433 INFO:teuthology.orchestra.run.smithi033.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-23T18:14:19.458 INFO:teuthology.orchestra.run.smithi033.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-23T18:14:19.550 INFO:teuthology.orchestra.run.smithi033.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-23T18:14:19.592 INFO:teuthology.orchestra.run.smithi033.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-23T18:14:19.667 INFO:teuthology.orchestra.run.smithi033.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-23T18:14:19.701 INFO:teuthology.orchestra.run.smithi033.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-04-23T18:14:19.734 INFO:teuthology.orchestra.run.smithi033.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 938 kB/s | 133 kB 00:00 2024-04-23T18:14:19.776 INFO:teuthology.orchestra.run.smithi033.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-23T18:14:19.801 INFO:teuthology.orchestra.run.smithi033.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-23T18:14:19.843 INFO:teuthology.orchestra.run.smithi033.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-23T18:14:19.935 INFO:teuthology.orchestra.run.smithi033.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-23T18:14:19.960 INFO:teuthology.orchestra.run.smithi033.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-23T18:14:19.986 INFO:teuthology.orchestra.run.smithi033.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-04-23T18:14:20.011 INFO:teuthology.orchestra.run.smithi033.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-23T18:14:20.036 INFO:teuthology.orchestra.run.smithi033.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-23T18:14:20.062 INFO:teuthology.orchestra.run.smithi033.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-23T18:14:20.145 INFO:teuthology.orchestra.run.smithi033.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-23T18:14:20.170 INFO:teuthology.orchestra.run.smithi033.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-23T18:14:20.196 INFO:teuthology.orchestra.run.smithi033.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-23T18:14:20.212 INFO:teuthology.orchestra.run.smithi033.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-23T18:14:20.238 INFO:teuthology.orchestra.run.smithi033.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-23T18:14:20.263 INFO:teuthology.orchestra.run.smithi033.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-23T18:14:20.288 INFO:teuthology.orchestra.run.smithi033.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-23T18:14:20.313 INFO:teuthology.orchestra.run.smithi033.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-23T18:14:20.330 INFO:teuthology.orchestra.run.smithi033.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-23T18:14:20.356 INFO:teuthology.orchestra.run.smithi033.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-04-23T18:14:20.440 INFO:teuthology.orchestra.run.smithi033.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-04-23T18:14:20.465 INFO:teuthology.orchestra.run.smithi033.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-04-23T18:14:20.490 INFO:teuthology.orchestra.run.smithi033.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-04-23T18:14:20.515 INFO:teuthology.orchestra.run.smithi033.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-23T18:14:20.541 INFO:teuthology.orchestra.run.smithi033.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-04-23T18:14:20.566 INFO:teuthology.orchestra.run.smithi033.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-23T18:14:20.591 INFO:teuthology.orchestra.run.smithi033.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-23T18:14:20.617 INFO:teuthology.orchestra.run.smithi033.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-23T18:14:20.642 INFO:teuthology.orchestra.run.smithi033.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-23T18:14:20.667 INFO:teuthology.orchestra.run.smithi033.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-23T18:14:20.692 INFO:teuthology.orchestra.run.smithi033.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-23T18:14:20.793 INFO:teuthology.orchestra.run.smithi033.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-04-23T18:14:20.951 INFO:teuthology.orchestra.run.smithi033.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-04-23T18:14:21.013 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-23T18:14:21.043 INFO:teuthology.orchestra.run.smithi033.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-04-23T18:14:21.106 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:14:21.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:14:21.168 INFO:teuthology.orchestra.run.smithi033.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-04-23T18:14:21.202 INFO:teuthology.orchestra.run.smithi033.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 122 kB/s | 19 kB 00:00 2024-04-23T18:14:21.260 INFO:teuthology.orchestra.run.smithi033.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-23T18:14:21.281 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:14:21.319 INFO:teuthology.orchestra.run.smithi033.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 835 kB/s | 49 kB 00:00 2024-04-23T18:14:21.323 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:14:21.349 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:14:21.361 INFO:teuthology.orchestra.run.smithi033.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-04-23T18:14:21.378 INFO:teuthology.orchestra.run.smithi033.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-23T18:14:21.391 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:14:21.470 INFO:teuthology.orchestra.run.smithi033.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-23T18:14:21.537 INFO:teuthology.orchestra.run.smithi033.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 861 kB/s | 151 kB 00:00 2024-04-23T18:14:21.595 INFO:teuthology.orchestra.run.smithi033.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-23T18:14:21.620 INFO:teuthology.orchestra.run.smithi033.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-04-23T18:14:21.646 INFO:teuthology.orchestra.run.smithi033.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 3.0 MB/s | 548 kB 00:00 2024-04-23T18:14:21.671 INFO:teuthology.orchestra.run.smithi033.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 427 kB/s | 32 kB 00:00 2024-04-23T18:14:21.705 INFO:teuthology.orchestra.run.smithi033.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-23T18:14:21.710 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:14:21.730 INFO:teuthology.orchestra.run.smithi033.stdout:(118/152): python3-certifi-2021.10.8-2.el9.noar 137 kB/s | 15 kB 00:00 2024-04-23T18:14:21.772 INFO:teuthology.orchestra.run.smithi033.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-23T18:14:21.797 INFO:teuthology.orchestra.run.smithi033.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-23T18:14:21.823 INFO:teuthology.orchestra.run.smithi033.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 382 kB/s | 35 kB 00:00 2024-04-23T18:14:21.825 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:21.915 INFO:teuthology.orchestra.run.smithi033.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.4 MB/s | 206 kB 00:00 2024-04-23T18:14:21.930 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-23T18:14:21.959 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:22.007 INFO:teuthology.orchestra.run.smithi033.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-23T18:14:22.032 INFO:teuthology.orchestra.run.smithi033.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-23T18:14:22.050 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:22.080 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:14:22.082 INFO:teuthology.orchestra.run.smithi033.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-04-23T18:14:22.097 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-23T18:14:22.107 INFO:teuthology.orchestra.run.smithi033.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-23T18:14:22.141 INFO:teuthology.orchestra.run.smithi033.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-23T18:14:22.166 INFO:teuthology.orchestra.run.smithi033.stdout:(129/152): python3-jaraco-functools-3.5.0-2.el9 145 kB/s | 19 kB 00:00 2024-04-23T18:14:22.186 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:14:22.200 INFO:teuthology.orchestra.run.smithi033.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-04-23T18:14:22.224 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:14:22.227 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:14:22.233 INFO:teuthology.orchestra.run.smithi033.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-23T18:14:22.259 INFO:teuthology.orchestra.run.smithi033.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-23T18:14:22.277 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:14:22.278 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:14:22.306 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:22.320 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:14:22.346 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:14:22.367 INFO:teuthology.orchestra.run.smithi033.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-23T18:14:22.388 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:14:22.392 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:22.415 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:22.426 INFO:teuthology.orchestra.run.smithi033.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-23T18:14:22.460 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:22.482 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:22.518 INFO:teuthology.orchestra.run.smithi033.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 308 kB/s | 28 kB 00:00 2024-04-23T18:14:22.546 INFO:teuthology.orchestra.run.smithi033.stdout:(136/152): python3-more-itertools-8.12.0-2.el9. 209 kB/s | 79 kB 00:00 2024-04-23T18:14:22.569 INFO:teuthology.orchestra.run.smithi033.stdout:(137/152): python3-portend-3.1.0-2.el9.noarch.r 53 kB/s | 16 kB 00:00 2024-04-23T18:14:22.619 INFO:teuthology.orchestra.run.smithi033.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-23T18:14:22.644 INFO:teuthology.orchestra.run.smithi033.stdout:(139/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-04-23T18:14:22.698 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:22.702 INFO:teuthology.orchestra.run.smithi033.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-23T18:14:22.715 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:14:22.719 INFO:teuthology.orchestra.run.smithi033.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-23T18:14:22.732 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:22.736 INFO:teuthology.orchestra.run.smithi033.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-04-23T18:14:22.795 INFO:teuthology.orchestra.run.smithi033.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-23T18:14:22.830 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:22.838 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:14:22.845 INFO:teuthology.orchestra.run.smithi033.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-23T18:14:22.879 INFO:teuthology.orchestra.run.smithi033.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-23T18:14:22.904 INFO:teuthology.orchestra.run.smithi033.stdout:(146/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-04-23T18:14:22.932 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:14:22.946 INFO:teuthology.orchestra.run.smithi033.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-23T18:14:22.955 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:22.974 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:14:22.982 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:23.008 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:14:23.046 INFO:teuthology.orchestra.run.smithi033.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-23T18:14:23.071 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:23.082 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:14:23.111 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:14:23.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:14:23.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:14:23.192 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:14:23.220 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:14:23.254 INFO:teuthology.orchestra.run.smithi033.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 10 MB/s | 3.1 MB 00:00 2024-04-23T18:14:23.256 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:14:23.292 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:14:23.316 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:14:23.345 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:23.350 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:14:23.381 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:14:23.425 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:14:23.430 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:23.454 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:23.510 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:14:23.516 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:23.539 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:23.544 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:14:23.559 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:23.587 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:14:23.588 INFO:teuthology.orchestra.run.smithi033.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.0 MB/s | 3.4 MB 00:00 2024-04-23T18:14:23.616 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:14:23.676 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:14:23.722 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:14:23.747 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:23.780 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:23.796 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:14:23.899 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:14:23.950 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:14:23.992 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:14:23.999 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:14:24.030 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:14:24.066 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:14:24.138 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:14:24.173 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:14:24.217 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:14:24.249 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:14:24.286 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:14:24.355 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:14:24.408 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:14:24.439 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:14:24.483 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:14:24.520 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:14:24.559 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:14:24.593 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:14:24.602 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:14:24.633 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:14:24.637 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:14:24.665 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:14:24.675 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:14:24.713 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:14:24.735 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:14:24.754 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:14:24.781 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:14:24.856 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:14:24.911 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:14:24.974 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:14:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:14:25.028 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:14:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:14:25.094 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:14:25.125 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:14:25.161 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:14:25.209 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:14:25.244 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:14:25.288 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:14:25.330 INFO:teuthology.orchestra.run.smithi033.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 605 kB/s | 1.3 MB 00:02 2024-04-23T18:14:25.338 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:14:25.474 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:14:25.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:14:25.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:14:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:14:25.590 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:14:25.645 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:14:25.683 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:14:25.684 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:14:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:14:25.732 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:14:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:14:25.772 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:14:25.798 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:14:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:14:25.822 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:14:25.896 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:14:25.970 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:14:26.023 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:14:26.034 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:14:26.063 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:14:26.088 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:14:26.090 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:14:26.125 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:14:26.153 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:14:26.164 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:14:26.194 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:14:26.196 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:14:26.226 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:14:26.230 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:14:26.260 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:14:26.276 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:14:26.293 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:14:26.313 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:14:26.335 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:14:26.355 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:14:26.369 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:14:26.398 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:14:26.455 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:14:26.500 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:14:26.531 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:14:26.532 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:14:26.578 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:14:26.579 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:14:26.610 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:14:26.651 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:14:26.676 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:14:26.724 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:14:26.743 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:14:26.785 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:14:26.825 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:14:26.830 INFO:teuthology.orchestra.run.smithi033.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 686 kB/s | 2.4 MB 00:03 2024-04-23T18:14:26.833 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:14:26.833 INFO:teuthology.orchestra.run.smithi033.stdout:Total 12 MB/s | 222 MB 00:17 2024-04-23T18:14:26.850 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:14:26.875 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:14:26.949 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:14:27.076 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:14:27.117 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:14:27.145 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:14:27.179 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:14:27.218 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:14:27.250 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:14:27.280 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:14:27.314 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:14:27.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:14:27.389 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:14:27.431 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:14:27.433 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:14:27.480 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:14:27.513 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:14:27.528 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:14:27.580 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:14:27.610 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:14:27.611 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:14:27.658 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:14:27.663 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:14:27.752 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:14:27.764 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:14:27.807 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:14:27.812 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:14:27.950 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-23T18:14:27.982 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:14:28.013 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:14:28.090 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:14:28.112 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-23T18:14:28.112 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-23T18:14:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:14:28.569 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:14:28.603 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:14:28.700 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:14:28.718 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:14:28.760 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:14:28.827 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:14:28.839 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:14:28.872 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:14:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:14:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:14:29.172 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:14:29.809 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:14:29.926 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:14:29.989 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-23T18:14:29.990 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-23T18:14:30.031 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:14:30.126 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:14:30.154 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:14:30.203 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:14:30.514 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:14:30.566 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:30.656 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:30.688 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:14:30.724 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:14:31.138 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:14:31.200 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:14:31.228 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:31.235 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:14:31.263 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:14:31.296 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:14:31.298 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:31.329 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:14:31.605 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:14:31.676 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:31.766 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:31.798 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:14:31.834 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:14:32.310 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:14:32.338 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:32.409 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:32.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:14:33.592 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:33.626 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:34.169 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:34.213 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:34.295 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:34.296 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:14:34.296 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:14:34.296 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:34.322 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:14:34.665 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:34.704 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:35.251 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:35.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:35.383 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:35.383 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:14:35.383 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:14:35.383 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:35.417 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:14:36.016 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-23T18:14:36.101 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-23T18:14:36.139 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-23T18:14:36.193 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-23T18:14:36.235 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-23T18:14:36.260 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-23T18:14:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-23T18:14:36.649 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-23T18:14:36.760 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:36.885 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:14:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:37.012 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:14:37.050 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:14:37.211 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-23T18:14:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-23T18:14:37.291 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:37.396 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:14:37.425 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:37.487 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-23T18:14:37.509 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:37.537 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-23T18:14:37.719 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:37.750 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-23T18:14:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-23T18:14:37.958 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-23T18:14:37.991 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-23T18:14:38.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-23T18:14:38.102 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-23T18:14:38.150 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-23T18:14:38.194 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-23T18:14:38.226 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-23T18:14:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-23T18:14:38.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-23T18:14:38.378 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-23T18:14:38.406 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-23T18:14:38.442 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-23T18:14:38.521 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-23T18:14:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-23T18:14:38.604 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-23T18:14:38.641 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-23T18:14:38.715 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-23T18:14:38.764 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-23T18:14:38.842 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-23T18:14:38.991 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-23T18:14:39.060 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-23T18:14:39.625 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-23T18:14:39.707 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-23T18:14:39.741 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-23T18:14:39.783 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-23T18:14:39.822 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-23T18:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-23T18:14:40.019 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-23T18:14:40.075 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-23T18:14:40.131 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-23T18:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-23T18:14:40.241 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-23T18:14:40.278 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-23T18:14:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-23T18:14:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-23T18:14:40.405 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-23T18:14:40.447 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-23T18:14:40.592 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-23T18:14:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-23T18:14:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-23T18:14:40.706 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-23T18:14:40.805 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-23T18:14:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-23T18:14:40.888 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-23T18:14:40.913 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-23T18:14:40.938 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-23T18:14:41.017 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-23T18:14:41.146 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-23T18:14:41.178 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-23T18:14:41.291 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-23T18:14:41.341 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-23T18:14:41.380 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-23T18:14:41.420 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-23T18:14:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-23T18:14:41.476 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-23T18:14:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-23T18:14:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-23T18:14:41.585 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-23T18:14:41.678 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-23T18:14:41.725 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-23T18:14:41.755 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-23T18:14:41.804 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-23T18:14:41.907 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-23T18:14:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-23T18:14:42.756 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-23T18:14:42.795 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-23T18:14:42.829 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-23T18:14:42.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-23T18:14:42.996 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-23T18:14:43.087 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-23T18:14:43.122 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-23T18:14:43.307 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-23T18:14:43.503 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-23T18:14:43.583 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-23T18:14:44.277 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-23T18:14:44.398 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-23T18:14:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-23T18:14:45.782 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-23T18:14:45.810 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-23T18:14:45.843 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-23T18:14:45.942 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:14:45.942 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-23T18:14:45.943 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-23T18:14:45.944 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:46.110 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:14:46.124 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:14:46.124 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:46.124 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:14:46.124 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:46.171 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-23T18:14:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:46.305 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-23T18:14:46.335 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-23T18:14:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-23T18:14:46.841 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-04-23T18:14:46.842 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:46.877 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-23T18:14:46.900 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:46.971 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-23T18:14:46.993 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-23T18:14:47.018 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:14:47.035 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:14:47.035 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:47.035 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:14:47.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:47.596 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:14:47.621 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:14:47.869 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:48.306 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:14:48.327 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:14:48.327 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:48.328 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:14:48.328 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:14:48.328 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:14:48.328 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:48.427 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:14:48.475 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:14:48.513 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:14:48.530 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:14:48.538 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:14:48.539 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:48.539 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:14:48.539 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:14:48.539 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:14:48.539 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:48.713 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:14:48.756 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:14:48.772 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:14:48.795 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:48.825 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:14:48.913 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:14:49.043 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:14:49.081 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:14:49.113 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:14:49.141 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:14:49.240 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:14:49.262 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:14:49.262 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:49.263 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:14:49.263 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:14:49.263 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:14:49.263 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:49.354 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:49.370 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:14:49.391 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:49.418 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:14:49.473 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:14:49.593 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:14:49.623 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:14:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:14:49.658 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:14:49.716 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:14:49.716 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:14:49.755 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:14:49.802 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:14:49.891 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:14:49.963 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-23T18:14:50.011 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:50.022 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:14:50.058 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:14:50.090 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:14:50.103 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-23T18:14:50.104 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-23T18:14:50.104 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-23T18:14:50.104 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:14:50.118 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:14:50.137 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:14:50.580 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:14:50.609 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:14:50.649 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:14:50.711 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:14:50.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:14:50.779 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:14:50.815 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:14:51.767 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:14:51.771 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:14:51.799 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:14:51.801 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:14:51.899 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:14:52.062 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:14:52.092 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:14:52.115 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:14:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:14:52.267 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:14:52.323 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:14:52.757 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:14:52.785 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:14:52.885 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:14:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:14:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:14:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:14:53.111 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:53.111 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:14:53.111 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:14:53.111 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:14:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:53.154 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:14:53.166 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:14:53.242 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:14:53.262 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:14:53.270 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:14:53.294 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:53.311 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:14:53.338 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:14:53.359 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:14:53.359 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:53.360 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:14:53.360 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:53.405 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:14:53.427 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:53.723 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:14:53.744 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:14:53.744 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:53.744 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:14:53.745 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:14:53.745 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:14:53.745 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:14:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:14:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:14:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:14:54.288 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:54.288 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:14:54.288 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:14:54.289 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:14:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:54.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:14:54.355 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:14:54.356 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:54.356 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:14:54.356 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:54.400 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:14:54.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:54.726 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:14:54.749 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:14:58.376 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:14:58.417 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:14:58.457 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:14:58.490 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:14:58.517 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:14:58.696 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:14:58.755 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:14:58.785 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:14:59.330 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:14:59.372 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:14:59.385 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:14:59.412 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:14:59.425 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:14:59.444 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:14:59.453 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:14:59.505 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:14:59.689 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:14:59.727 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:14:59.757 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:15:00.260 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:15:00.369 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:00.379 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:15:00.405 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:00.420 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:15:00.426 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:00.510 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:15:00.542 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:15:00.542 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:00.626 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:00.651 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:00.651 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:00.734 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:00.799 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:01.257 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:15:01.353 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:15:01.386 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:15:01.478 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:15:01.516 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:15:01.517 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:01.592 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:01.625 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:01.625 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:01.715 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:01.781 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:02.173 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-23T18:15:02.173 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-04-23T18:15:02.174 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:02.347 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:15:02.363 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-23T18:15:02.364 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:02.364 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-23T18:15:02.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:03.892 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-23T18:15:03.919 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:15:04.174 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-23T18:15:04.174 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:04.175 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-23T18:15:04.175 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:15:04.175 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-23T18:15:04.175 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:15:04.658 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-23T18:15:04.658 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:04.658 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-23T18:15:04.658 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:15:04.658 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-23T18:15:04.659 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:04.784 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-23T18:15:04.841 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-23T18:15:04.914 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-23T18:15:05.110 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-23T18:15:05.161 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-23T18:15:05.243 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-23T18:15:05.332 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:15:05.485 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:15:05.528 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-23T18:15:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-23T18:15:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-23T18:15:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:15:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-23T18:15:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-23T18:15:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-23T18:15:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:15:06.494 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:15:06.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:15:06.496 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:15:06.497 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:15:06.498 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:15:06.499 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:15:06.500 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:15:06.501 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:15:06.502 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:15:06.503 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:15:06.504 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:06.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:15:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-23T18:15:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:15:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:15:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:15:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:15:07.952 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:15:07.953 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:15:07.954 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:15:07.955 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:15:07.956 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:15:07.957 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:15:07.958 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:15:07.959 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:15:07.960 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:15:07.961 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:15:07.962 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:15:07.963 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:07.964 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:15:08.351 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-23T18:15:08.377 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:15:08.488 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:08.580 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.581 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.582 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:15:08.583 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:15:08.584 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:08.585 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.586 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:15:08.587 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:08.588 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.589 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:15:08.590 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:15:08.591 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:15:08.592 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:15:08.592 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:15:08.592 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-23T18:15:08.592 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-23T18:15:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-23T18:15:08.696 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-23T18:15:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:08.772 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:15:08.783 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-23T18:15:08.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-23T18:15:08.938 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-23T18:15:08.985 DEBUG:teuthology.parallel:result is None 2024-04-23T18:15:09.912 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-23T18:15:09.940 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:15:09.964 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-23T18:15:09.964 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:09.964 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-23T18:15:09.965 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:15:09.965 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-23T18:15:09.965 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:10.008 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:15:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-23T18:15:10.031 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:10.031 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-23T18:15:10.031 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:10.047 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:10.047 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:15:10.047 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-04-23T18:15:10.047 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.048 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.049 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:15:10.050 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.051 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:15:10.052 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:15:10.053 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:15:10.054 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:15:10.055 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:15:10.056 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:15:10.057 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.058 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:15:10.059 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:15:10.060 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:15:10.061 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-23T18:15:10.075 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:15:10.099 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-23T18:15:10.099 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:10.099 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-23T18:15:10.099 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:15:10.100 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-23T18:15:10.100 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:10.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-23T18:15:10.442 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:10.461 DEBUG:teuthology.parallel:result is None 2024-04-23T18:15:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-23T18:15:15.338 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-23T18:15:15.380 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-23T18:15:15.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-23T18:15:15.437 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-23T18:15:15.639 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-23T18:15:15.692 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-23T18:15:15.721 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-23T18:15:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:16.460 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:16.481 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-23T18:15:17.395 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-23T18:15:17.498 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-23T18:15:17.541 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-23T18:15:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-23T18:15:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-23T18:15:17.669 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-23T18:15:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:17.778 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:17.873 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-23T18:15:17.945 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-23T18:15:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-23T18:15:23.891 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-23T18:15:23.892 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-23T18:15:23.893 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-23T18:15:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-23T18:15:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-23T18:15:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-23T18:15:23.907 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-23T18:15:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-23T18:15:23.909 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-23T18:15:23.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-23T18:15:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-23T18:15:23.912 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-23T18:15:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-23T18:15:23.914 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-23T18:15:23.915 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-23T18:15:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-23T18:15:23.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-23T18:15:26.025 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-23T18:15:26.025 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:26.025 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-04-23T18:15:26.025 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.026 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-23T18:15:26.028 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-23T18:15:26.029 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.14-1.el9.x86_64 2024-04-23T18:15:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-23T18:15:26.031 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-23T18:15:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-23T18:15:26.033 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-23T18:15:26.034 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-23T18:15:26.035 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-23T18:15:26.036 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-23T18:15:26.037 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-23T18:15:26.037 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:15:26.037 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-23T18:15:26.440 DEBUG:teuthology.parallel:result is None 2024-04-23T18:15:26.440 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:26.588 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:15:26.611 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:26.612 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:26.612 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:15:26.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:26.831 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:15:26.857 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:26.857 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:26.857 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:15:26.859 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:26.992 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-23T18:15:27.017 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:27.017 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-23T18:15:27.017 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-23T18:15:27.019 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-23T18:15:27.019 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:15:27.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:15:27.046 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:15:27.046 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:15:27.078 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:15:27.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-23T18:15:27.109 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-23T18:15:27.110 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:15:27.110 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:15:27.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:15:27.200 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:15:27.200 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:15:27.228 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-23T18:15:27.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-23T18:15:27.391 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-23T18:15:27.392 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:15:27.392 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:15:27.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:15:27.482 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:15:27.483 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:15:27.507 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:15:27.570 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:15:27.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-23T18:15:27.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-23T18:15:27.659 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-23T18:15:27.659 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:15:27.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:15:27.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:15:27.751 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:15:27.751 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:15:27.775 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:15:27.839 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:15:27.839 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-23T18:15:27.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-23T18:15:27.926 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-23T18:15:28.023 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2}} 2024-04-23T18:15:28.023 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:28.024 INFO:tasks.cephadm:Cluster fsid is 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:15:28.024 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-23T18:15:28.024 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.122'} 2024-04-23T18:15:28.024 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-04-23T18:15:28.024 INFO:tasks.cephadm:First mgr is x 2024-04-23T18:15:28.024 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-23T18:15:28.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-04-23T18:15:28.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-04-23T18:15:28.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-04-23T18:15:28.096 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-23T18:15:28.096 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:28.233 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-23T18:15:28.368 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:15:28.370 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-23T18:15:28.370 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:29.879 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:15 /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:29.880 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:31.423 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:15 /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:31.424 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:33.293 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 23 18:15 /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:33.294 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:33.313 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:33.332 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-23T18:15:33.367 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-23T18:15:33.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:15:33.370 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:15:33.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-23T18:15:34.804 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:15:34.887 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:15:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:16:02.770 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T18:16:02.771 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-23T18:16:04.823 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-23T18:16:08.221 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-23T18:16:08.221 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-23T18:16:08.221 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-23T18:16:08.222 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-04-23T18:16:08.222 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-23T18:16:08.222 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-23T18:16:08.222 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-23T18:16:08.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-04-23T18:16:08.279 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-04-23T18:16:08.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-04-23T18:16:08.336 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-04-23T18:16:08.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-04-23T18:16:08.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-04-23T18:16:08.415 INFO:tasks.cephadm:Writing seed config... 2024-04-23T18:16:08.416 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-23T18:16:08.416 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-23T18:16:08.416 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-23T18:16:08.417 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-23T18:16:08.418 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-23T18:16:08.419 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-23T18:16:08.420 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-23T18:16:08.421 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:16:08.421 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-23T18:16:08.440 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 766ecab6-019d-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-23T18:16:08.440 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service 2024-04-23T18:16:08.482 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service 2024-04-23T18:16:08.525 INFO:tasks.cephadm:Bootstrapping... 2024-04-23T18:16:08.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:16:08.728 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-23T18:16:08.728 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '766ecab6-019d-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-04-23T18:16:08.729 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-23T18:16:08.729 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-04-23T18:16:08.756 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:16:08.756 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-04-23T18:16:08.757 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-04-23T18:16:08.764 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:16:08.764 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:16:08.770 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:16:08.770 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:16:08.777 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-23T18:16:08.783 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-23T18:16:08.783 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:16:08.783 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-04-23T18:16:08.807 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-23T18:16:08.808 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-23T18:16:08.808 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-04-23T18:16:08.808 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-04-23T18:16:08.814 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-23T18:16:08.814 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-23T18:16:08.820 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-23T18:16:08.820 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:16:08.827 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-23T18:16:08.833 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-23T18:16:08.833 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-23T18:16:08.833 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-04-23T18:16:08.833 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:08.833 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140077381106848 on /run/cephadm/766ecab6-019d-11ef-bc93-c7b262605968.lock 2024-04-23T18:16:08.834 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140077381106848 acquired on /run/cephadm/766ecab6-019d-11ef-bc93-c7b262605968.lock 2024-04-23T18:16:08.834 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-04-23T18:16:08.834 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-04-23T18:16:08.834 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-23T18:16:08.837 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-04-23T18:16:08.837 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-04-23T18:16:08.840 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-23T18:16:08.840 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-23T18:16:08.840 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-23T18:16:08.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-23T18:16:08.842 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-23T18:16:08.843 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-23T18:16:08.844 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-23T18:16:08.844 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-23T18:16:08.845 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-23T18:16:09.247 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-23T18:16:10.227 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:16:10.227 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:16:10.227 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-04-23T18:16:11.211 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-04-23T18:16:11.211 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-04-23T18:16:12.189 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDr+idm3BeuIBAAUGPn7qPteTQf+oZ39JwGrw== 2024-04-23T18:16:13.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDs+idmiwMRIBAA9dHD1IFxfhLJzIxVeUeKNA== 2024-04-23T18:16:19.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDt+idmqcu1HxAABnerAoTI7R92zqoJjqNltA== 2024-04-23T18:16:19.867 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-04-23T18:16:23.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:16:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-23T18:16:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:16:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-23T18:16:23.688 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-04-23T18:16:23.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:23.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-23T18:16:23.689 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:16:23.689 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-04-23T18:16:24.617 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-04-23T18:16:24.821 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-23T18:16:24.970 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-23T18:16:25.145 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target. 2024-04-23T18:16:25.145 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target. 2024-04-23T18:16:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a 2024-04-23T18:16:26.247 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-23T18:16:26.501 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target.wants/ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968@.service. 2024-04-23T18:16:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:26 smithi033 podman[28878]: 2024-04-23 18:16:26.654694672 +0000 UTC m=+0.017955058 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:16:27.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:26 smithi033 podman[28878]: 2024-04-23 18:16:26.991359482 +0000 UTC m=+0.354619868 container create d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-23T18:16:27.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 podman[28878]: 2024-04-23 18:16:27.202021208 +0000 UTC m=+0.565281595 container init d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:16:27.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 podman[28878]: 2024-04-23 18:16:27.205450194 +0000 UTC m=+0.568710581 container start d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T18:16:27.301 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:16:27.301 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-23T18:16:27.308 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:16:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:16:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-23T18:16:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:16:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-04-23T18:16:27.309 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-04-23T18:16:27.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 ceph-mon[28897]: mkfs 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:27.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 ceph-mon[28897]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:16:27.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 bash[28878]: d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 2024-04-23T18:16:27.497 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:27 smithi033 systemd[1]: Started Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:16:28.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: monmap epoch 1 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: last_changed 2024-04-23T18:16:22.571495+0000 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: min_mon_release 19 (squid) 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: election_strategy: 1 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: fsmap 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: mgrmap e1: no daemons active 2024-04-23T18:16:28.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:28 smithi033 ceph-mon[28897]: from='client.? 172.21.15.33:0/1364583848' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.547318s) 2024-04-23T18:16:29.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.713+0000 7f78292a1700 1 Processor -- start 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.713+0000 7f78292a1700 1 -- start start 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f78292a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f78292a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f782407be20 con 0x7f7824108fc0 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f7822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:29.499 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f7822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53874/0 (socket says 172.21.15.33:53874) 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f7822ffd700 1 -- 172.21.15.33:0/2652505860 learned_addr learned my addr 172.21.15.33:0/2652505860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.714+0000 7f7822ffd700 1 -- 172.21.15.33:0/2652505860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782410b8f0 con 0x7f7824108fc0 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f7822ffd700 1 --2- 172.21.15.33:0/2652505860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78140098a0 tx=0x7f7814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5bebe55249b317 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f7821ffb700 1 -- 172.21.15.33:0/2652505860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7814004160 con 0x7f7824108fc0 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f7821ffb700 1 -- 172.21.15.33:0/2652505860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814004300 con 0x7f7824108fc0 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f7821ffb700 1 -- 172.21.15.33:0/2652505860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78140036a0 con 0x7f7824108fc0 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f78292a1700 1 -- 172.21.15.33:0/2652505860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 msgr2=0x7f782410b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.715+0000 7f78292a1700 1 --2- 172.21.15.33:0/2652505860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f78140098a0 tx=0x7f7814009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 -- 172.21.15.33:0/2652505860 shutdown_connections 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 --2- 172.21.15.33:0/2652505860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782410b3b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 -- 172.21.15.33:0/2652505860 >> 172.21.15.33:0/2652505860 conn(0x7f7824102970 msgr2=0x7f7824104d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 -- 172.21.15.33:0/2652505860 shutdown_connections 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 -- 172.21.15.33:0/2652505860 wait complete. 2024-04-23T18:16:29.500 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 Processor -- start 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.716+0000 7f78292a1700 1 -- start start 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f78292a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f78292a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f782410bd00 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f7822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f7822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53890/0 (socket says 172.21.15.33:53890) 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f7822ffd700 1 -- 172.21.15.33:0/1364583848 learned_addr learned my addr 172.21.15.33:0/1364583848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f7822ffd700 1 -- 172.21.15.33:0/1364583848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f782419c170 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f7822ffd700 1 --2- 172.21.15.33:0/1364583848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7814000c00 tx=0x7f7814004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7814003db0 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7814011630 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f782419c400 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7814011950 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.717+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f782419f0f0 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.718+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f781401a460 con 0x7f7824108fc0 2024-04-23T18:16:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.718+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f781401bb40 con 0x7f7824108fc0 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.718+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240640a0 con 0x7f7824108fc0 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.721+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7814031020 con 0x7f7824108fc0 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.795+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f782402e070 con 0x7f7824108fc0 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.795+0000 7f780bfff700 1 -- 172.21.15.33:0/1364583848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7814004260 con 0x7f7824108fc0 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 msgr2=0x7f782419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 --2- 172.21.15.33:0/1364583848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7814000c00 tx=0x7f7814004750 comp rx=0 tx=0).stop 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 shutdown_connections 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 --2- 172.21.15.33:0/1364583848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7824108fc0 0x7f782419bc30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 >> 172.21.15.33:0/1364583848 conn(0x7f7824102970 msgr2=0x7f7824104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 shutdown_connections 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:27.796+0000 7f78292a1700 1 -- 172.21.15.33:0/1364583848 wait complete. 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-04-23T18:16:29.502 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-04-23T18:16:30.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:30 smithi033 ceph-mon[28897]: from='client.? 172.21.15.33:0/2290691292' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:16:30.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:30 smithi033 ceph-mon[28897]: from='client.? 172.21.15.33:0/2290691292' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.771+0000 7fe7a39e0700 1 Processor -- start 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a39e0700 1 -- start start 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a39e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a39e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c107d80 con 0x7fe79c107390 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a177c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a177c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53904/0 (socket says 172.21.15.33:53904) 2024-04-23T18:16:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a177c700 1 -- 172.21.15.33:0/1925905052 learned_addr learned my addr 172.21.15.33:0/1925905052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a177c700 1 -- 172.21.15.33:0/1925905052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c108590 con 0x7fe79c107390 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.772+0000 7fe7a177c700 1 --2- 172.21.15.33:0/1925905052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b0e57eaa936026 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe793fff700 1 -- 172.21.15.33:0/1925905052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c004160 con 0x7fe79c107390 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe793fff700 1 -- 172.21.15.33:0/1925905052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c004300 con 0x7fe79c107390 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe793fff700 1 -- 172.21.15.33:0/1925905052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c0036a0 con 0x7fe79c107390 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 -- 172.21.15.33:0/1925905052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 msgr2=0x7fe79c1077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 --2- 172.21.15.33:0/1925905052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe78c0098a0 tx=0x7fe78c009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 -- 172.21.15.33:0/1925905052 shutdown_connections 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 --2- 172.21.15.33:0/1925905052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c1077b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:31.419 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 -- 172.21.15.33:0/1925905052 >> 172.21.15.33:0/1925905052 conn(0x7fe79c1029b0 msgr2=0x7fe79c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 -- 172.21.15.33:0/1925905052 shutdown_connections 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.773+0000 7fe7a39e0700 1 -- 172.21.15.33:0/1925905052 wait complete. 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a39e0700 1 Processor -- start 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a39e0700 1 -- start start 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a39e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a39e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c108ac0 con 0x7fe79c107390 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a177c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a177c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53910/0 (socket says 172.21.15.33:53910) 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.774+0000 7fe7a177c700 1 -- 172.21.15.33:0/2290691292 learned_addr learned my addr 172.21.15.33:0/2290691292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7a177c700 1 -- 172.21.15.33:0/2290691292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c19c1b0 con 0x7fe79c107390 2024-04-23T18:16:31.420 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7a177c700 1 --2- 172.21.15.33:0/2290691292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000c00 tx=0x7fe78c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c003db0 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe78c011630 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c19c440 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe78c011950 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.775+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c19f130 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.776+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe78c01a460 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.776+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe78c01b780 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.776+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c0640a0 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.779+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe78c031020 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.849+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe79c19f300 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.852+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fe78c01b9d0 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.853+0000 7fe7927fc700 1 -- 172.21.15.33:0/2290691292 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe78c01ad70 con 0x7fe79c107390 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 msgr2=0x7fe79c19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 --2- 172.21.15.33:0/2290691292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000c00 tx=0x7fe78c004750 comp rx=0 tx=0).stop 2024-04-23T18:16:31.421 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 shutdown_connections 2024-04-23T18:16:31.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 --2- 172.21.15.33:0/2290691292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe79c107390 0x7fe79c19bc70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:31.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 >> 172.21.15.33:0/2290691292 conn(0x7fe79c1029b0 msgr2=0x7fe79c18ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:31.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 shutdown_connections 2024-04-23T18:16:31.422 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:30.854+0000 7fe7a39e0700 1 -- 172.21.15.33:0/2290691292 wait complete. 2024-04-23T18:16:31.422 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-04-23T18:16:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:31 smithi033 ceph-mon[28897]: from='client.? 172.21.15.33:0/286136877' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:16:33.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bd4e21700 1 Processor -- start 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bd4e21700 1 -- start start 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bd4e21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bd4e21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd0108ec0 con 0x7f3bd01084d0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53916/0 (socket says 172.21.15.33:53916) 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.869+0000 7f3bce59c700 1 -- 172.21.15.33:0/558783520 learned_addr learned my addr 172.21.15.33:0/558783520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.870+0000 7f3bce59c700 1 -- 172.21.15.33:0/558783520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd0109680 con 0x7f3bd01084d0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.870+0000 7f3bce59c700 1 --2- 172.21.15.33:0/558783520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00098a0 tx=0x7f3bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac53a4691d51ff4e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.870+0000 7f3bcd59a700 1 -- 172.21.15.33:0/558783520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bc0004160 con 0x7f3bd01084d0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.870+0000 7f3bcd59a700 1 -- 172.21.15.33:0/558783520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3bc0004300 con 0x7f3bd01084d0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.870+0000 7f3bcd59a700 1 -- 172.21.15.33:0/558783520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bc0011af0 con 0x7f3bd01084d0 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- 172.21.15.33:0/558783520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 msgr2=0x7f3bd01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:33.606 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 --2- 172.21.15.33:0/558783520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00098a0 tx=0x7f3bc0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- 172.21.15.33:0/558783520 shutdown_connections 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 --2- 172.21.15.33:0/558783520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd01088f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- 172.21.15.33:0/558783520 >> 172.21.15.33:0/558783520 conn(0x7f3bd0103b10 msgr2=0x7f3bd0105f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- 172.21.15.33:0/558783520 shutdown_connections 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- 172.21.15.33:0/558783520 wait complete. 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 Processor -- start 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.871+0000 7f3bd4e21700 1 -- start start 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bd4e21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bd4e21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd010ca30 con 0x7f3bd01084d0 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bce59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53924/0 (socket says 172.21.15.33:53924) 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bce59c700 1 -- 172.21.15.33:0/286136877 learned_addr learned my addr 172.21.15.33:0/286136877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:33.607 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bce59c700 1 -- 172.21.15.33:0/286136877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd019c9a0 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bce59c700 1 --2- 172.21.15.33:0/286136877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0000c00 tx=0x7f3bc0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bc0003b20 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3bc001a8c0 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd019cc30 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3bc0024940 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.872+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd019f920 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.873+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3bc0022020 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.873+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3bc002dde0 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.873+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd00640a0 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.876+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3bc00042a0 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.946+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3bd019fe60 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.947+0000 7f3bbf7fe700 1 -- 172.21.15.33:0/286136877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f3bc001aa70 con 0x7f3bd01084d0 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.947+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 msgr2=0x7f3bd019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:33.608 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.947+0000 7f3bd4e21700 1 --2- 172.21.15.33:0/286136877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0000c00 tx=0x7f3bc0011b70 comp rx=0 tx=0).stop 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.948+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 shutdown_connections 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.948+0000 7f3bd4e21700 1 --2- 172.21.15.33:0/286136877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3bd01084d0 0x7f3bd019c460 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.948+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 >> 172.21.15.33:0/286136877 conn(0x7f3bd0103b10 msgr2=0x7f3bd01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.948+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 shutdown_connections 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:31.948+0000 7f3bd4e21700 1 -- 172.21.15.33:0/286136877 wait complete. 2024-04-23T18:16:33.609 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-04-23T18:16:33.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:33 smithi033 systemd[1]: Stopping Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:16:34.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a[28893]: 2024-04-23T18:16:34.073+0000 7feb268b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T18:16:34.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a[28893]: 2024-04-23T18:16:34.073+0000 7feb268b0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-23T18:16:34.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 podman[29146]: 2024-04-23 18:16:34.207288969 +0000 UTC m=+0.414073451 container died d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:16:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 podman[29146]: 2024-04-23 18:16:34.556604601 +0000 UTC m=+0.763389089 container cleanup d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:16:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 bash[29146]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a 2024-04-23T18:16:35.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:34 smithi033 podman[29160]: 2024-04-23 18:16:34.83240511 +0000 UTC m=+0.623694927 container remove d8efeaefebefeca266ca4dcbe6e74e52a3ee3e4420b61688baa8c4e59a43cb52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-23T18:16:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-23T18:16:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 systemd[1]: Stopped Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:16:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 systemd[1]: Starting Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:16:35.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 podman[29255]: 2024-04-23 18:16:35.139223154 +0000 UTC m=+0.018552076 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:16:35.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 podman[29255]: 2024-04-23 18:16:35.258105617 +0000 UTC m=+0.137434518 container create 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:16:35.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 podman[29255]: 2024-04-23 18:16:35.485249891 +0000 UTC m=+0.364578793 container init 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-23T18:16:35.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 podman[29255]: 2024-04-23 18:16:35.488344178 +0000 UTC m=+0.367673079 container start 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T18:16:35.583 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: set uid:gid to 167:167 (ceph:ceph) 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: pidfile_write: ignore empty --pid-file 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: load: jerasure load: lrc 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: RocksDB version: 7.9.2 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Git sha 0 2024-04-23T18:16:35.803 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: DB SUMMARY 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: DB Session ID: KNOZH0HMEYX0HZJGO0PP 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: CURRENT file: CURRENT 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: IDENTITY file: IDENTITY 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105678 ; 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.error_if_exists: 0 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.create_if_missing: 0 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.paranoid_checks: 1 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-23T18:16:35.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.env: 0x55e5f6ebbcc0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.fs: PosixFileSystem 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.info_log: 0x55e5f7c43be0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_file_opening_threads: 16 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.statistics: (nil) 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.use_fsync: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_log_file_size: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.keep_log_file_num: 1000 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.recycle_log_file_num: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_fallocate: 1 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_mmap_reads: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_mmap_writes: 0 2024-04-23T18:16:35.805 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.use_direct_reads: 0 2024-04-23T18:16:35.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-23T18:16:35.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.create_missing_column_families: 0 2024-04-23T18:16:35.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.db_log_dir: 2024-04-23T18:16:35.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.wal_dir: 2024-04-23T18:16:35.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-23T18:16:35.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.advise_random_on_open: 1 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.db_write_buffer_size: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.write_buffer_manager: 0x55e5f7bd70e0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.rate_limiter: (nil) 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.wal_recovery_mode: 2 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enable_thread_tracking: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enable_pipelined_write: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.unordered_write: 0 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-23T18:16:35.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.row_cache: None 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.wal_filter: None 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_ingest_behind: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.two_write_queues: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.manual_wal_flush: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.wal_compression: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.atomic_flush: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.log_readahead_size: 0 2024-04-23T18:16:35.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.best_efforts_recovery: 0 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.allow_data_in_errors: 0 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.db_host_id: __hostname__ 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_background_jobs: 2 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_background_compactions: -1 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_subcompactions: 1 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_total_wal_size: 0 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-23T18:16:35.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_open_files: -1 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bytes_per_sync: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_readahead_size: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_background_flushes: -1 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Compression algorithms supported: 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kZSTD supported: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kXpressCompression supported: 0 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kLZ4HCCompression supported: 1 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kZlibCompression supported: 1 2024-04-23T18:16:35.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kSnappyCompression supported: 1 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kLZ4Compression supported: 1 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: kBZip2Compression supported: 0 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.merge_operator: 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_filter: None 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_filter_factory: None 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.sst_partitioner_factory: None 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5f7c43dc0) 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-04-23T18:16:35.812 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x55e5f7c091f0 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-04-23T18:16:35.813 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-04-23T18:16:35.814 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.write_buffer_size: 33554432 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_write_buffer_number: 2 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression: NoCompression 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression: Disabled 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.prefix_extractor: nullptr 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.num_levels: 7 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-23T18:16:35.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.level: 32767 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.strategy: 0 2024-04-23T18:16:35.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.enabled: false 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.target_file_size_base: 67108864 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-23T18:16:35.817 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.arena_block_size: 1048576 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.disable_auto_compactions: 0 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-23T18:16:35.818 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.inplace_update_support: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.bloom_locality: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.max_successive_merges: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-23T18:16:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.paranoid_file_checks: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.force_consistency_checks: 1 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.report_bg_io_stats: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.ttl: 2592000 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enable_blob_files: false 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.min_blob_size: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_file_size: 268435456 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.blob_file_starting_level: 0 2024-04-23T18:16:35.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b011a063-9f97-451a-9abe-f5bf4c1d97aa 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896195518445, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896195519521, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100785, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98673, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90891, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713896195, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b011a063-9f97-451a-9abe-f5bf4c1d97aa", "db_session_id": "KNOZH0HMEYX0HZJGO0PP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713896195519605, "job": 1, "event": "recovery_finished"} 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5f7d1e000 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: DB pointer 0x55e5f7d08000 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-23T18:16:35.821 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 100.28 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 100.28 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:16:35.822 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 31.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:16:35.823 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 31.05 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x55e5f7c091f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???) e1 preinit fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).mds e1 new map 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).mds e1 print_map 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-04-23T18:16:35.824 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: monmap epoch 1 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: last_changed 2024-04-23T18:16:22.571495+0000 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: min_mon_release 19 (squid) 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: election_strategy: 1 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: fsmap 2024-04-23T18:16:35.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: osdmap e1: 0 total, 0 up, 0 in 2024-04-23T18:16:35.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 ceph-mon[29274]: mgrmap e1: no daemons active 2024-04-23T18:16:35.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 bash[29255]: 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 2024-04-23T18:16:35.826 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:35 smithi033 systemd[1]: Started Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.069+0000 7f3dc72c1700 1 Processor -- start 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc72c1700 1 -- start start 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc72c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc72c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc007c900 con 0x7f3dc007de80 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc505d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc505d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53936/0 (socket says 172.21.15.33:53936) 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.070+0000 7f3dc505d700 1 -- 172.21.15.33:0/1486778570 learned_addr learned my addr 172.21.15.33:0/1486778570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.071+0000 7f3dc505d700 1 -- 172.21.15.33:0/1486778570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc007ca80 con 0x7f3dc007de80 2024-04-23T18:16:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.071+0000 7f3dc505d700 1 --2- 172.21.15.33:0/1486778570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098a0 tx=0x7f3dbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b6100be6da949e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.071+0000 7f3db7fff700 1 -- 172.21.15.33:0/1486778570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dbc004160 con 0x7f3dc007de80 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.071+0000 7f3db7fff700 1 -- 172.21.15.33:0/1486778570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3dbc004300 con 0x7f3dc007de80 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.071+0000 7f3db7fff700 1 -- 172.21.15.33:0/1486778570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dbc011af0 con 0x7f3dc007de80 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1486778570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 msgr2=0x7f3dc007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 --2- 172.21.15.33:0/1486778570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc0098a0 tx=0x7f3dbc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1486778570 shutdown_connections 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 --2- 172.21.15.33:0/1486778570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc007c330 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1486778570 >> 172.21.15.33:0/1486778570 conn(0x7f3dc0103b70 msgr2=0x7f3dc0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1486778570 shutdown_connections 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1486778570 wait complete. 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.072+0000 7f3dc72c1700 1 Processor -- start 2024-04-23T18:16:36.710 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc72c1700 1 -- start start 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc72c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc72c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc010ea40 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc505d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc505d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53946/0 (socket says 172.21.15.33:53946) 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc505d700 1 -- 172.21.15.33:0/1613031524 learned_addr learned my addr 172.21.15.33:0/1613031524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc505d700 1 -- 172.21.15.33:0/1613031524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc019ee10 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc505d700 1 --2- 172.21.15.33:0/1613031524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc011ac0 tx=0x7f3dbc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dbc003b20 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3dbc01a8c0 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc019f0a0 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.073+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dbc0248d0 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.074+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc01a1d90 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.074+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3dbc022020 con 0x7f3dc007de80 2024-04-23T18:16:36.711 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.074+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3dbc0193c0 con 0x7f3dc007de80 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.075+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc00640a0 con 0x7f3dc007de80 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.077+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3dbc042e50 con 0x7f3dc007de80 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.149+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3dc002df80 con 0x7f3dc007de80 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3db67fc700 1 -- 172.21.15.33:0/1613031524 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3dbc031020 con 0x7f3dc007de80 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 msgr2=0x7f3dc019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3dc72c1700 1 --2- 172.21.15.33:0/1613031524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc011ac0 tx=0x7f3dbc011ba0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 shutdown_connections 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3dc72c1700 1 --2- 172.21.15.33:0/1613031524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dc007de80 0x7f3dc019e8d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.151+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 >> 172.21.15.33:0/1613031524 conn(0x7f3dc0103b70 msgr2=0x7f3dc0104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.152+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 shutdown_connections 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:36.152+0000 7f3dc72c1700 1 -- 172.21.15.33:0/1613031524 wait complete. 2024-04-23T18:16:36.712 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-23T18:16:36.734 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:16:36.734 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-04-23T18:16:36.735 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-23T18:16:36.735 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-23T18:16:36.999 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x 2024-04-23T18:16:37.000 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-23T18:16:37.150 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968.target.wants/ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-766ecab6-019d-11ef-bc93-c7b262605968@.service. 2024-04-23T18:16:37.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1613031524' entity='client.admin' 2024-04-23T18:16:37.387 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 systemd[1]: Starting Ceph mgr.x for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:16:37.698 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 podman[29488]: 2024-04-23 18:16:37.393895054 +0000 UTC m=+0.110551851 container create d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T18:16:37.698 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 podman[29488]: 2024-04-23 18:16:37.301630187 +0000 UTC m=+0.018286985 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-23T18:16:37.698 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 podman[29488]: 2024-04-23 18:16:37.629803582 +0000 UTC m=+0.346460378 container init d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-23T18:16:37.698 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 podman[29488]: 2024-04-23 18:16:37.632372396 +0000 UTC m=+0.349029199 container start d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:16:37.698 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 bash[29488]: d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f 2024-04-23T18:16:37.711 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:16:37.711 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-23T18:16:37.719 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:16:37.719 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:16:37.719 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-23T18:16:37.719 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-23T18:16:37.727 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:16:37.727 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-04-23T18:16:37.735 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-04-23T18:16:37.977 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 systemd[1]: Started Ceph mgr.x for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:16:37.978 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:37.793+0000 7f05e1a5e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:16:38.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:37.987+0000 7f05e1a5e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:38.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:38.093+0000 7f05e1a5e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:16:38.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3499448923' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:16:38.596 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:38.428+0000 7f05e1a5e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:16:38.596 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:38.516+0000 7f05e1a5e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:16:38.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:16:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:16:38.852 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:16:38.860 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:16:38.861 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:38.862 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:16:27.250251+0000", 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.202+0000 7ff24d31c700 1 Processor -- start 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.202+0000 7ff24d31c700 1 -- start start 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff24d31c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff24d31c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff248108ed0 con 0x7ff2481084e0 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff246ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff246ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55210/0 (socket says 172.21.15.33:55210) 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff246ffd700 1 -- 172.21.15.33:0/4089112457 learned_addr learned my addr 172.21.15.33:0/4089112457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff246ffd700 1 -- 172.21.15.33:0/4089112457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff248109690 con 0x7ff2481084e0 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.203+0000 7ff246ffd700 1 --2- 172.21.15.33:0/4089112457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098a0 tx=0x7ff238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a5b5276d9be88d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:38.863 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff245ffb700 1 -- 172.21.15.33:0/4089112457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238004160 con 0x7ff2481084e0 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff245ffb700 1 -- 172.21.15.33:0/4089112457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff238004300 con 0x7ff2481084e0 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff245ffb700 1 -- 172.21.15.33:0/4089112457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238011af0 con 0x7ff2481084e0 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff24d31c700 1 -- 172.21.15.33:0/4089112457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 msgr2=0x7ff248108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff24d31c700 1 --2- 172.21.15.33:0/4089112457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098a0 tx=0x7ff238009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff24d31c700 1 -- 172.21.15.33:0/4089112457 shutdown_connections 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff24d31c700 1 --2- 172.21.15.33:0/4089112457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff248108900 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.204+0000 7ff24d31c700 1 -- 172.21.15.33:0/4089112457 >> 172.21.15.33:0/4089112457 conn(0x7ff248103ae0 msgr2=0x7ff248105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 -- 172.21.15.33:0/4089112457 shutdown_connections 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 -- 172.21.15.33:0/4089112457 wait complete. 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 Processor -- start 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 -- start start 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:38.864 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.205+0000 7ff24d31c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff24810ca40 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff246ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff246ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55212/0 (socket says 172.21.15.33:55212) 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff246ffd700 1 -- 172.21.15.33:0/3499448923 learned_addr learned my addr 172.21.15.33:0/3499448923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff246ffd700 1 -- 172.21.15.33:0/3499448923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24819eec0 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff246ffd700 1 --2- 172.21.15.33:0/3499448923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff238009fb0 tx=0x7ff238011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2380036a0 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24819f150 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff23801a430 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff238019550 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.206+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481a1e40 con 0x7ff2481084e0 2024-04-23T18:16:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.207+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff238019be0 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.207+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff238024730 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.207+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2480640a0 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.210+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff238043e50 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.283+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff24802df80 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.284+0000 7ff22ffff700 1 -- 172.21.15.33:0/3499448923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ff238032020 con 0x7ff2481084e0 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 msgr2=0x7ff24819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 --2- 172.21.15.33:0/3499448923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff238009fb0 tx=0x7ff238011940 comp rx=0 tx=0).stop 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 shutdown_connections 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 --2- 172.21.15.33:0/3499448923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2481084e0 0x7ff24819e980 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 >> 172.21.15.33:0/3499448923 conn(0x7ff248103ae0 msgr2=0x7ff248104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 shutdown_connections 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:38.285+0000 7ff24d31c700 1 -- 172.21.15.33:0/3499448923 wait complete. 2024-04-23T18:16:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-04-23T18:16:40.119 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:39.775+0000 7f05e1a5e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:16:40.119 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:39.950+0000 7f05e1a5e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:40.119 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:40.125+0000 7f05e1a5e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:16:40.720 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:40.422+0000 7f05e1a5e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:16:40.721 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:40.517+0000 7f05e1a5e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:16:40.721 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:40.726+0000 7f05e1a5e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:16:40.979 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:40.928+0000 7f05e1a5e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:41.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:41 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2762268180' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:16:41.562 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:41.568+0000 7f05e1a5e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:16:41.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:16:41.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:41.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:16:27.250251+0000", 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7fadef708700 1 Processor -- start 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7fadef708700 1 -- start start 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7fadef708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7fadef708700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fade807be80 con 0x7fade8108f80 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7faded4a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7faded4a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55228/0 (socket says 172.21.15.33:55228) 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.274+0000 7faded4a4700 1 -- 172.21.15.33:0/2266540603 learned_addr learned my addr 172.21.15.33:0/2266540603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:41.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.275+0000 7faded4a4700 1 -- 172.21.15.33:0/2266540603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade810b8b0 con 0x7fade8108f80 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.275+0000 7faded4a4700 1 --2- 172.21.15.33:0/2266540603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fade40098a0 tx=0x7fade4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d360cc93a2ae0672 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.275+0000 7faddffff700 1 -- 172.21.15.33:0/2266540603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade4004160 con 0x7fade8108f80 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.275+0000 7faddffff700 1 -- 172.21.15.33:0/2266540603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fade4004300 con 0x7fade8108f80 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.275+0000 7faddffff700 1 -- 172.21.15.33:0/2266540603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade4011af0 con 0x7fade8108f80 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 -- 172.21.15.33:0/2266540603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 msgr2=0x7fade810b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 --2- 172.21.15.33:0/2266540603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fade40098a0 tx=0x7fade4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 -- 172.21.15.33:0/2266540603 shutdown_connections 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 --2- 172.21.15.33:0/2266540603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade810b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 -- 172.21.15.33:0/2266540603 >> 172.21.15.33:0/2266540603 conn(0x7fade8102a10 msgr2=0x7fade8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 -- 172.21.15.33:0/2266540603 shutdown_connections 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.276+0000 7fadef708700 1 -- 172.21.15.33:0/2266540603 wait complete. 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7fadef708700 1 Processor -- start 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7fadef708700 1 -- start start 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7fadef708700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:41.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7fadef708700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fade810c870 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7faded4a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7faded4a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55244/0 (socket says 172.21.15.33:55244) 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7faded4a4700 1 -- 172.21.15.33:0/2762268180 learned_addr learned my addr 172.21.15.33:0/2762268180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7faded4a4700 1 -- 172.21.15.33:0/2762268180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade819eb80 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.277+0000 7faded4a4700 1 --2- 172.21.15.33:0/2762268180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fade4011ac0 tx=0x7fade4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade4003970 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade819ee10 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fade4003b10 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade4024610 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade81a1b00 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fade4024c20 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.278+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fade4019700 con 0x7fade8108f80 2024-04-23T18:16:41.962 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.279+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade80640a0 con 0x7fade8108f80 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.283+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fade4043e50 con 0x7fade8108f80 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.357+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fade81a1cd0 con 0x7fade8108f80 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.358+0000 7fadde7fc700 1 -- 172.21.15.33:0/2762268180 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fade4032030 con 0x7fade8108f80 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 msgr2=0x7fade819e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 --2- 172.21.15.33:0/2762268180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fade4011ac0 tx=0x7fade4011ba0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 shutdown_connections 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 --2- 172.21.15.33:0/2762268180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fade8108f80 0x7fade819e640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 >> 172.21.15.33:0/2762268180 conn(0x7fade8102a10 msgr2=0x7fade81034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 shutdown_connections 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:41.359+0000 7fadef708700 1 -- 172.21.15.33:0/2762268180 wait complete. 2024-04-23T18:16:41.963 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-04-23T18:16:42.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:42.007+0000 7f05e1a5e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:16:42.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:42.106+0000 7f05e1a5e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:16:42.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:42.314+0000 7f05e1a5e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:16:43.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:42.875+0000 7f05e1a5e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:16:43.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:42.969+0000 7f05e1a5e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:16:43.597 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:43.245+0000 7f05e1a5e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:16:43.597 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:43.384+0000 7f05e1a5e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:16:43.597 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:43.484+0000 7f05e1a5e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:16:43.598 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:43.603+0000 7f05e1a5e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:44.165 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:43.915+0000 7f05e1a5e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:16:44.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:44 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2667830301' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:16:44.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:44.569+0000 7f05e1a5e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:16:45.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:16:45.004 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:16:45.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:16:45.006 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:45.007 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:16:27.250251+0000", 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.360+0000 7fac2c689700 1 Processor -- start 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.360+0000 7fac2c689700 1 -- start start 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2c689700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:45.008 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2c689700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac2407be40 con 0x7fac24106e10 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2a425700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2a425700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55254/0 (socket says 172.21.15.33:55254) 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2a425700 1 -- 172.21.15.33:0/189571845 learned_addr learned my addr 172.21.15.33:0/189571845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2a425700 1 -- 172.21.15.33:0/189571845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac24107770 con 0x7fac24106e10 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac2a425700 1 --2- 172.21.15.33:0/189571845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8184fdf880c924 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.361+0000 7fac29423700 1 -- 172.21.15.33:0/189571845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac14004160 con 0x7fac24106e10 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.362+0000 7fac29423700 1 -- 172.21.15.33:0/189571845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac14004300 con 0x7fac24106e10 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.362+0000 7fac29423700 1 -- 172.21.15.33:0/189571845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac14011af0 con 0x7fac24106e10 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.362+0000 7fac2c689700 1 -- 172.21.15.33:0/189571845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 msgr2=0x7fac24107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.362+0000 7fac2c689700 1 --2- 172.21.15.33:0/189571845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fac140098a0 tx=0x7fac14009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.362+0000 7fac2c689700 1 -- 172.21.15.33:0/189571845 shutdown_connections 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 --2- 172.21.15.33:0/189571845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac24107230 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.009 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 -- 172.21.15.33:0/189571845 >> 172.21.15.33:0/189571845 conn(0x7fac24102a10 msgr2=0x7fac24104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 -- 172.21.15.33:0/189571845 shutdown_connections 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 -- 172.21.15.33:0/189571845 wait complete. 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 Processor -- start 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 -- start start 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.363+0000 7fac2c689700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac24108730 con 0x7fac24106e10 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2a425700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2a425700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55262/0 (socket says 172.21.15.33:55262) 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2a425700 1 -- 172.21.15.33:0/2667830301 learned_addr learned my addr 172.21.15.33:0/2667830301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2a425700 1 -- 172.21.15.33:0/2667830301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2419eb80 con 0x7fac24106e10 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2a425700 1 --2- 172.21.15.33:0/2667830301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac14009fb0 tx=0x7fac14011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac140036a0 con 0x7fac24106e10 2024-04-23T18:16:45.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac2419ee10 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac1401a430 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac14019550 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.364+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac241a1b00 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.365+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac14019850 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.365+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac1402dc70 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.365+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac240640a0 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.368+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fac14003bd0 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.441+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fac2402e180 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.441+0000 7fac1b7fe700 1 -- 172.21.15.33:0/2667830301 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fac1402d440 con 0x7fac24106e10 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 msgr2=0x7fac2419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 --2- 172.21.15.33:0/2667830301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fac14009fb0 tx=0x7fac14011940 comp rx=0 tx=0).stop 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 shutdown_connections 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 --2- 172.21.15.33:0/2667830301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac24106e10 0x7fac2419e640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:45.011 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 >> 172.21.15.33:0/2667830301 conn(0x7fac24102a10 msgr2=0x7fac24195220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:45.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 shutdown_connections 2024-04-23T18:16:45.012 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:44.443+0000 7fac2c689700 1 -- 172.21.15.33:0/2667830301 wait complete. 2024-04-23T18:16:45.012 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-04-23T18:16:45.080 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:44.794+0000 7f05e1a5e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: Activating manager daemon x 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: mgrmap e2: x(active, starting, since 0.00297516s) 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:16:45.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: Manager daemon x is now available 2024-04-23T18:16:45.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:16:45.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:16:45.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' 2024-04-23T18:16:45.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' 2024-04-23T18:16:45.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:45 smithi033 ceph-mon[29274]: from='mgr.14100 172.21.15.33:0/2477435855' entity='mgr.x' 2024-04-23T18:16:47.108 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:46 smithi033 ceph-mon[29274]: mgrmap e3: x(active, since 1.00716s) 2024-04-23T18:16:47.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:47 smithi033 ceph-mon[29274]: mgrmap e4: x(active, since 2s) 2024-04-23T18:16:47.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:47 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4181975415' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-23T18:16:48.268 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-23T18:16:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-23T18:16:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-23T18:16:48.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-23T18:16:48.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-23T18:16:27.250251+0000", 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f9e8c8700 1 Processor -- start 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f9e8c8700 1 -- start start 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f9e8c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f9e8c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f98108f00 con 0x7f6f98108510 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:48.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53120/0 (socket says 172.21.15.33:53120) 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f97fff700 1 -- 172.21.15.33:0/3918898789 learned_addr learned my addr 172.21.15.33:0/3918898789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.424+0000 7f6f97fff700 1 -- 172.21.15.33:0/3918898789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f981096c0 con 0x7f6f98108510 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f97fff700 1 --2- 172.21.15.33:0/3918898789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f880098a0 tx=0x7f6f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e332cb0cac4fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f96ffd700 1 -- 172.21.15.33:0/3918898789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f88004160 con 0x7f6f98108510 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f96ffd700 1 -- 172.21.15.33:0/3918898789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f88004300 con 0x7f6f98108510 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f96ffd700 1 -- 172.21.15.33:0/3918898789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f88011af0 con 0x7f6f98108510 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/3918898789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 msgr2=0x7f6f98108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/3918898789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6f880098a0 tx=0x7f6f88009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/3918898789 shutdown_connections 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/3918898789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f98108930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.425+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/3918898789 >> 172.21.15.33:0/3918898789 conn(0x7f6f98103b70 msgr2=0x7f6f98105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.426+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/3918898789 shutdown_connections 2024-04-23T18:16:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.426+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/3918898789 wait complete. 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.426+0000 7f6f9e8c8700 1 Processor -- start 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.426+0000 7f6f9e8c8700 1 -- start start 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f9e8c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f9e8c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9810ca70 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53124/0 (socket says 172.21.15.33:53124) 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f97fff700 1 -- 172.21.15.33:0/4181975415 learned_addr learned my addr 172.21.15.33:0/4181975415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f97fff700 1 -- 172.21.15.33:0/4181975415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f9807e070 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f97fff700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003f40 tx=0x7f6f88011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f880038b0 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6f88003a50 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9807c2b0 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f8801a5c0 con 0x7f6f98108510 2024-04-23T18:16:48.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.427+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9807c790 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.428+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f6f88024430 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.429+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f980640a0 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.429+0000 7f6f957fa700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 0x7f6f8003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.429+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6f8804da30 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.429+0000 7f6f977fe700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 0x7f6f8003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.429+0000 7f6f977fe700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 0x7f6f8003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006fd0 tx=0x7f6f8c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.434+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f8804d450 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.728+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6f98195e80 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.728+0000 7f6f957fa700 1 -- 172.21.15.33:0/4181975415 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6f8802d910 con 0x7f6f98108510 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.731+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 msgr2=0x7f6f8003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.731+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 0x7f6f8003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c006fd0 tx=0x7f6f8c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.731+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 msgr2=0x7f6f9807db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.731+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f88003f40 tx=0x7f6f88011890 comp rx=0 tx=0).stop 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 shutdown_connections 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f6f80039560 0x7f6f8003ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 --2- 172.21.15.33:0/4181975415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f98108510 0x7f6f9807db30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:48.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 >> 172.21.15.33:0/4181975415 conn(0x7f6f98103b70 msgr2=0x7f6f981048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:48.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 shutdown_connections 2024-04-23T18:16:48.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:47.732+0000 7f6f9e8c8700 1 -- 172.21.15.33:0/4181975415 wait complete. 2024-04-23T18:16:48.285 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-04-23T18:16:49.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:48 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3352331667' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-23T18:16:49.505 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:49.505 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-23T18:16:49.505 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:16:49.505 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-23T18:16:49.506 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a7048700 1 Processor -- start 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a7048700 1 -- start start 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a7048700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a7048700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0108f00 con 0x7f43a0108510 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a4de4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a4de4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53138/0 (socket says 172.21.15.33:53138) 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.703+0000 7f43a4de4700 1 -- 172.21.15.33:0/4153291075 learned_addr learned my addr 172.21.15.33:0/4153291075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.704+0000 7f43a4de4700 1 -- 172.21.15.33:0/4153291075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a01096c0 con 0x7f43a0108510 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.704+0000 7f43a4de4700 1 --2- 172.21.15.33:0/4153291075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f43940098a0 tx=0x7f4394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8357bfee5e969710 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.704+0000 7f439f7fe700 1 -- 172.21.15.33:0/4153291075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4394004160 con 0x7f43a0108510 2024-04-23T18:16:49.507 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.704+0000 7f439f7fe700 1 -- 172.21.15.33:0/4153291075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4394004300 con 0x7f43a0108510 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.704+0000 7f439f7fe700 1 -- 172.21.15.33:0/4153291075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4394011af0 con 0x7f43a0108510 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- 172.21.15.33:0/4153291075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 msgr2=0x7f43a0108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 --2- 172.21.15.33:0/4153291075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f43940098a0 tx=0x7f4394009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- 172.21.15.33:0/4153291075 shutdown_connections 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 --2- 172.21.15.33:0/4153291075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a0108930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- 172.21.15.33:0/4153291075 >> 172.21.15.33:0/4153291075 conn(0x7f43a0103b70 msgr2=0x7f43a0105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- 172.21.15.33:0/4153291075 shutdown_connections 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- 172.21.15.33:0/4153291075 wait complete. 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 Processor -- start 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.705+0000 7f43a7048700 1 -- start start 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a7048700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a7048700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f43a010ca70 con 0x7f43a0108510 2024-04-23T18:16:49.508 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a4de4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a4de4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53146/0 (socket says 172.21.15.33:53146) 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a4de4700 1 -- 172.21.15.33:0/3352331667 learned_addr learned my addr 172.21.15.33:0/3352331667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a4de4700 1 -- 172.21.15.33:0/3352331667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a019ee90 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a4de4700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4394003f40 tx=0x7f4394011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43940038b0 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4394003a50 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a019f120 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.706+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f439401a5c0 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.707+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a01a1e10 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f439401a8c0 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f439dffb700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 0x7f439003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f439404dc10 con 0x7f43a0108510 2024-04-23T18:16:49.509 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a00640a0 con 0x7f43a0108510 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f439ffff700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 0x7f439003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.708+0000 7f439ffff700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 0x7f439003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f438c006fd0 tx=0x7f438c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.713+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4394019c70 con 0x7f43a0108510 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.919+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f43a01a3160 con 0x7f43a0108510 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.920+0000 7f439dffb700 1 -- 172.21.15.33:0/3352331667 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f439404d630 con 0x7f43a0108510 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 msgr2=0x7f439003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 0x7f439003b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f438c006fd0 tx=0x7f438c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 msgr2=0x7f43a019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4394003f40 tx=0x7f4394011890 comp rx=0 tx=0).stop 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 shutdown_connections 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7f4390039510 0x7f439003b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:49.510 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 --2- 172.21.15.33:0/3352331667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43a0108510 0x7f43a019e950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:49.511 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 >> 172.21.15.33:0/3352331667 conn(0x7f43a0103b70 msgr2=0x7f43a0104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:49.511 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 shutdown_connections 2024-04-23T18:16:49.511 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:48.922+0000 7f43a7048700 1 -- 172.21.15.33:0/3352331667 wait complete. 2024-04-23T18:16:49.511 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-04-23T18:16:50.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1645215624' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-23T18:16:50.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setuser ceph since I am not root 2024-04-23T18:16:50.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setgroup ceph since I am not root 2024-04-23T18:16:50.482 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:50.400+0000 7f1d1e131200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:16:50.800 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:50.595+0000 7f1d1e131200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:50.800 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:50.701+0000 7f1d1e131200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.896+0000 7fae67625700 1 Processor -- start 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae67625700 1 -- start start 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae67625700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae67625700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae6007c900 con 0x7fae6007de80 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae653c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae653c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53160/0 (socket says 172.21.15.33:53160) 2024-04-23T18:16:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae653c1700 1 -- 172.21.15.33:0/741428175 learned_addr learned my addr 172.21.15.33:0/741428175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae653c1700 1 -- 172.21.15.33:0/741428175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6007ca80 con 0x7fae6007de80 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.897+0000 7fae653c1700 1 --2- 172.21.15.33:0/741428175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae500098a0 tx=0x7fae50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6437500345f058e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae57fff700 1 -- 172.21.15.33:0/741428175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae50004160 con 0x7fae6007de80 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae57fff700 1 -- 172.21.15.33:0/741428175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae50004300 con 0x7fae6007de80 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae57fff700 1 -- 172.21.15.33:0/741428175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae50011af0 con 0x7fae6007de80 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 -- 172.21.15.33:0/741428175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 msgr2=0x7fae6007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 --2- 172.21.15.33:0/741428175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae500098a0 tx=0x7fae50009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 -- 172.21.15.33:0/741428175 shutdown_connections 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 --2- 172.21.15.33:0/741428175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6007c330 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 -- 172.21.15.33:0/741428175 >> 172.21.15.33:0/741428175 conn(0x7fae60103b70 msgr2=0x7fae60105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.898+0000 7fae67625700 1 -- 172.21.15.33:0/741428175 shutdown_connections 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae67625700 1 -- 172.21.15.33:0/741428175 wait complete. 2024-04-23T18:16:50.810 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae67625700 1 Processor -- start 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae67625700 1 -- start start 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae67625700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae67625700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fae6010ea40 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae653c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae653c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53164/0 (socket says 172.21.15.33:53164) 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.899+0000 7fae653c1700 1 -- 172.21.15.33:0/1645215624 learned_addr learned my addr 172.21.15.33:0/1645215624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae653c1700 1 -- 172.21.15.33:0/1645215624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6019ee10 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae653c1700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae50003f40 tx=0x7fae50011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae500038b0 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fae50003a50 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6019f0a0 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae5001a5c0 con 0x7fae6007de80 2024-04-23T18:16:50.811 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.900+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae601a1d90 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.901+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fae5001a8c0 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.901+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae600640a0 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.901+0000 7fae567fc700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 0x7fae4c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.901+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fae5004da90 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.901+0000 7fae64bc0700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 0x7fae4c03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.902+0000 7fae64bc0700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 0x7fae4c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae5c006fd0 tx=0x7fae5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:49.906+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae5004d4b0 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.177+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fae6002df80 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.229+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fae5002d3f0 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.231+0000 7fae567fc700 1 -- 172.21.15.33:0/1645215624 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fae5000bce0 con 0x7fae6007de80 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 msgr2=0x7fae4c03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 0x7fae4c03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fae5c006fd0 tx=0x7fae5c006e40 comp rx=0 tx=0).stop 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 msgr2=0x7fae6019e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:50.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae50003f40 tx=0x7fae50011890 comp rx=0 tx=0).stop 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 shutdown_connections 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fae4c039510 0x7fae4c03b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 --2- 172.21.15.33:0/1645215624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae6007de80 0x7fae6019e8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 >> 172.21.15.33:0/1645215624 conn(0x7fae60103b70 msgr2=0x7fae60104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 shutdown_connections 2024-04-23T18:16:50.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:50.232+0000 7fae67625700 1 -- 172.21.15.33:0/1645215624 wait complete. 2024-04-23T18:16:51.068 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:51.032+0000 7f1d1e131200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:16:51.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:51 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1645215624' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-23T18:16:51.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:51 smithi033 ceph-mon[29274]: mgrmap e5: x(active, since 5s) 2024-04-23T18:16:51.479 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:51.122+0000 7f1d1e131200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:16:52.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:52.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.279+0000 7fe485d4b700 1 Processor -- start 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.279+0000 7fe485d4b700 1 -- start start 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.279+0000 7fe485d4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.279+0000 7fe485d4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe48007be80 con 0x7fe480108f40 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.279+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:52.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53200/0 (socket says 172.21.15.33:53200) 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47f7fe700 1 -- 172.21.15.33:0/3590712522 learned_addr learned my addr 172.21.15.33:0/3590712522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47f7fe700 1 -- 172.21.15.33:0/3590712522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48010b870 con 0x7fe480108f40 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47f7fe700 1 --2- 172.21.15.33:0/3590712522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4680098a0 tx=0x7fe468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfb1cb100469632 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47e7fc700 1 -- 172.21.15.33:0/3590712522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe468004160 con 0x7fe480108f40 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47e7fc700 1 -- 172.21.15.33:0/3590712522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe468004300 con 0x7fe480108f40 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.280+0000 7fe47e7fc700 1 -- 172.21.15.33:0/3590712522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe468011af0 con 0x7fe480108f40 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 -- 172.21.15.33:0/3590712522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 msgr2=0x7fe48010b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3590712522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4680098a0 tx=0x7fe468009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 -- 172.21.15.33:0/3590712522 shutdown_connections 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3590712522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48010b330 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 -- 172.21.15.33:0/3590712522 >> 172.21.15.33:0/3590712522 conn(0x7fe4801029b0 msgr2=0x7fe480104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 -- 172.21.15.33:0/3590712522 shutdown_connections 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.281+0000 7fe485d4b700 1 -- 172.21.15.33:0/3590712522 wait complete. 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe485d4b700 1 Processor -- start 2024-04-23T18:16:52.215 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe485d4b700 1 -- start start 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe485d4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe485d4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe48010c830 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53214/0 (socket says 172.21.15.33:53214) 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe47f7fe700 1 -- 172.21.15.33:0/3927177163 learned_addr learned my addr 172.21.15.33:0/3927177163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.282+0000 7fe47f7fe700 1 -- 172.21.15.33:0/3927177163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48019c990 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe47f7fe700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fb0 tx=0x7fe468011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4680036a0 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe46801a430 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48019cc20 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe468019550 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.283+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48019f910 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fe468019850 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4800640a0 con 0x7fe480108f40 2024-04-23T18:16:52.216 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe47cff9700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 0x7fe46c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe46801aa30 con 0x7fe480108f40 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe47effd700 1 -- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 msgr2=0x7fe46c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.284+0000 7fe47effd700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 0x7fe46c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.289+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe468023c90 con 0x7fe480108f40 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.485+0000 7fe47effd700 1 -- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 msgr2=0x7fe46c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.485+0000 7fe47effd700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 0x7fe46c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.601+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe48019fae0 con 0x7fe480108f40 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.602+0000 7fe47cff9700 1 -- 172.21.15.33:0/3927177163 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fe48019fae0 con 0x7fe480108f40 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 msgr2=0x7fe46c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 0x7fe46c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 msgr2=0x7fe48019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fb0 tx=0x7fe468011940 comp rx=0 tx=0).stop 2024-04-23T18:16:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 shutdown_connections 2024-04-23T18:16:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fe46c039560 0x7fe46c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 --2- 172.21.15.33:0/3927177163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480108f40 0x7fe48019c450 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:52.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 >> 172.21.15.33:0/3927177163 conn(0x7fe4801029b0 msgr2=0x7fe480103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:52.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 shutdown_connections 2024-04-23T18:16:52.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:51.604+0000 7fe485d4b700 1 -- 172.21.15.33:0/3927177163 wait complete. 2024-04-23T18:16:52.219 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-23T18:16:52.219 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-04-23T18:16:52.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3927177163' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:16:52.463 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:52.376+0000 7f1d1e131200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:16:52.727 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:52.554+0000 7f1d1e131200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:52.727 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:52.734+0000 7f1d1e131200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:16:53.290 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:53.034+0000 7f1d1e131200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:16:53.290 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:53.130+0000 7f1d1e131200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:16:53.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:53.339+0000 7f1d1e131200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:16:53.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:53.539+0000 7f1d1e131200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:16:54.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:54.160+0000 7f1d1e131200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:16:54.895 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:54.595+0000 7f1d1e131200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:16:54.896 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:54.694+0000 7f1d1e131200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:16:54.896 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:54.902+0000 7f1d1e131200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:16:55.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:55.443+0000 7f1d1e131200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:16:55.692 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:55.538+0000 7f1d1e131200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:16:55.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:55.789+0000 7f1d1e131200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:16:55.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:55.926+0000 7f1d1e131200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:16:56.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:56.024+0000 7f1d1e131200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:16:56.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:56.143+0000 7f1d1e131200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:16:56.448 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:56.454+0000 7f1d1e131200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:16:57.347 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:57.078+0000 7f1d1e131200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:16:57.347 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:16:57.300+0000 7f1d1e131200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: Active manager daemon x restarted 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: Activating manager daemon x 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: osdmap e2: 0 total, 0 up, 0 in 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: mgrmap e6: x(active, starting, since 0.00316983s) 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:16:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: Manager daemon x is now available 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:16:57.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:57 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:16:58.604 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:58 smithi033 ceph-mon[29274]: Found migration_current of "None". Setting to last migration. 2024-04-23T18:16:58.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:58 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:16:58.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:58 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:16:58.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:58 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:16:58.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:58 smithi033 ceph-mon[29274]: mgrmap e7: x(active, since 1.00567s) 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.688+0000 7fde64813700 1 Processor -- start 2024-04-23T18:16:58.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.688+0000 7fde64813700 1 -- start start 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde64813700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde64813700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde60107e20 con 0x7fde60107430 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53226/0 (socket says 172.21.15.33:53226) 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5e59c700 1 -- 172.21.15.33:0/582296805 learned_addr learned my addr 172.21.15.33:0/582296805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5e59c700 1 -- 172.21.15.33:0/582296805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde60108630 con 0x7fde60107430 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5e59c700 1 --2- 172.21.15.33:0/582296805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde480098a0 tx=0x7fde48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a828fca015219fee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5d59a700 1 -- 172.21.15.33:0/582296805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde48004160 con 0x7fde60107430 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5d59a700 1 -- 172.21.15.33:0/582296805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde48004300 con 0x7fde60107430 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.689+0000 7fde5d59a700 1 -- 172.21.15.33:0/582296805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde48011af0 con 0x7fde60107430 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 -- 172.21.15.33:0/582296805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 msgr2=0x7fde60107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 --2- 172.21.15.33:0/582296805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fde480098a0 tx=0x7fde48009bb0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 -- 172.21.15.33:0/582296805 shutdown_connections 2024-04-23T18:16:58.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 --2- 172.21.15.33:0/582296805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde60107850 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 -- 172.21.15.33:0/582296805 >> 172.21.15.33:0/582296805 conn(0x7fde60102a10 msgr2=0x7fde60104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 -- 172.21.15.33:0/582296805 shutdown_connections 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.690+0000 7fde64813700 1 -- 172.21.15.33:0/582296805 wait complete. 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.691+0000 7fde64813700 1 Processor -- start 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde64813700 1 -- start start 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde64813700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde64813700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010b860 con 0x7fde60107430 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde5e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53232/0 (socket says 172.21.15.33:53232) 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde5e59c700 1 -- 172.21.15.33:0/2956416959 learned_addr learned my addr 172.21.15.33:0/2956416959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde5e59c700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6019f1d0 con 0x7fde60107430 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.692+0000 7fde5e59c700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde48009fb0 tx=0x7fde48011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde480036a0 con 0x7fde60107430 2024-04-23T18:16:58.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde4801a430 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde64813700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6019f3a0 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde48019550 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde64813700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6019f740 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.693+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fde48019850 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.694+0000 7fde577fe700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.694+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde60069b90 con 0x7fde4c039560 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.694+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde4804ecd0 con 0x7fde60107430 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.694+0000 7fde5dd9b700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.694+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.895+0000 7fde5dd9b700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:52.895+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:53.296+0000 7fde5dd9b700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:53.296+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:54.097+0000 7fde5dd9b700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:58.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:54.097+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:55.698+0000 7fde5dd9b700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1526532469 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:55.698+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:57.303+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fde4802d960 con 0x7fde60107430 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:57.304+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 msgr2=0x7fde4c03ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:57.304+0000 7fde577fe700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.305+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fde4802d440 con 0x7fde60107430 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.305+0000 7fde577fe700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 0x7fde4c03f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.305+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde48004300 con 0x7fde4c03cc60 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.305+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 0x7fde4c03f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.306+0000 7fde5dd9b700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 0x7fde4c03f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde50003a10 tx=0x7fde50009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.306+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fde48004300 con 0x7fde4c03cc60 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.313+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fde6010b9e0 con 0x7fde4c03cc60 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.313+0000 7fde577fe700 1 -- 172.21.15.33:0/2956416959 <== mgr.14120 v2:172.21.15.33:6800/4245296611 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fde6010b9e0 con 0x7fde4c03cc60 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 msgr2=0x7fde4c03f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:58.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 0x7fde4c03f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fde50003a10 tx=0x7fde50009290 comp rx=0 tx=0).stop 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 msgr2=0x7fde6007ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fde48009fb0 tx=0x7fde48011940 comp rx=0 tx=0).stop 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 shutdown_connections 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde60107430 0x7fde6007ba60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fde4c03cc60 0x7fde4c03f050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 --2- 172.21.15.33:0/2956416959 >> [v2:172.21.15.33:6800/1526532469,v1:172.21.15.33:6801/1526532469] conn(0x7fde4c039560 0x7fde4c03ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 >> 172.21.15.33:0/2956416959 conn(0x7fde60102a10 msgr2=0x7fde601035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 shutdown_connections 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:58.314+0000 7fde557fa700 1 -- 172.21.15.33:0/2956416959 wait complete. 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-04-23T18:16:58.886 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: [23/Apr/2024:18:16:58] ENGINE Bus STARTING 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: [23/Apr/2024:18:16:58] ENGINE Serving on http://172.21.15.33:8765 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: [23/Apr/2024:18:16:58] ENGINE Serving on https://172.21.15.33:7150 2024-04-23T18:16:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: [23/Apr/2024:18:16:58] ENGINE Bus STARTED 2024-04-23T18:16:59.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:16:59 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.308+0000 7fc1bbbbb700 1 Processor -- start 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.308+0000 7fc1bbbbb700 1 -- start start 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1bbbbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1bbbbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b407be80 con 0x7fc1b4108f40 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b9957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b9957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49918/0 (socket says 172.21.15.33:49918) 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b9957700 1 -- 172.21.15.33:0/94902778 learned_addr learned my addr 172.21.15.33:0/94902778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b9957700 1 -- 172.21.15.33:0/94902778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b410b870 con 0x7fc1b4108f40 2024-04-23T18:17:00.111 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b9957700 1 --2- 172.21.15.33:0/94902778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00098a0 tx=0x7fc1b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97127e6a223db3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.309+0000 7fc1b8955700 1 -- 172.21.15.33:0/94902778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1b0004160 con 0x7fc1b4108f40 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1b8955700 1 -- 172.21.15.33:0/94902778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1b0004300 con 0x7fc1b4108f40 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1b8955700 1 -- 172.21.15.33:0/94902778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1b0011af0 con 0x7fc1b4108f40 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/94902778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 msgr2=0x7fc1b410b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/94902778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00098a0 tx=0x7fc1b0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/94902778 shutdown_connections 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/94902778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b410b330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/94902778 >> 172.21.15.33:0/94902778 conn(0x7fc1b41029b0 msgr2=0x7fc1b4104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/94902778 shutdown_connections 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.310+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/94902778 wait complete. 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1bbbbb700 1 Processor -- start 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1bbbbb700 1 -- start start 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1bbbbb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1bbbbb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b410c830 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1b9957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1b9957700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49926/0 (socket says 172.21.15.33:49926) 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1b9957700 1 -- 172.21.15.33:0/3514750137 learned_addr learned my addr 172.21.15.33:0/3514750137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.311+0000 7fc1b9957700 1 -- 172.21.15.33:0/3514750137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b4108d00 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1b9957700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009fb0 tx=0x7fc1b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1b00036a0 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc1b001a430 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1b0019550 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b4106ef0 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.312+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b4107430 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.313+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fc1b0019800 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.313+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b40640a0 con 0x7fc1b4108f40 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.313+0000 7fc1aaffd700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 0x7fc1a003b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:00.113 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.313+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc1b004d6f0 con 0x7fc1b4108f40 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.313+0000 7fc1b9156700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 0x7fc1a003b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.314+0000 7fc1b9156700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 0x7fc1a003b940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.318+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1b000fda0 con 0x7fc1b4108f40 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.361+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc1b002d960 con 0x7fc1b4108f40 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.553+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fc1b4103c00 con 0x7fc1a0039480 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.558+0000 7fc1aaffd700 1 -- 172.21.15.33:0/3514750137 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc1b4103c00 con 0x7fc1a0039480 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 msgr2=0x7fc1a003b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 0x7fc1a003b940 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4006fd0 tx=0x7fc1a4006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 msgr2=0x7fc1b41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009fb0 tx=0x7fc1b0011940 comp rx=0 tx=0).stop 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 shutdown_connections 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fc1a0039480 0x7fc1a003b940 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.114 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 --2- 172.21.15.33:0/3514750137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1b4108f40 0x7fc1b41087c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:00.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.560+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 >> 172.21.15.33:0/3514750137 conn(0x7fc1b41029b0 msgr2=0x7fc1b41034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:00.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.561+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 shutdown_connections 2024-04-23T18:17:00.115 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:16:59.561+0000 7fc1bbbbb700 1 -- 172.21.15.33:0/3514750137 wait complete. 2024-04-23T18:17:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:00 smithi033 ceph-mon[29274]: mgrmap e8: x(active, since 2s) 2024-04-23T18:17:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:00 smithi033 ceph-mon[29274]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:00 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:00 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.519+0000 7f2ce9a58700 1 Processor -- start 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.519+0000 7f2ce9a58700 1 -- start start 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.519+0000 7f2ce9a58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.519+0000 7f2ce9a58700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4107210 con 0x7f2ce4106820 2024-04-23T18:17:01.318 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.519+0000 7f2ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49932/0 (socket says 172.21.15.33:49932) 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce2ffd700 1 -- 172.21.15.33:0/2980476083 learned_addr learned my addr 172.21.15.33:0/2980476083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce2ffd700 1 -- 172.21.15.33:0/2980476083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce41097b0 con 0x7f2ce4106820 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce2ffd700 1 --2- 172.21.15.33:0/2980476083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7c408470f23883 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce1ffb700 1 -- 172.21.15.33:0/2980476083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4004160 con 0x7f2ce4106820 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce1ffb700 1 -- 172.21.15.33:0/2980476083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2cd4004300 con 0x7f2ce4106820 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.520+0000 7f2ce1ffb700 1 -- 172.21.15.33:0/2980476083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4011af0 con 0x7f2ce4106820 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 -- 172.21.15.33:0/2980476083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 msgr2=0x7f2ce4106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/2980476083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 -- 172.21.15.33:0/2980476083 shutdown_connections 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/2980476083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4106c40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 -- 172.21.15.33:0/2980476083 >> 172.21.15.33:0/2980476083 conn(0x7f2ce41021b0 msgr2=0x7f2ce41045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 -- 172.21.15.33:0/2980476083 shutdown_connections 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 -- 172.21.15.33:0/2980476083 wait complete. 2024-04-23T18:17:01.319 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.521+0000 7f2ce9a58700 1 Processor -- start 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce9a58700 1 -- start start 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce9a58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce9a58700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410cb60 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49942/0 (socket says 172.21.15.33:49942) 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce2ffd700 1 -- 172.21.15.33:0/3078959778 learned_addr learned my addr 172.21.15.33:0/3078959778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce2ffd700 1 -- 172.21.15.33:0/3078959778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce4109570 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce2ffd700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4003f40 tx=0x7f2cd4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd40038b0 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2cd4003a50 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce4107760 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd401a5c0 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.522+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce4107ca0 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.523+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2cd401a870 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.523+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce4069b90 con 0x7f2ce4106820 2024-04-23T18:17:01.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.523+0000 7f2ce8a56700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 0x7f2ccc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.524+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2cd404dc70 con 0x7f2ce4106820 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.524+0000 7f2ce27fc700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 0x7f2ccc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.524+0000 7f2ce27fc700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 0x7f2ccc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.529+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cd4019c70 con 0x7f2ce4106820 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.739+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2ce4196040 con 0x7f2ccc0395b0 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.740+0000 7f2ce8a56700 1 -- 172.21.15.33:0/3078959778 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2ce4196040 con 0x7f2ccc0395b0 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 msgr2=0x7f2ccc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 0x7f2ccc03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006fd0 tx=0x7f2cd8006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 msgr2=0x7f2ce4109030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4003f40 tx=0x7f2cd4011890 comp rx=0 tx=0).stop 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 shutdown_connections 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f2ccc0395b0 0x7f2ccc03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 --2- 172.21.15.33:0/3078959778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce4106820 0x7f2ce4109030 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 >> 172.21.15.33:0/3078959778 conn(0x7f2ce41021b0 msgr2=0x7f2ce4102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:01.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.742+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 shutdown_connections 2024-04-23T18:17:01.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:00.743+0000 7f2ce9a58700 1 -- 172.21.15.33:0/3078959778 wait complete. 2024-04-23T18:17:01.322 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-04-23T18:17:01.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:01 smithi033 ceph-mon[29274]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:02.231 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: Generating public/private rsa key pair. 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: Your identification has been saved in /tmp/tmpbgsxt_c7/key. 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: Your public key has been saved in /tmp/tmpbgsxt_c7/key.pub. 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: The key fingerprint is: 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: SHA256:MRl1h1Afx9BhixoJYP59Ez15MndLx0SpBXVVDbFLW1o ceph-766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: The key's randomart image is: 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: +---[RSA 3072]----+ 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | oooooooO@&| 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | o o..oo+OB| 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | .+ o oO*E| 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | .o. oooX*| 2024-04-23T18:17:02.232 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | S. o o+. | 2024-04-23T18:17:02.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | . . | 2024-04-23T18:17:02.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | | 2024-04-23T18:17:02.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | | 2024-04-23T18:17:02.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: | | 2024-04-23T18:17:02.233 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: +----[SHA256]-----+ 2024-04-23T18:17:02.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98ded9a700 1 Processor -- start 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98ded9a700 1 -- start start 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98ded9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98ded9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98d807be40 con 0x7f98d8108fd0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98dcb36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98dcb36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49946/0 (socket says 172.21.15.33:49946) 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.760+0000 7f98dcb36700 1 -- 172.21.15.33:0/3116994305 learned_addr learned my addr 172.21.15.33:0/3116994305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.761+0000 7f98dcb36700 1 -- 172.21.15.33:0/3116994305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d810b900 con 0x7f98d8108fd0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.761+0000 7f98dcb36700 1 --2- 172.21.15.33:0/3116994305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0098a0 tx=0x7f98cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adf33c25c02d11f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.761+0000 7f98d77fe700 1 -- 172.21.15.33:0/3116994305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98cc004160 con 0x7f98d8108fd0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.761+0000 7f98d77fe700 1 -- 172.21.15.33:0/3116994305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f98cc004300 con 0x7f98d8108fd0 2024-04-23T18:17:02.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.761+0000 7f98d77fe700 1 -- 172.21.15.33:0/3116994305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98cc011af0 con 0x7f98d8108fd0 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- 172.21.15.33:0/3116994305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 msgr2=0x7f98d810b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 --2- 172.21.15.33:0/3116994305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0098a0 tx=0x7f98cc009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- 172.21.15.33:0/3116994305 shutdown_connections 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 --2- 172.21.15.33:0/3116994305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d810b3c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- 172.21.15.33:0/3116994305 >> 172.21.15.33:0/3116994305 conn(0x7f98d81029c0 msgr2=0x7f98d8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- 172.21.15.33:0/3116994305 shutdown_connections 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- 172.21.15.33:0/3116994305 wait complete. 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 Processor -- start 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.762+0000 7f98ded9a700 1 -- start start 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98ded9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98ded9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98d810c860 con 0x7f98d8108fd0 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98dcb36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98dcb36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49954/0 (socket says 172.21.15.33:49954) 2024-04-23T18:17:02.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98dcb36700 1 -- 172.21.15.33:0/1713584850 learned_addr learned my addr 172.21.15.33:0/1713584850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98dcb36700 1 -- 172.21.15.33:0/1713584850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d819cc90 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98dcb36700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98cc011ac0 tx=0x7f98cc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98cc003b20 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f98cc01a8c0 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d819cf20 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.763+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98cc0248d0 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.764+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d819fc10 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f98cc022020 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d80640a0 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98d5ffb700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 0x7f98c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f98cc04d9a0 con 0x7f98d8108fd0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98d7fff700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 0x7f98c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:02.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.765+0000 7f98d7fff700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 0x7f98c803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006fd0 tx=0x7f98c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.770+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98cc02cdf0 con 0x7f98d8108fd0 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:01.978+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f98d802dff0 con 0x7f98c8039600 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.171+0000 7f98d5ffb700 1 -- 172.21.15.33:0/1713584850 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f98d802dff0 con 0x7f98c8039600 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 msgr2=0x7f98c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 0x7f98c803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006fd0 tx=0x7f98c4006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 msgr2=0x7f98d819c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98cc011ac0 tx=0x7f98cc011ba0 comp rx=0 tx=0).stop 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 shutdown_connections 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f98c8039600 0x7f98c803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 --2- 172.21.15.33:0/1713584850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98d8108fd0 0x7f98d819c750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 >> 172.21.15.33:0/1713584850 conn(0x7f98d81029c0 msgr2=0x7f98d8103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 shutdown_connections 2024-04-23T18:17:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:02.173+0000 7f98ded9a700 1 -- 172.21.15.33:0/1713584850 wait complete. 2024-04-23T18:17:03.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:03 smithi033 ceph-mon[29274]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:03.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:03 smithi033 ceph-mon[29274]: Generating ssh key... 2024-04-23T18:17:03.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:03 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:03.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:03 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:03.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:03.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.150+0000 7f6b862d3700 1 Processor -- start 2024-04-23T18:17:03.997 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.150+0000 7f6b862d3700 1 -- start start 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.150+0000 7f6b862d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b862d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80108ed0 con 0x7f6b801084e0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49964/0 (socket says 172.21.15.33:49964) 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b7ffff700 1 -- 172.21.15.33:0/324065474 learned_addr learned my addr 172.21.15.33:0/324065474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b7ffff700 1 -- 172.21.15.33:0/324065474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b80109690 con 0x7f6b801084e0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.151+0000 7f6b7ffff700 1 --2- 172.21.15.33:0/324065474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098a0 tx=0x7f6b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359c6ccc0ae49142 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.152+0000 7f6b7effd700 1 -- 172.21.15.33:0/324065474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b70004160 con 0x7f6b801084e0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.152+0000 7f6b7effd700 1 -- 172.21.15.33:0/324065474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b70004300 con 0x7f6b801084e0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.152+0000 7f6b7effd700 1 -- 172.21.15.33:0/324065474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b70011af0 con 0x7f6b801084e0 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.152+0000 7f6b862d3700 1 -- 172.21.15.33:0/324065474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 msgr2=0x7f6b80108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.152+0000 7f6b862d3700 1 --2- 172.21.15.33:0/324065474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b700098a0 tx=0x7f6b70009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:03.998 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 -- 172.21.15.33:0/324065474 shutdown_connections 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 --2- 172.21.15.33:0/324065474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b80108900 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 -- 172.21.15.33:0/324065474 >> 172.21.15.33:0/324065474 conn(0x7f6b80103b40 msgr2=0x7f6b80105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 -- 172.21.15.33:0/324065474 shutdown_connections 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 -- 172.21.15.33:0/324065474 wait complete. 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 Processor -- start 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.153+0000 7f6b862d3700 1 -- start start 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b862d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b862d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8010ca40 con 0x7f6b801084e0 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49980/0 (socket says 172.21.15.33:49980) 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7ffff700 1 -- 172.21.15.33:0/2348505865 learned_addr learned my addr 172.21.15.33:0/2348505865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7ffff700 1 -- 172.21.15.33:0/2348505865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8007e0c0 con 0x7f6b801084e0 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7ffff700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009fb0 tx=0x7f6b70011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b700036a0 con 0x7f6b801084e0 2024-04-23T18:17:03.999 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6b7001a430 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8007c2b0 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b70019550 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.154+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8007c7f0 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.155+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c005360 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.156+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6b70019850 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.156+0000 7f6b7d7fa700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 0x7f6b6803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.156+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b7004d8c0 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.156+0000 7f6b7f7fe700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 0x7f6b6803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.156+0000 7f6b7f7fe700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 0x7f6b6803ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006fd0 tx=0x7f6b74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.161+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b7000fda0 con 0x7f6b801084e0 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.373+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f6b6c000bf0 con 0x7f6b68039560 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.373+0000 7f6b7d7fa700 1 -- 172.21.15.33:0/2348505865 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f6b6c000bf0 con 0x7f6b68039560 2024-04-23T18:17:04.000 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 msgr2=0x7f6b6803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 0x7f6b6803ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6b74006fd0 tx=0x7f6b74006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 msgr2=0x7f6b8007db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009fb0 tx=0x7f6b70011940 comp rx=0 tx=0).stop 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 shutdown_connections 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f6b68039560 0x7f6b6803ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 --2- 172.21.15.33:0/2348505865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6b801084e0 0x7f6b8007db80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.376+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 >> 172.21.15.33:0/2348505865 conn(0x7f6b80103b40 msgr2=0x7f6b801046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.377+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 shutdown_connections 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:03.377+0000 7f6b862d3700 1 -- 172.21.15.33:0/2348505865 wait complete. 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-04-23T18:17:04.001 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-04-23T18:17:04.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:04 smithi033 ceph-mon[29274]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:05.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:05 smithi033 ceph-mon[29274]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:05.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:05 smithi033 ceph-mon[29274]: Deploying cephadm binary to smithi033 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.402+0000 7f8e27fbd700 1 Processor -- start 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.402+0000 7f8e27fbd700 1 -- start start 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.402+0000 7f8e27fbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.402+0000 7f8e27fbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2007bea0 con 0x7f8e20108f20 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e25d59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e25d59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49988/0 (socket says 172.21.15.33:49988) 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e25d59700 1 -- 172.21.15.33:0/46481743 learned_addr learned my addr 172.21.15.33:0/46481743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:09.445 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e25d59700 1 -- 172.21.15.33:0/46481743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2010b850 con 0x7f8e20108f20 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e25d59700 1 --2- 172.21.15.33:0/46481743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009e80 tx=0x7f8e1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4e5bd85da8b085 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e24d57700 1 -- 172.21.15.33:0/46481743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10017070 con 0x7f8e20108f20 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e24d57700 1 -- 172.21.15.33:0/46481743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e1000b8c0 con 0x7f8e20108f20 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.403+0000 7f8e24d57700 1 -- 172.21.15.33:0/46481743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10004490 con 0x7f8e20108f20 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 -- 172.21.15.33:0/46481743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 msgr2=0x7f8e2010b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/46481743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009e80 tx=0x7f8e1000b330 comp rx=0 tx=0).stop 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 -- 172.21.15.33:0/46481743 shutdown_connections 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/46481743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2010b310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 -- 172.21.15.33:0/46481743 >> 172.21.15.33:0/46481743 conn(0x7f8e201029b0 msgr2=0x7f8e20104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 -- 172.21.15.33:0/46481743 shutdown_connections 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.404+0000 7f8e27fbd700 1 -- 172.21.15.33:0/46481743 wait complete. 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e27fbd700 1 Processor -- start 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e27fbd700 1 -- start start 2024-04-23T18:17:09.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e27fbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e27fbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2010c7a0 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e25d59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e25d59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49990/0 (socket says 172.21.15.33:49990) 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e25d59700 1 -- 172.21.15.33:0/3950615091 learned_addr learned my addr 172.21.15.33:0/3950615091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e25d59700 1 -- 172.21.15.33:0/3950615091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2019c8e0 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.405+0000 7f8e25d59700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000b7b0 tx=0x7f8e1000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10020020 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e10003d10 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2019cb70 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e10017040 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2019f860 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.406+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8e10023020 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.407+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e200640a0 con 0x7f8e20108f20 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.407+0000 7f8e1effd700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 0x7f8e0803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:09.447 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.407+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8e1000f050 con 0x7f8e20108f20 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.407+0000 7f8e25558700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 0x7f8e0803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.407+0000 7f8e25558700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 0x7f8e0803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8e14006fd0 tx=0x7f8e14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.412+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e1002a020 con 0x7f8e20108f20 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:04.628+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) -- 0x7f8e20103bf0 con 0x7f8e08039600 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.869+0000 7f8e1effd700 1 -- 172.21.15.33:0/3950615091 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8e20103bf0 con 0x7f8e08039600 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 msgr2=0x7f8e0803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 0x7f8e0803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8e14006fd0 tx=0x7f8e14006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 msgr2=0x7f8e2019c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000b7b0 tx=0x7f8e1000be90 comp rx=0 tx=0).stop 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 shutdown_connections 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f8e08039600 0x7f8e0803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 --2- 172.21.15.33:0/3950615091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e20108f20 0x7f8e2019c3a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 >> 172.21.15.33:0/3950615091 conn(0x7f8e201029b0 msgr2=0x7f8e201034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:09.448 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 shutdown_connections 2024-04-23T18:17:09.449 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:08.872+0000 7f8e27fbd700 1 -- 172.21.15.33:0/3950615091 wait complete. 2024-04-23T18:17:09.449 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-04-23T18:17:10.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:09 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:10.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:09 smithi033 ceph-mon[29274]: Added host smithi033 2024-04-23T18:17:10.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:09 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:10.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-23T18:17:10.847 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.908+0000 7f0694243700 1 Processor -- start 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.908+0000 7f0694243700 1 -- start start 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.908+0000 7f0694243700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.908+0000 7f0694243700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f068c07be40 con 0x7f068c106e10 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0691fdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0691fdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50340/0 (socket says 172.21.15.33:50340) 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0691fdf700 1 -- 172.21.15.33:0/2393247860 learned_addr learned my addr 172.21.15.33:0/2393247860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0691fdf700 1 -- 172.21.15.33:0/2393247860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c107770 con 0x7f068c106e10 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0691fdf700 1 --2- 172.21.15.33:0/2393247860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e80 tx=0x7f068400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b9d24c86544ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0690fdd700 1 -- 172.21.15.33:0/2393247860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684017070 con 0x7f068c106e10 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0690fdd700 1 -- 172.21.15.33:0/2393247860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f068400b8c0 con 0x7f068c106e10 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.909+0000 7f0690fdd700 1 -- 172.21.15.33:0/2393247860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684004490 con 0x7f068c106e10 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 -- 172.21.15.33:0/2393247860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 msgr2=0x7f068c107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 --2- 172.21.15.33:0/2393247860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e80 tx=0x7f068400b330 comp rx=0 tx=0).stop 2024-04-23T18:17:10.848 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 -- 172.21.15.33:0/2393247860 shutdown_connections 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 --2- 172.21.15.33:0/2393247860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c107230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 -- 172.21.15.33:0/2393247860 >> 172.21.15.33:0/2393247860 conn(0x7f068c102a10 msgr2=0x7f068c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 -- 172.21.15.33:0/2393247860 shutdown_connections 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 -- 172.21.15.33:0/2393247860 wait complete. 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.910+0000 7f0694243700 1 Processor -- start 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0694243700 1 -- start start 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0694243700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0694243700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f068c108730 con 0x7f068c106e10 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0691fdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0691fdf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50344/0 (socket says 172.21.15.33:50344) 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0691fdf700 1 -- 172.21.15.33:0/1046068387 learned_addr learned my addr 172.21.15.33:0/1046068387 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0691fdf700 1 -- 172.21.15.33:0/1046068387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068c19eb50 con 0x7f068c106e10 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0691fdf700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f068400b7b0 tx=0x7f068400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684020020 con 0x7f068c106e10 2024-04-23T18:17:10.849 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068c19ede0 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0684003d10 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0684017040 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.911+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068c1a1ad0 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.912+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0684023020 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.912+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068c0640a0 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.912+0000 7f068affd700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 0x7f067403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.912+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f068400f050 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.913+0000 7f06917de700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 0x7f067403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.913+0000 7f06917de700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 0x7f067403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:09.918+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f068402a020 con 0x7f068c106e10 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.143+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f068c02e360 con 0x7f06740395b0 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.147+0000 7f068affd700 1 -- 172.21.15.33:0/1046068387 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f068c02e360 con 0x7f06740395b0 2024-04-23T18:17:10.850 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 msgr2=0x7f067403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 0x7f067403ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 msgr2=0x7f068c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f068400b7b0 tx=0x7f068400be90 comp rx=0 tx=0).stop 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 shutdown_connections 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f06740395b0 0x7f067403ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 --2- 172.21.15.33:0/1046068387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068c106e10 0x7f068c19e610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 >> 172.21.15.33:0/1046068387 conn(0x7f068c102a10 msgr2=0x7f068c195280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 shutdown_connections 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:10.150+0000 7f0694243700 1 -- 172.21.15.33:0/1046068387 wait complete. 2024-04-23T18:17:10.851 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-04-23T18:17:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:11 smithi033 ceph-mon[29274]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:11 smithi033 ceph-mon[29274]: Saving service mon spec with placement count:5 2024-04-23T18:17:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:11 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:12.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:12 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:12.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:12 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:12.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:12 smithi033 ceph-mon[29274]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:12.824 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:12 smithi033 ceph-mon[29274]: Saving service mgr spec with placement count:2 2024-04-23T18:17:12.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:12 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbdbcfd700 1 Processor -- start 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbdbcfd700 1 -- start start 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbdbcfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbdbcfd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd4108f30 con 0x7fcbd4108540 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbd9a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbd9a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50350/0 (socket says 172.21.15.33:50350) 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.005+0000 7fcbd9a99700 1 -- 172.21.15.33:0/2933151238 learned_addr learned my addr 172.21.15.33:0/2933151238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.006+0000 7fcbd9a99700 1 -- 172.21.15.33:0/2933151238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd41096f0 con 0x7fcbd4108540 2024-04-23T18:17:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.006+0000 7fcbd9a99700 1 --2- 172.21.15.33:0/2933151238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098a0 tx=0x7fcbd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d60e1f871bdf2e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.006+0000 7fcbd8a97700 1 -- 172.21.15.33:0/2933151238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbd0004160 con 0x7fcbd4108540 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.006+0000 7fcbd8a97700 1 -- 172.21.15.33:0/2933151238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd0004300 con 0x7fcbd4108540 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.006+0000 7fcbd8a97700 1 -- 172.21.15.33:0/2933151238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbd0011a20 con 0x7fcbd4108540 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/2933151238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 msgr2=0x7fcbd4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/2933151238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00098a0 tx=0x7fcbd0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/2933151238 shutdown_connections 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/2933151238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd4108960 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/2933151238 >> 172.21.15.33:0/2933151238 conn(0x7fcbd4103ba0 msgr2=0x7fcbd4105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/2933151238 shutdown_connections 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/2933151238 wait complete. 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.007+0000 7fcbdbcfd700 1 Processor -- start 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbdbcfd700 1 -- start start 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbdbcfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbdbcfd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410caa0 con 0x7fcbd4108540 2024-04-23T18:17:12.833 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbd9a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbd9a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50360/0 (socket says 172.21.15.33:50360) 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbd9a99700 1 -- 172.21.15.33:0/1984472558 learned_addr learned my addr 172.21.15.33:0/1984472558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbd9a99700 1 -- 172.21.15.33:0/1984472558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd419eb30 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbd9a99700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00119f0 tx=0x7fcbd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbd00039e0 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbd001b8f0 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbd00237a0 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.008+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd419edc0 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.009+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd41a1ab0 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.009+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcbd001b440 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.010+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd410cc20 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.010+0000 7fcbcaffd700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 0x7fcbc003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.010+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbd004df50 con 0x7fcbd4108540 2024-04-23T18:17:12.834 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.010+0000 7fcbd9298700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 0x7fcbc003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.010+0000 7fcbd9298700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 0x7fcbc003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006fd0 tx=0x7fcbc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.015+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbd001f030 con 0x7fcbd4108540 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.264+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcbd4104d50 con 0x7fcbc0039560 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.271+0000 7fcbcaffd700 1 -- 172.21.15.33:0/1984472558 <== mgr.14120 v2:172.21.15.33:6800/4245296611 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcbd4104d50 con 0x7fcbc0039560 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 msgr2=0x7fcbc003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 0x7fcbc003ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006fd0 tx=0x7fcbc4006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 msgr2=0x7fcbd419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00119f0 tx=0x7fcbd0011ad0 comp rx=0 tx=0).stop 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 shutdown_connections 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcbc0039560 0x7fcbc003ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 --2- 172.21.15.33:0/1984472558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbd4108540 0x7fcbd419e5f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 >> 172.21.15.33:0/1984472558 conn(0x7fcbd4103ba0 msgr2=0x7fcbd4104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 shutdown_connections 2024-04-23T18:17:12.835 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:12.275+0000 7fcbdbcfd700 1 -- 172.21.15.33:0/1984472558 wait complete. 2024-04-23T18:17:14.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.272+0000 7f527332c700 1 Processor -- start 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.272+0000 7f527332c700 1 -- start start 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.272+0000 7f527332c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.272+0000 7f527332c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f526c107d80 con 0x7f526c107390 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.272+0000 7f52710c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f52710c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50364/0 (socket says 172.21.15.33:50364) 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f52710c8700 1 -- 172.21.15.33:0/1817234921 learned_addr learned my addr 172.21.15.33:0/1817234921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f52710c8700 1 -- 172.21.15.33:0/1817234921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c108590 con 0x7f526c107390 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f52710c8700 1 --2- 172.21.15.33:0/1817234921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52680098a0 tx=0x7f5268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82bf08858cc7d20 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f5263fff700 1 -- 172.21.15.33:0/1817234921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5268004160 con 0x7f526c107390 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f5263fff700 1 -- 172.21.15.33:0/1817234921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5268004300 con 0x7f526c107390 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.273+0000 7f5263fff700 1 -- 172.21.15.33:0/1817234921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5268011a20 con 0x7f526c107390 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 -- 172.21.15.33:0/1817234921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 msgr2=0x7f526c1077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:14.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 --2- 172.21.15.33:0/1817234921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52680098a0 tx=0x7f5268009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 -- 172.21.15.33:0/1817234921 shutdown_connections 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 --2- 172.21.15.33:0/1817234921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c1077b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 -- 172.21.15.33:0/1817234921 >> 172.21.15.33:0/1817234921 conn(0x7f526c1029b0 msgr2=0x7f526c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 -- 172.21.15.33:0/1817234921 shutdown_connections 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.274+0000 7f527332c700 1 -- 172.21.15.33:0/1817234921 wait complete. 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f527332c700 1 Processor -- start 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f527332c700 1 -- start start 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f527332c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f527332c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f526c07afe0 con 0x7f526c107390 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f52710c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f52710c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50378/0 (socket says 172.21.15.33:50378) 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.275+0000 7f52710c8700 1 -- 172.21.15.33:0/282072529 learned_addr learned my addr 172.21.15.33:0/282072529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:14.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f52710c8700 1 -- 172.21.15.33:0/282072529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19a750 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f52710c8700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52680119f0 tx=0x7f5268011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52680039e0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f526801b8f0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19a9e0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52680237a0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.276+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c19d6d0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.277+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f526802c3f0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.277+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c0640a0 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.277+0000 7f52627fc700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 0x7f525803dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.277+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f526804de70 con 0x7f526c107390 2024-04-23T18:17:14.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.278+0000 7f52708c7700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 0x7f525803dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.278+0000 7f52708c7700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 0x7f525803dca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f525c006fd0 tx=0x7f525c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.283+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f526801f030 con 0x7f526c107390 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.495+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f526c193770 con 0x7f526c107390 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.497+0000 7f52627fc700 1 -- 172.21.15.33:0/282072529 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f526c193770 con 0x7f526c107390 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 msgr2=0x7f525803dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 0x7f525803dca0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f525c006fd0 tx=0x7f525c006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 msgr2=0x7f526c19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52680119f0 tx=0x7f5268011ad0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 shutdown_connections 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f525803b7e0 0x7f525803dca0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 --2- 172.21.15.33:0/282072529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c107390 0x7f526c19a210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:14.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 >> 172.21.15.33:0/282072529 conn(0x7f526c1029b0 msgr2=0x7f526c190830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:14.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 shutdown_connections 2024-04-23T18:17:14.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:13.500+0000 7f527332c700 1 -- 172.21.15.33:0/282072529 wait complete. 2024-04-23T18:17:14.769 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:14 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/282072529' entity='client.admin' 2024-04-23T18:17:15.773 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.016+0000 7f0bc7a8a700 1 Processor -- start 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.016+0000 7f0bc7a8a700 1 -- start start 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc7a8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc7a8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc007be00 con 0x7f0bc0108fc0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc5826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc5826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50394/0 (socket says 172.21.15.33:50394) 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc5826700 1 -- 172.21.15.33:0/1231293769 learned_addr learned my addr 172.21.15.33:0/1231293769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc5826700 1 -- 172.21.15.33:0/1231293769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc010b8f0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc5826700 1 --2- 172.21.15.33:0/1231293769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098a0 tx=0x7f0bb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17f44334eb28296 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc4824700 1 -- 172.21.15.33:0/1231293769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0004160 con 0x7f0bc0108fc0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc4824700 1 -- 172.21.15.33:0/1231293769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb0004300 con 0x7f0bc0108fc0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.017+0000 7f0bc4824700 1 -- 172.21.15.33:0/1231293769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0011a20 con 0x7f0bc0108fc0 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/1231293769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 msgr2=0x7f0bc010b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:15.774 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/1231293769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098a0 tx=0x7f0bb0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/1231293769 shutdown_connections 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/1231293769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc010b3b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/1231293769 >> 172.21.15.33:0/1231293769 conn(0x7f0bc01029b0 msgr2=0x7f0bc0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/1231293769 shutdown_connections 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.018+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/1231293769 wait complete. 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc7a8a700 1 Processor -- start 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc7a8a700 1 -- start start 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc7a8a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc7a8a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010c840 con 0x7f0bc0108fc0 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc5826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc5826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50406/0 (socket says 172.21.15.33:50406) 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.019+0000 7f0bc5826700 1 -- 172.21.15.33:0/470540784 learned_addr learned my addr 172.21.15.33:0/470540784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bc5826700 1 -- 172.21.15.33:0/470540784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc019c940 con 0x7f0bc0108fc0 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bc5826700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0004660 tx=0x7f0bb0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:15.775 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0003aa0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bb001b8f0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc019cbd0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0bb0023900 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.020+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc019f8c0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.021+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0bb0023b70 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.021+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc00640a0 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.021+0000 7f0bb6ffd700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 0x7f0bac03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.021+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0bb004de30 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.021+0000 7f0bc5025700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 0x7f0bac03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.022+0000 7f0bc5025700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 0x7f0bac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.026+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bb001f030 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.242+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0bc019ff20 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.245+0000 7f0bb6ffd700 1 -- 172.21.15.33:0/470540784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0bb0048020 con 0x7f0bc0108fc0 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 msgr2=0x7f0bac03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:15.776 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 0x7f0bac03bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc006fd0 tx=0x7f0bbc006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 msgr2=0x7f0bc019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0004660 tx=0x7f0bb0011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 shutdown_connections 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f0bac039600 0x7f0bac03bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 --2- 172.21.15.33:0/470540784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bc0108fc0 0x7f0bc019c400 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 >> 172.21.15.33:0/470540784 conn(0x7f0bc01029b0 msgr2=0x7f0bc0103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 shutdown_connections 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:15.248+0000 7f0bc7a8a700 1 -- 172.21.15.33:0/470540784 wait complete. 2024-04-23T18:17:15.777 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-04-23T18:17:15.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:15 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:15.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:15 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/470540784' entity='client.admin' 2024-04-23T18:17:17.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:16 smithi033 ceph-mon[29274]: from='mgr.14120 172.21.15.33:0/3560161992' entity='mgr.x' 2024-04-23T18:17:17.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:16 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4107111587' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-23T18:17:17.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setuser ceph since I am not root 2024-04-23T18:17:17.066 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setgroup ceph since I am not root 2024-04-23T18:17:17.066 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:17.071+0000 7f1f59a82200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:17:17.359 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:17.263+0000 7f1f59a82200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:17.609 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:17.376+0000 7f1f59a82200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:17:17.885 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:17.695+0000 7f1f59a82200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:17:17.885 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:17.783+0000 7f1f59a82200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.206+0000 7fcec68fb700 1 Processor -- start 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.206+0000 7fcec68fb700 1 -- start start 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcec68fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcec68fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec0107d80 con 0x7fcec0107390 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50420/0 (socket says 172.21.15.33:50420) 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebffff700 1 -- 172.21.15.33:0/1989733647 learned_addr learned my addr 172.21.15.33:0/1989733647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebffff700 1 -- 172.21.15.33:0/1989733647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec0108590 con 0x7fcec0107390 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebffff700 1 --2- 172.21.15.33:0/1989733647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fceb00098a0 tx=0x7fceb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccc65716affcd18 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebeffd700 1 -- 172.21.15.33:0/1989733647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceb0004160 con 0x7fcec0107390 2024-04-23T18:17:18.046 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebeffd700 1 -- 172.21.15.33:0/1989733647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb0004300 con 0x7fcec0107390 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.207+0000 7fcebeffd700 1 -- 172.21.15.33:0/1989733647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceb0011a20 con 0x7fcec0107390 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 -- 172.21.15.33:0/1989733647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 msgr2=0x7fcec01077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 --2- 172.21.15.33:0/1989733647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fceb00098a0 tx=0x7fceb0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 -- 172.21.15.33:0/1989733647 shutdown_connections 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 --2- 172.21.15.33:0/1989733647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec01077b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 -- 172.21.15.33:0/1989733647 >> 172.21.15.33:0/1989733647 conn(0x7fcec01029b0 msgr2=0x7fcec0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 -- 172.21.15.33:0/1989733647 shutdown_connections 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.208+0000 7fcec68fb700 1 -- 172.21.15.33:0/1989733647 wait complete. 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcec68fb700 1 Processor -- start 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcec68fb700 1 -- start start 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcec68fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcec68fb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010b940 con 0x7fcec0107390 2024-04-23T18:17:18.047 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcebffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcebffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50428/0 (socket says 172.21.15.33:50428) 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.209+0000 7fcebffff700 1 -- 172.21.15.33:0/4107111587 learned_addr learned my addr 172.21.15.33:0/4107111587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcebffff700 1 -- 172.21.15.33:0/4107111587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec019cf80 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcebffff700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fceb00119f0 tx=0x7fceb0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceb00039e0 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec01a1590 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb001b8f0 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fceb00237a0 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.210+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec01a1930 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.211+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fceb0023aa0 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.211+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec00640a0 con 0x7fcec0107390 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.211+0000 7fcebd7fa700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 0x7fcea803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:18.048 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.211+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fceb004de90 con 0x7fcec0107390 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.211+0000 7fcebf7fe700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 0x7fcea803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.212+0000 7fcebf7fe700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 0x7fcea803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006fd0 tx=0x7fceb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.217+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fceb001f030 con 0x7fcec0107390 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.514+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fcec01a1df0 con 0x7fcec0107390 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.891+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fceb0018e30 con 0x7fcec0107390 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.892+0000 7fcebd7fa700 1 -- 172.21.15.33:0/4107111587 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fceb002c910 con 0x7fcec0107390 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 msgr2=0x7fcea803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 0x7fcea803ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fceb4006fd0 tx=0x7fceb4006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 msgr2=0x7fcec007ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fceb00119f0 tx=0x7fceb0011ad0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 shutdown_connections 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fcea8039560 0x7fcea803ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 --2- 172.21.15.33:0/4107111587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec0107390 0x7fcec007ba60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:18.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 >> 172.21.15.33:0/4107111587 conn(0x7fcec01029b0 msgr2=0x7fcec0103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:18.050 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 shutdown_connections 2024-04-23T18:17:18.050 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:16.894+0000 7fcec68fb700 1 -- 172.21.15.33:0/4107111587 wait complete. 2024-04-23T18:17:18.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4107111587' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-23T18:17:18.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:17 smithi033 ceph-mon[29274]: mgrmap e9: x(active, since 19s) 2024-04-23T18:17:19.052 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3927797325' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-23T18:17:19.317 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:19.058+0000 7f1f59a82200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:17:19.317 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:19.237+0000 7f1f59a82200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:19.327 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:17:19.327 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-23T18:17:19.327 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.482+0000 7f59a6e4b700 1 Processor -- start 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.482+0000 7f59a6e4b700 1 -- start start 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.482+0000 7f59a6e4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.482+0000 7f59a6e4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59a007bec0 con 0x7f59a0108f00 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f59a4be7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f59a4be7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51584/0 (socket says 172.21.15.33:51584) 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f59a4be7700 1 -- 172.21.15.33:0/3245397768 learned_addr learned my addr 172.21.15.33:0/3245397768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f59a4be7700 1 -- 172.21.15.33:0/3245397768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a010b830 con 0x7f59a0108f00 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f59a4be7700 1 --2- 172.21.15.33:0/3245397768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2940b3f308ca0adb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f599f7fe700 1 -- 172.21.15.33:0/3245397768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994004160 con 0x7f59a0108f00 2024-04-23T18:17:19.328 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f599f7fe700 1 -- 172.21.15.33:0/3245397768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5994004300 con 0x7f59a0108f00 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.483+0000 7f599f7fe700 1 -- 172.21.15.33:0/3245397768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994011a20 con 0x7f59a0108f00 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3245397768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 msgr2=0x7f59a010b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3245397768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59940098a0 tx=0x7f5994009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3245397768 shutdown_connections 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3245397768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a010b2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3245397768 >> 172.21.15.33:0/3245397768 conn(0x7f59a01029b0 msgr2=0x7f59a0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3245397768 shutdown_connections 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.484+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3245397768 wait complete. 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a6e4b700 1 Processor -- start 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a6e4b700 1 -- start start 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a6e4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a6e4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59a010c840 con 0x7f59a0108f00 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a4be7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a4be7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51592/0 (socket says 172.21.15.33:51592) 2024-04-23T18:17:19.329 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.485+0000 7f59a4be7700 1 -- 172.21.15.33:0/3927797325 learned_addr learned my addr 172.21.15.33:0/3927797325 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f59a4be7700 1 -- 172.21.15.33:0/3927797325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a019c900 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f59a4be7700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5994004660 tx=0x7f5994011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994003aa0 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a019cb90 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599401b8f0 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5994023900 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.486+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a019f880 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5994023b70 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a00640a0 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f599dffb700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 0x7f599003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f599404ee20 con 0x7f59a0108f00 2024-04-23T18:17:19.330 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f599ffff700 1 -- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 msgr2=0x7f599003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.487+0000 7f599ffff700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 0x7f599003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.492+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f599401f030 con 0x7f59a0108f00 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.688+0000 7f599ffff700 1 -- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 msgr2=0x7f599003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.688+0000 7f599ffff700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 0x7f599003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.762+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f59a019fa50 con 0x7f59a0108f00 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.762+0000 7f599dffb700 1 -- 172.21.15.33:0/3927797325 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5994010380 con 0x7f59a0108f00 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 msgr2=0x7f599003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 0x7f599003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 msgr2=0x7f59a019c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5994004660 tx=0x7f5994011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 shutdown_connections 2024-04-23T18:17:19.331 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7f5990039660 0x7f599003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 --2- 172.21.15.33:0/3927797325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59a0108f00 0x7f59a019c3c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 >> 172.21.15.33:0/3927797325 conn(0x7f59a01029b0 msgr2=0x7f59a01034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 shutdown_connections 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:18.764+0000 7f59a6e4b700 1 -- 172.21.15.33:0/3927797325 wait complete. 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-23T18:17:19.332 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-04-23T18:17:19.588 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:19.423+0000 7f1f59a82200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:17:19.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:19.730+0000 7f1f59a82200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:17:19.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:19.830+0000 7f1f59a82200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:17:20.429 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:20.043+0000 7f1f59a82200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:17:20.430 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:20.246+0000 7f1f59a82200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:17:21.230 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:20.875+0000 7f1f59a82200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:17:21.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:21.319+0000 7f1f59a82200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:17:21.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:21.419+0000 7f1f59a82200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:17:21.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:21.631+0000 7f1f59a82200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:17:22.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.177+0000 7f1f59a82200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:17:22.481 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.274+0000 7f1f59a82200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:17:22.832 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.529+0000 7f1f59a82200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:17:22.832 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.668+0000 7f1f59a82200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:17:22.832 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.770+0000 7f1f59a82200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:17:23.201 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:22.893+0000 7f1f59a82200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:17:23.201 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:23.207+0000 7f1f59a82200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:17:24.112 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:23.837+0000 7f1f59a82200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:17:24.113 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:17:24.065+0000 7f1f59a82200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: Active manager daemon x restarted 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: Activating manager daemon x 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: osdmap e3: 0 total, 0 up, 0 in 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: mgrmap e10: x(active, starting, since 0.00314938s) 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:17:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:17:24.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:17:24.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:17:24.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: Manager daemon x is now available 2024-04-23T18:17:24.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: [23/Apr/2024:18:17:24] ENGINE Bus STARTING 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: [23/Apr/2024:18:17:24] ENGINE Serving on https://172.21.15.33:7150 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: [23/Apr/2024:18:17:24] ENGINE Serving on http://172.21.15.33:8765 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: [23/Apr/2024:18:17:24] ENGINE Bus STARTED 2024-04-23T18:17:25.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:25 smithi033 ceph-mon[29274]: mgrmap e11: x(active, since 1.00595s) 2024-04-23T18:17:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-23T18:17:25.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcb6826700 1 Processor -- start 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcb6826700 1 -- start start 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcb6826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcb6826700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0108ed0 con 0x7fbcb01084e0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51602/0 (socket says 172.21.15.33:51602) 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcaffff700 1 -- 172.21.15.33:0/324676537 learned_addr learned my addr 172.21.15.33:0/324676537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.829+0000 7fbcaffff700 1 -- 172.21.15.33:0/324676537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb0109690 con 0x7fbcb01084e0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.830+0000 7fbcaffff700 1 --2- 172.21.15.33:0/324676537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbca00098a0 tx=0x7fbca0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823dcae8c4b3f1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.830+0000 7fbcaeffd700 1 -- 172.21.15.33:0/324676537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0004160 con 0x7fbcb01084e0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.830+0000 7fbcaeffd700 1 -- 172.21.15.33:0/324676537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca0004300 con 0x7fbcb01084e0 2024-04-23T18:17:25.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.830+0000 7fbcaeffd700 1 -- 172.21.15.33:0/324676537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0011a20 con 0x7fbcb01084e0 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 -- 172.21.15.33:0/324676537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 msgr2=0x7fbcb0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 --2- 172.21.15.33:0/324676537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbca00098a0 tx=0x7fbca0009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 -- 172.21.15.33:0/324676537 shutdown_connections 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 --2- 172.21.15.33:0/324676537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb0108900 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 -- 172.21.15.33:0/324676537 >> 172.21.15.33:0/324676537 conn(0x7fbcb0103b40 msgr2=0x7fbcb0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 -- 172.21.15.33:0/324676537 shutdown_connections 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.831+0000 7fbcb6826700 1 -- 172.21.15.33:0/324676537 wait complete. 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcb6826700 1 Processor -- start 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcb6826700 1 -- start start 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcb6826700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcb6826700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010ca40 con 0x7fbcb01084e0 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:25.906 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcaffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51606/0 (socket says 172.21.15.33:51606) 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcaffff700 1 -- 172.21.15.33:0/1129970335 learned_addr learned my addr 172.21.15.33:0/1129970335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.832+0000 7fbcaffff700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb019c940 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcaffff700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbca0000c00 tx=0x7fbca0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0003aa0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca001b8f0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbca0023890 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb019cbd0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.833+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb019f8c0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fbca002c3f0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcad7fa700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbc9803c1d0 con 0x7fbc98039600 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbca004e0a0 con 0x7fbcb01084e0 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:19.834+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:17:25.907 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:20.035+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:20.035+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:20.436+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:20.436+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:21.237+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:21.237+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:22.838+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4245296611 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:22.838+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:24.069+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fbca00193d0 con 0x7fbcb01084e0 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:24.069+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 msgr2=0x7fbc9803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:24.069+0000 7fbcad7fa700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.070+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fbca000bd00 con 0x7fbcb01084e0 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.071+0000 7fbcad7fa700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 0x7fbc9803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.071+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbc9803c1d0 con 0x7fbc9803cfe0 2024-04-23T18:17:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.071+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 0x7fbc9803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.072+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 0x7fbc9803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbca4003a10 tx=0x7fbca40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.072+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fbc9803c1d0 con 0x7fbc9803cfe0 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.079+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fbcb0069b90 con 0x7fbc9803cfe0 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.079+0000 7fbcad7fa700 1 -- 172.21.15.33:0/1129970335 <== mgr.14152 v2:172.21.15.33:6800/2158874591 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fbcb0069b90 con 0x7fbc9803cfe0 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 msgr2=0x7fbc9803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 0x7fbc9803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbca4003a10 tx=0x7fbca40092b0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 msgr2=0x7fbcb019c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbca0000c00 tx=0x7fbca0011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 shutdown_connections 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbc9803cfe0 0x7fbc9803f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcb01084e0 0x7fbcb019c400 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 --2- 172.21.15.33:0/1129970335 >> [v2:172.21.15.33:6800/4245296611,v1:172.21.15.33:6801/4245296611] conn(0x7fbc98039600 0x7fbc9803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 >> 172.21.15.33:0/1129970335 conn(0x7fbcb0103b40 msgr2=0x7fbcb01046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 shutdown_connections 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:25.080+0000 7fbcb6826700 1 -- 172.21.15.33:0/1129970335 wait complete. 2024-04-23T18:17:25.909 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-04-23T18:17:25.910 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-04-23T18:17:26.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:26 smithi033 ceph-mon[29274]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-23T18:17:26.147 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:26 smithi033 ceph-mon[29274]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.338+0000 7fe69019f700 1 Processor -- start 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe69019f700 1 -- start start 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe69019f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe69019f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe688108f00 con 0x7fe688108510 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe68df3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:27.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe68df3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51660/0 (socket says 172.21.15.33:51660) 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe68df3b700 1 -- 172.21.15.33:0/2104177858 learned_addr learned my addr 172.21.15.33:0/2104177858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.339+0000 7fe68df3b700 1 -- 172.21.15.33:0/2104177858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6881096c0 con 0x7fe688108510 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe68df3b700 1 --2- 172.21.15.33:0/2104177858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098a0 tx=0x7fe684009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57b3e81c6a0030c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe68cf39700 1 -- 172.21.15.33:0/2104177858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684004160 con 0x7fe688108510 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe68cf39700 1 -- 172.21.15.33:0/2104177858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684004300 con 0x7fe688108510 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe68cf39700 1 -- 172.21.15.33:0/2104177858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684011a20 con 0x7fe688108510 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 -- 172.21.15.33:0/2104177858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 msgr2=0x7fe688108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 --2- 172.21.15.33:0/2104177858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098a0 tx=0x7fe684009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 -- 172.21.15.33:0/2104177858 shutdown_connections 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 --2- 172.21.15.33:0/2104177858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe688108930 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 -- 172.21.15.33:0/2104177858 >> 172.21.15.33:0/2104177858 conn(0x7fe688103b70 msgr2=0x7fe688105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.340+0000 7fe69019f700 1 -- 172.21.15.33:0/2104177858 shutdown_connections 2024-04-23T18:17:27.208 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe69019f700 1 -- 172.21.15.33:0/2104177858 wait complete. 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe69019f700 1 Processor -- start 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe69019f700 1 -- start start 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe69019f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe69019f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe68810ca70 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.341+0000 7fe68df3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe68df3b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51676/0 (socket says 172.21.15.33:51676) 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe68df3b700 1 -- 172.21.15.33:0/2325572123 learned_addr learned my addr 172.21.15.33:0/2325572123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe68df3b700 1 -- 172.21.15.33:0/2325572123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68819cc00 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe68df3b700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6840116e0 tx=0x7fe684011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6840039e0 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68401b8f0 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68819ce90 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684023760 con 0x7fe688108510 2024-04-23T18:17:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.342+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68819fb80 con 0x7fe688108510 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.343+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fe684023900 con 0x7fe688108510 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.343+0000 7fe67effd700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 0x7fe67403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.343+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe68404e070 con 0x7fe688108510 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.343+0000 7fe68d73a700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 0x7fe67403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.343+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6880640a0 con 0x7fe688108510 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.344+0000 7fe68d73a700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 0x7fe67403bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.349+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe68401f030 con 0x7fe688108510 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.568+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe688062e80 con 0x7fe674039610 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.637+0000 7fe67effd700 1 -- 172.21.15.33:0/2325572123 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe688062e80 con 0x7fe674039610 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 msgr2=0x7fe67403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 0x7fe67403bad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe678006fd0 tx=0x7fe678006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 msgr2=0x7fe68819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe6840116e0 tx=0x7fe684011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 shutdown_connections 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe674039610 0x7fe67403bad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 --2- 172.21.15.33:0/2325572123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe688108510 0x7fe68819c6c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 >> 172.21.15.33:0/2325572123 conn(0x7fe688103b70 msgr2=0x7fe6881047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 shutdown_connections 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:26.641+0000 7fe69019f700 1 -- 172.21.15.33:0/2325572123 wait complete. 2024-04-23T18:17:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-04-23T18:17:27.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:27 smithi033 ceph-mon[29274]: mgrmap e12: x(active, since 2s) 2024-04-23T18:17:27.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:27 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:27.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:27 smithi033 ceph-mon[29274]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:27.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:27 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:27.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:27 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:28.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:28 smithi033 ceph-mon[29274]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vqX3ckHDzCIdI4xevyJ5a.xu8aZevBmy6KAxpnlVAZA.LVxPE93by", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713896248, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.640+0000 7fef9fb14700 1 Processor -- start 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.640+0000 7fef9fb14700 1 -- start start 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9fb14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9fb14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef9807c8e0 con 0x7fef9807de60 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9d8b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9d8b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59752/0 (socket says 172.21.15.33:59752) 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9d8b0700 1 -- 172.21.15.33:0/682305112 learned_addr learned my addr 172.21.15.33:0/682305112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9d8b0700 1 -- 172.21.15.33:0/682305112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9807ca60 con 0x7fef9807de60 2024-04-23T18:17:28.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9d8b0700 1 --2- 172.21.15.33:0/682305112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef940098a0 tx=0x7fef94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d323eb3a6af126 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9c8ae700 1 -- 172.21.15.33:0/682305112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef94004160 con 0x7fef9807de60 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9c8ae700 1 -- 172.21.15.33:0/682305112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef94004300 con 0x7fef9807de60 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.641+0000 7fef9c8ae700 1 -- 172.21.15.33:0/682305112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef94011a20 con 0x7fef9807de60 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 -- 172.21.15.33:0/682305112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 msgr2=0x7fef9807c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 --2- 172.21.15.33:0/682305112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef940098a0 tx=0x7fef94009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 -- 172.21.15.33:0/682305112 shutdown_connections 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 --2- 172.21.15.33:0/682305112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9807c310 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 -- 172.21.15.33:0/682305112 >> 172.21.15.33:0/682305112 conn(0x7fef98103b70 msgr2=0x7fef98105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 -- 172.21.15.33:0/682305112 shutdown_connections 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.642+0000 7fef9fb14700 1 -- 172.21.15.33:0/682305112 wait complete. 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9fb14700 1 Processor -- start 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9fb14700 1 -- start start 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9fb14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9fb14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef9810eab0 con 0x7fef9807de60 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9d8b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:28.789 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9d8b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59762/0 (socket says 172.21.15.33:59762) 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9d8b0700 1 -- 172.21.15.33:0/451082431 learned_addr learned my addr 172.21.15.33:0/451082431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9d8b0700 1 -- 172.21.15.33:0/451082431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9819edd0 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.643+0000 7fef9d8b0700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef940116e0 tx=0x7fef94011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef94003a60 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9401b8f0 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9819f060 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef94019860 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef981a1d50 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.644+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fef94019b60 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.645+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef980640a0 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.645+0000 7fef8effd700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 0x7fef8403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.645+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef9404de40 con 0x7fef9807de60 2024-04-23T18:17:28.790 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.645+0000 7fef9d0af700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 0x7fef8403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.645+0000 7fef9d0af700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 0x7fef8403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.650+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef9401f030 con 0x7fef9807de60 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:27.900+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fef98196d90 con 0x7fef840395b0 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.147+0000 7fef8effd700 1 -- 172.21.15.33:0/451082431 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fef98196d90 con 0x7fef840395b0 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 msgr2=0x7fef8403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 0x7fef8403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fef88006fd0 tx=0x7fef88006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 msgr2=0x7fef9819e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fef940116e0 tx=0x7fef94011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 shutdown_connections 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fef840395b0 0x7fef8403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 --2- 172.21.15.33:0/451082431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef9807de60 0x7fef9819e890 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.149+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 >> 172.21.15.33:0/451082431 conn(0x7fef98103b70 msgr2=0x7fef98104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.150+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 shutdown_connections 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:28.150+0000 7fef9fb14700 1 -- 172.21.15.33:0/451082431 wait complete. 2024-04-23T18:17:28.791 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-04-23T18:17:29.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:29.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:30.326 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:30 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/779605523' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f9358a13700 1 Processor -- start 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f9358a13700 1 -- start start 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f9358a13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f9358a13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9354108f30 con 0x7f9354108540 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f935259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f935259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59764/0 (socket says 172.21.15.33:59764) 2024-04-23T18:17:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.203+0000 7f935259c700 1 -- 172.21.15.33:0/3441771632 learned_addr learned my addr 172.21.15.33:0/3441771632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.204+0000 7f935259c700 1 -- 172.21.15.33:0/3441771632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93541096f0 con 0x7f9354108540 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.204+0000 7f935259c700 1 --2- 172.21.15.33:0/3441771632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93440098a0 tx=0x7f9344009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3647e315f9fca156 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.204+0000 7f935159a700 1 -- 172.21.15.33:0/3441771632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9344004160 con 0x7f9354108540 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.204+0000 7f935159a700 1 -- 172.21.15.33:0/3441771632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9344004300 con 0x7f9354108540 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.204+0000 7f935159a700 1 -- 172.21.15.33:0/3441771632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9344011a20 con 0x7f9354108540 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 -- 172.21.15.33:0/3441771632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 msgr2=0x7f9354108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 --2- 172.21.15.33:0/3441771632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93440098a0 tx=0x7f9344009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 -- 172.21.15.33:0/3441771632 shutdown_connections 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 --2- 172.21.15.33:0/3441771632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f9354108960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 -- 172.21.15.33:0/3441771632 >> 172.21.15.33:0/3441771632 conn(0x7f9354103ba0 msgr2=0x7f9354105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 -- 172.21.15.33:0/3441771632 shutdown_connections 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.205+0000 7f9358a13700 1 -- 172.21.15.33:0/3441771632 wait complete. 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.206+0000 7f9358a13700 1 Processor -- start 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.206+0000 7f9358a13700 1 -- start start 2024-04-23T18:17:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.206+0000 7f9358a13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f9358a13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f935410caa0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f935259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f935259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59770/0 (socket says 172.21.15.33:59770) 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f935259c700 1 -- 172.21.15.33:0/779605523 learned_addr learned my addr 172.21.15.33:0/779605523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f935259c700 1 -- 172.21.15.33:0/779605523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935419c930 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f935259c700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93440119f0 tx=0x7f9344011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93440039e0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934401b8f0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.207+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935419cbc0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.208+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935419f8b0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.208+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93440237a0 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.208+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f9344023a50 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.209+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935410cc20 con 0x7f9354108540 2024-04-23T18:17:30.336 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.209+0000 7f93437fe700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 0x7f933c03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.209+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f934401b4d0 con 0x7f9354108540 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.209+0000 7f9351d9b700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 0x7f933c03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.209+0000 7f9351d9b700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 0x7f933c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9348006fd0 tx=0x7f9348006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.214+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f934401f030 con 0x7f9354108540 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.448+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f935419fe80 con 0x7f9354108540 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.449+0000 7f93437fe700 1 -- 172.21.15.33:0/779605523 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9344018e00 con 0x7f9354108540 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 msgr2=0x7f933c03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 0x7f933c03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9348006fd0 tx=0x7f9348006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 msgr2=0x7f935419c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93440119f0 tx=0x7f9344011ad0 comp rx=0 tx=0).stop 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 shutdown_connections 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f933c039610 0x7f933c03bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 --2- 172.21.15.33:0/779605523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9354108540 0x7f935419c3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:30.337 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 >> 172.21.15.33:0/779605523 conn(0x7f9354103ba0 msgr2=0x7f93541046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:30.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 shutdown_connections 2024-04-23T18:17:30.338 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:29.451+0000 7f9358a13700 1 -- 172.21.15.33:0/779605523 wait complete. 2024-04-23T18:17:30.342 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-23T18:17:30.342 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-23T18:17:30.349 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-23T18:17:30.350 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-23T18:17:30.350 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-23T18:17:30.350 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033:8443/ 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: Password: 4m7u9n3h8e 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config directory 2024-04-23T18:17:30.352 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-04-23T18:17:31.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:31 smithi033 ceph-mon[29274]: mgrmap e13: x(active, since 6s) 2024-04-23T18:17:31.552 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.753+0000 7feeaf929700 1 Processor -- start 2024-04-23T18:17:31.552 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.753+0000 7feeaf929700 1 -- start start 2024-04-23T18:17:31.552 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.753+0000 7feeaf929700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feeaf929700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feea807be60 con 0x7feea8106d50 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feead6c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feead6c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59786/0 (socket says 172.21.15.33:59786) 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feead6c5700 1 -- 172.21.15.33:0/2951870556 learned_addr learned my addr 172.21.15.33:0/2951870556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feead6c5700 1 -- 172.21.15.33:0/2951870556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea81076b0 con 0x7feea8106d50 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feead6c5700 1 --2- 172.21.15.33:0/2951870556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee980098a0 tx=0x7fee98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81a8c722d8eaf2b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feea7fff700 1 -- 172.21.15.33:0/2951870556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee98004160 con 0x7feea8106d50 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feea7fff700 1 -- 172.21.15.33:0/2951870556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee98004300 con 0x7feea8106d50 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.754+0000 7feea7fff700 1 -- 172.21.15.33:0/2951870556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee98011a20 con 0x7feea8106d50 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 -- 172.21.15.33:0/2951870556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 msgr2=0x7feea8107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 --2- 172.21.15.33:0/2951870556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fee980098a0 tx=0x7fee98009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 -- 172.21.15.33:0/2951870556 shutdown_connections 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 --2- 172.21.15.33:0/2951870556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea8107170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:31.553 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 -- 172.21.15.33:0/2951870556 >> 172.21.15.33:0/2951870556 conn(0x7feea81029b0 msgr2=0x7feea8104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 -- 172.21.15.33:0/2951870556 shutdown_connections 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.755+0000 7feeaf929700 1 -- 172.21.15.33:0/2951870556 wait complete. 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feeaf929700 1 Processor -- start 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feeaf929700 1 -- start start 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feeaf929700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feeaf929700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feea8108720 con 0x7feea8106d50 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feead6c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feead6c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59794/0 (socket says 172.21.15.33:59794) 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feead6c5700 1 -- 172.21.15.33:0/2743616331 learned_addr learned my addr 172.21.15.33:0/2743616331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.756+0000 7feead6c5700 1 -- 172.21.15.33:0/2743616331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea819eb30 con 0x7feea8106d50 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feead6c5700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee98004660 tx=0x7fee98011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee98003aa0 con 0x7feea8106d50 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9801b8f0 con 0x7feea8106d50 2024-04-23T18:17:31.554 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea819edc0 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee98023900 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.757+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81a1ab0 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fee98023b70 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea80640a0 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feea67fc700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 0x7fee9003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fee9804e130 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feeacec4700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 0x7fee9003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.758+0000 7feeacec4700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 0x7fee9003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee9c006fd0 tx=0x7fee9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.763+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee9801f030 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.972+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7feea802e260 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.972+0000 7feea67fc700 1 -- 172.21.15.33:0/2743616331 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fee9801bde0 con 0x7feea8106d50 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 msgr2=0x7fee9003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 0x7fee9003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee9c006fd0 tx=0x7fee9c006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 msgr2=0x7feea819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:31.555 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fee98004660 tx=0x7fee98011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 shutdown_connections 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fee90039700 0x7fee9003bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 --2- 172.21.15.33:0/2743616331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feea8106d50 0x7feea819e5f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 >> 172.21.15.33:0/2743616331 conn(0x7feea81029b0 msgr2=0x7feea8195260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 shutdown_connections 2024-04-23T18:17:31.556 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:30.975+0000 7feeaf929700 1 -- 172.21.15.33:0/2743616331 wait complete. 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.962+0000 7f384616a700 1 Processor -- start 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.962+0000 7f384616a700 1 -- start start 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f384616a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f384616a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f384007c8e0 con 0x7f384007de60 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59802/0 (socket says 172.21.15.33:59802) 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383f7fe700 1 -- 172.21.15.33:0/3579302621 learned_addr learned my addr 172.21.15.33:0/3579302621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383f7fe700 1 -- 172.21.15.33:0/3579302621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384007ca60 con 0x7f384007de60 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383f7fe700 1 --2- 172.21.15.33:0/3579302621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38300098a0 tx=0x7f3830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c65ac91fcb35974 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383e7fc700 1 -- 172.21.15.33:0/3579302621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3830004160 con 0x7f384007de60 2024-04-23T18:17:32.826 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383e7fc700 1 -- 172.21.15.33:0/3579302621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3830004300 con 0x7f384007de60 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.963+0000 7f383e7fc700 1 -- 172.21.15.33:0/3579302621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3830011a20 con 0x7f384007de60 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 -- 172.21.15.33:0/3579302621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 msgr2=0x7f384007c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 --2- 172.21.15.33:0/3579302621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f38300098a0 tx=0x7f3830009bb0 comp rx=0 tx=0).stop 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 -- 172.21.15.33:0/3579302621 shutdown_connections 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 --2- 172.21.15.33:0/3579302621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384007c310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 -- 172.21.15.33:0/3579302621 >> 172.21.15.33:0/3579302621 conn(0x7f3840103ab0 msgr2=0x7f3840105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 -- 172.21.15.33:0/3579302621 shutdown_connections 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.964+0000 7f384616a700 1 -- 172.21.15.33:0/3579302621 wait complete. 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f384616a700 1 Processor -- start 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f384616a700 1 -- start start 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f384616a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f384616a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f384010ea90 con 0x7f384007de60 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f383f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59814/0 (socket says 172.21.15.33:59814) 2024-04-23T18:17:32.827 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f383f7fe700 1 -- 172.21.15.33:0/1948660895 learned_addr learned my addr 172.21.15.33:0/1948660895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.965+0000 7f383f7fe700 1 -- 172.21.15.33:0/1948660895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384019ee30 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f383f7fe700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3830004660 tx=0x7f3830011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3830003aa0 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f383001b8f0 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384019f0c0 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3830023900 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.966+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a1db0 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.967+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3830023b70 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.967+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38400640a0 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.967+0000 7f383cff9700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 0x7f382803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.967+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f383004e110 con 0x7f384007de60 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.967+0000 7f383effd700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 0x7f382803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.968+0000 7f383effd700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 0x7f382803bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:32.828 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:31.972+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f383001f030 con 0x7f384007de60 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.297+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f384002df80 con 0x7f384007de60 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.299+0000 7f383cff9700 1 -- 172.21.15.33:0/1948660895 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f383000b3e0 con 0x7f384007de60 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 msgr2=0x7f382803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 0x7f382803bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3834006fd0 tx=0x7f3834006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 msgr2=0x7f384019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3830004660 tx=0x7f3830011aa0 comp rx=0 tx=0).stop 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 shutdown_connections 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3828039700 0x7f382803bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 --2- 172.21.15.33:0/1948660895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f384007de60 0x7f384019e8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 >> 172.21.15.33:0/1948660895 conn(0x7f3840103ab0 msgr2=0x7f3840104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 shutdown_connections 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-23T18:17:32.302+0000 7f384616a700 1 -- 172.21.15.33:0/1948660895 wait complete. 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-23T18:17:32.829 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:32.830 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-04-23T18:17:32.877 INFO:tasks.cephadm:Fetching config... 2024-04-23T18:17:32.878 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:17:32.878 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-23T18:17:32.895 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-23T18:17:32.895 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:17:32.895 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-23T18:17:32.950 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-23T18:17:32.951 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:17:32.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-23T18:17:33.016 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-23T18:17:33.016 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:17:33.016 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-23T18:17:33.072 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-23T18:17:33.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:17:33.148 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:33.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:17:33.197 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:33.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-23T18:17:33.248 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdnWj4Xxqv2Uv2pjuIf7ludEeZjTI+lraICaHVYA6etgkXwh1x6QJQy2/kVYkpYlEzvNHT9elx6Fn0yH6AwhZnEFUekUruf9ecmBtv5xB8SL55UadT+DVejvfozGi5q8aFm0EzR8XVgHIJZRLNDx9xz0xouJ1+XKUJNx7j5DlVAec5AyMIXPa61qG+DOR5RKuMydE7tsnuZnOTar7jHqZpgRZtuNhbecLpdWIILWlinlFbxEaI92cbk/2u79jlPyG+dCBWsQq2eHxm3JDnK3Akqaj/1vgIbKBvt42O1iFyygAjzsjvyALBREJunZWsKy5nDSOsSEw6N8htS+XIJDGcfG3JmK6FbtyBMGgCiNYxyw33epb/VSNgKjANITmAAitoa1RPrbrob7Mxd4widNSM75xatbyn5rhpnE3px5jeQse3Fu8xQn86PWQBAwnackcKrwoxYfI3yaHMV3yKNQeTb0/69Ali9WaVgMllDVA5t5RzKkEagGLnOrb+zQyeArc= ceph-766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:17:33.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-23T18:17:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1948660895' entity='client.admin' 2024-04-23T18:17:33.509 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- 172.21.15.33:0/2339071980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 msgr2=0x7f31ac102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 --2- 172.21.15.33:0/2339071980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac102900 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f319c009910 tx=0x7f319c009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- 172.21.15.33:0/2339071980 shutdown_connections 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 --2- 172.21.15.33:0/2339071980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac102900 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- 172.21.15.33:0/2339071980 >> 172.21.15.33:0/2339071980 conn(0x7f31ac0fda40 msgr2=0x7f31ac0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- 172.21.15.33:0/2339071980 shutdown_connections 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- 172.21.15.33:0/2339071980 wait complete. 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 Processor -- start 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.968+0000 7f31b2795700 1 -- start start 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31b2795700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31b2795700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac108420 con 0x7f31ac1024e0 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31abfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59832/0 (socket says 172.21.15.33:59832) 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31abfff700 1 -- 172.21.15.33:0/1232792364 learned_addr learned my addr 172.21.15.33:0/1232792364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31abfff700 1 -- 172.21.15.33:0/1232792364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac076bf0 con 0x7f31ac1024e0 2024-04-23T18:17:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31abfff700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f319c005950 tx=0x7f319c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c026070 con 0x7f31ac1024e0 2024-04-23T18:17:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f319c00bc00 con 0x7f31ac1024e0 2024-04-23T18:17:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac074e30 con 0x7f31ac1024e0 2024-04-23T18:17:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f319c003a10 con 0x7f31ac1024e0 2024-04-23T18:17:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.969+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac075310 con 0x7f31ac1024e0 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.970+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f319c029070 con 0x7f31ac1024e0 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.970+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac108ba0 con 0x7f31ac1024e0 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.970+0000 7f31a97fa700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 0x7f319403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.971+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f319c01e070 con 0x7f31ac1024e0 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.971+0000 7f31ab7fe700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 0x7f319403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:34.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.971+0000 7f31ab7fe700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 0x7f319403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31a0006fd0 tx=0x7f31a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:34.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:34.976+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f319c013070 con 0x7f31ac1024e0 2024-04-23T18:17:35.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.183+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f31ac190e80 con 0x7f31ac1024e0 2024-04-23T18:17:35.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.186+0000 7f31a97fa700 1 -- 172.21.15.33:0/1232792364 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f319c030020 con 0x7f31ac1024e0 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 msgr2=0x7f319403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 0x7f319403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f31a0006fd0 tx=0x7f31a0006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 msgr2=0x7f31ac0766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f319c005950 tx=0x7f319c004fa0 comp rx=0 tx=0).stop 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 shutdown_connections 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3194039650 0x7f319403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 --2- 172.21.15.33:0/1232792364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31ac1024e0 0x7f31ac0766b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.188+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 >> 172.21.15.33:0/1232792364 conn(0x7f31ac0fda40 msgr2=0x7f31ac0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.189+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 shutdown_connections 2024-04-23T18:17:35.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:35.189+0000 7f31b2795700 1 -- 172.21.15.33:0/1232792364 wait complete. 2024-04-23T18:17:36.141 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-23T18:17:36.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-23T18:17:36.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:36 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1232792364' entity='client.admin' 2024-04-23T18:17:36.678 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:38.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 -- 172.21.15.33:0/788494704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 msgr2=0x7f3ce0102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 --2- 172.21.15.33:0/788494704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce0102900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009910 tx=0x7f3cdc009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 -- 172.21.15.33:0/788494704 shutdown_connections 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 --2- 172.21.15.33:0/788494704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce0102900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 -- 172.21.15.33:0/788494704 >> 172.21.15.33:0/788494704 conn(0x7f3ce00fda40 msgr2=0x7f3ce00ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 -- 172.21.15.33:0/788494704 shutdown_connections 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 -- 172.21.15.33:0/788494704 wait complete. 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.736+0000 7f3ce808b700 1 Processor -- start 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce808b700 1 -- start start 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce808b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce808b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0108420 con 0x7f3ce01024e0 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce5e27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce5e27700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59492/0 (socket says 172.21.15.33:59492) 2024-04-23T18:17:38.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce5e27700 1 -- 172.21.15.33:0/2286102553 learned_addr learned my addr 172.21.15.33:0/2286102553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:38.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce5e27700 1 -- 172.21.15.33:0/2286102553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce0076bf0 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce5e27700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc005950 tx=0x7f3cdc004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cdc026070 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cdc00bc00 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.737+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce0074e30 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.738+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3cdc003a10 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.738+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce0075310 con 0x7f3ce01024e0 2024-04-23T18:17:38.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.738+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3cdc029070 con 0x7f3ce01024e0 2024-04-23T18:17:38.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.738+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0108ba0 con 0x7f3ce01024e0 2024-04-23T18:17:38.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.739+0000 7f3cd6ffd700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 0x7f3ccc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:38.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.739+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3cdc01e070 con 0x7f3ce01024e0 2024-04-23T18:17:38.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.739+0000 7f3ce5626700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 0x7f3ccc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:38.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.739+0000 7f3ce5626700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 0x7f3ccc03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0006fd0 tx=0x7f3cd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:38.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.744+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cdc013070 con 0x7f3ce01024e0 2024-04-23T18:17:38.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.960+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3ce0190e80 con 0x7f3ccc0396b0 2024-04-23T18:17:38.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.963+0000 7f3cd6ffd700 1 -- 172.21.15.33:0/2286102553 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3ce0190e80 con 0x7f3ccc0396b0 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 msgr2=0x7f3ccc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 0x7f3ccc03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0006fd0 tx=0x7f3cd0006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 msgr2=0x7f3ce00766b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc005950 tx=0x7f3cdc004fa0 comp rx=0 tx=0).stop 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 shutdown_connections 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3ccc0396b0 0x7f3ccc03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 --2- 172.21.15.33:0/2286102553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ce01024e0 0x7f3ce00766b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 >> 172.21.15.33:0/2286102553 conn(0x7f3ce00fda40 msgr2=0x7f3ce00fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 shutdown_connections 2024-04-23T18:17:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:38.965+0000 7f3ce808b700 1 -- 172.21.15.33:0/2286102553 wait complete. 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:39.783 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-04-23T18:17:39.783 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:17:39.783 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:39.802 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:17:39.802 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:39.860 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-04-23T18:17:39.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch host add smithi106 2024-04-23T18:17:40.101 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:40.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:40.222 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:40.222 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:40.222 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 -- 172.21.15.33:0/3852739688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 msgr2=0x7f10841088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 --2- 172.21.15.33:0/3852739688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f10841088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1080009910 tx=0x7f1080009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 -- 172.21.15.33:0/3852739688 shutdown_connections 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 --2- 172.21.15.33:0/3852739688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f10841088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 -- 172.21.15.33:0/3852739688 >> 172.21.15.33:0/3852739688 conn(0x7f1084076ea0 msgr2=0x7f10840772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 -- 172.21.15.33:0/3852739688 shutdown_connections 2024-04-23T18:17:41.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.586+0000 7f108be8f700 1 -- 172.21.15.33:0/3852739688 wait complete. 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f108be8f700 1 Processor -- start 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f108be8f700 1 -- start start 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f108be8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f108be8f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f108410c920 con 0x7f10841084e0 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f1089c2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f1089c2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59506/0 (socket says 172.21.15.33:59506) 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.587+0000 7f1089c2b700 1 -- 172.21.15.33:0/2566557870 learned_addr learned my addr 172.21.15.33:0/2566557870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:41.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f1089c2b700 1 -- 172.21.15.33:0/2566557870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1084198400 con 0x7f10841084e0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f1089c2b700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10800056d0 tx=0x7f10800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1080026070 con 0x7f10841084e0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f108000bba0 con 0x7f10841084e0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10841986f0 con 0x7f10841084e0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1080003920 con 0x7f10841084e0 2024-04-23T18:17:41.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.588+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108419c790 con 0x7f10841084e0 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.589+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1080029070 con 0x7f10841084e0 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.589+0000 7f107affd700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 0x7f107003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.589+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f108001e070 con 0x7f10841084e0 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.589+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1084103b90 con 0x7f10841084e0 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.589+0000 7f108942a700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 0x7f107003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:41.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.590+0000 7f108942a700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 0x7f107003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:41.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.594+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1080013070 con 0x7f10841084e0 2024-04-23T18:17:41.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:41.806+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}) -- 0x7f1084199220 con 0x7f1070039650 2024-04-23T18:17:42.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:42 smithi033 ceph-mon[29274]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:43.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:43 smithi033 ceph-mon[29274]: Deploying cephadm binary to smithi106 2024-04-23T18:17:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:45 smithi033 ceph-mon[29274]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.118+0000 7f107affd700 1 -- 172.21.15.33:0/2566557870 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f1084199220 con 0x7f1070039650 2024-04-23T18:17:46.112 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 msgr2=0x7f107003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 0x7f107003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1074006fd0 tx=0x7f1074006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 msgr2=0x7f1084197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10800056d0 tx=0x7f10800057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 shutdown_connections 2024-04-23T18:17:46.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1070039650 0x7f107003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 --2- 172.21.15.33:0/2566557870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f10841084e0 0x7f1084197ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 >> 172.21.15.33:0/2566557870 conn(0x7f1084076ea0 msgr2=0x7f10840fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.120+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 shutdown_connections 2024-04-23T18:17:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:46.121+0000 7f108be8f700 1 -- 172.21.15.33:0/2566557870 wait complete. 2024-04-23T18:17:46.706 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:17:46.954 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:47.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:47 smithi033 ceph-mon[29274]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:47.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:47.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:47 smithi033 ceph-mon[29274]: Added host smithi106 2024-04-23T18:17:47.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:47.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:48.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 -- 172.21.15.33:0/2818992294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 msgr2=0x7f66741088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2818992294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f66741088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f666c00b0a0 tx=0x7f666c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 -- 172.21.15.33:0/2818992294 shutdown_connections 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2818992294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f66741088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 -- 172.21.15.33:0/2818992294 >> 172.21.15.33:0/2818992294 conn(0x7f66740fdf60 msgr2=0x7f6674100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:48.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 -- 172.21.15.33:0/2818992294 shutdown_connections 2024-04-23T18:17:48.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.503+0000 7f667c2b1700 1 -- 172.21.15.33:0/2818992294 wait complete. 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667c2b1700 1 Processor -- start 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667c2b1700 1 -- start start 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667c2b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667c2b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f667410c940 con 0x7f66741084e0 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667a04d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667a04d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38466/0 (socket says 172.21.15.33:38466) 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.504+0000 7f667a04d700 1 -- 172.21.15.33:0/2136126300 learned_addr learned my addr 172.21.15.33:0/2136126300 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:48.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f667a04d700 1 -- 172.21.15.33:0/2136126300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6674076c30 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f667a04d700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f666c006130 tx=0x7f666c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c021020 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f666c0079b0 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66740752e0 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f666c017040 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.505+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66740757a0 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f666c024020 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6674103b90 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f66737fe700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 0x7f665c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f666c00f050 con 0x7f66741084e0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f667984c700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 0x7f665c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:48.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.506+0000 7f667984c700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 0x7f665c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:48.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.511+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f666c02b020 con 0x7f66741084e0 2024-04-23T18:17:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.724+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6674076270 con 0x7f665c039650 2024-04-23T18:17:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.725+0000 7f66737fe700 1 -- 172.21.15.33:0/2136126300 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6674076270 con 0x7f665c039650 2024-04-23T18:17:48.719 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:48.719 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-04-23T18:17:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 msgr2=0x7f665c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 0x7f665c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6668006fd0 tx=0x7f6668006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:48.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 msgr2=0x7f6674074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f666c006130 tx=0x7f666c0097d0 comp rx=0 tx=0).stop 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 shutdown_connections 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f665c039650 0x7f665c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.727+0000 7f667c2b1700 1 --2- 172.21.15.33:0/2136126300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66741084e0 0x7f6674074d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.728+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 >> 172.21.15.33:0/2136126300 conn(0x7f66740fdf60 msgr2=0x7f667410a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.728+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 shutdown_connections 2024-04-23T18:17:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:48.728+0000 7f667c2b1700 1 -- 172.21.15.33:0/2136126300 wait complete. 2024-04-23T18:17:49.325 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi122 2024-04-23T18:17:49.325 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:17:49.325 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.conf 2024-04-23T18:17:49.344 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:17:49.344 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:49.401 INFO:tasks.cephadm:Adding host smithi122 to orchestrator... 2024-04-23T18:17:49.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch host add smithi122 2024-04-23T18:17:49.646 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:49 smithi033 ceph-mon[29274]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:49 smithi033 ceph-mon[29274]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:17:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 -- 172.21.15.33:0/739724561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 msgr2=0x7f979c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 --2- 172.21.15.33:0/739724561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c1088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f978c009910 tx=0x7f978c009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 -- 172.21.15.33:0/739724561 shutdown_connections 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 --2- 172.21.15.33:0/739724561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c1088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 -- 172.21.15.33:0/739724561 >> 172.21.15.33:0/739724561 conn(0x7f979c076ea0 msgr2=0x7f979c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 -- 172.21.15.33:0/739724561 shutdown_connections 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.069+0000 7f97a2354700 1 -- 172.21.15.33:0/739724561 wait complete. 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f97a2354700 1 Processor -- start 2024-04-23T18:17:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f97a2354700 1 -- start start 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f97a2354700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f97a2354700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f979c10c920 con 0x7f979c1084e0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f979bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f979bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38478/0 (socket says 172.21.15.33:38478) 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f979bfff700 1 -- 172.21.15.33:0/3685839106 learned_addr learned my addr 172.21.15.33:0/3685839106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.070+0000 7f979bfff700 1 -- 172.21.15.33:0/3685839106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c198400 con 0x7f979c1084e0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f979bfff700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f978c0056d0 tx=0x7f978c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c026070 con 0x7f979c1084e0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c1986f0 con 0x7f979c1084e0 2024-04-23T18:17:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f978c00bba0 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c003920 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.071+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c19c790 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f978c029070 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c103b90 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f97997fa700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 0x7f978403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f978c01e070 con 0x7f979c1084e0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f979b7fe700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 0x7f978403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.072+0000 7f979b7fe700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 0x7f978403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:51.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.077+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f978c013070 con 0x7f979c1084e0 2024-04-23T18:17:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:51.291+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}) -- 0x7f979c199220 con 0x7f9784039650 2024-04-23T18:17:51.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:51 smithi033 ceph-mon[29274]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:17:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:17:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:17:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:17:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:52 smithi033 ceph-mon[29274]: Deploying cephadm binary to smithi122 2024-04-23T18:17:54.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:53 smithi033 ceph-mon[29274]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:54.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:53 smithi033 ceph-mon[29274]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:17:54.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:53 smithi033 ceph-mon[29274]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:17:55.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.580+0000 7f97997fa700 1 -- 172.21.15.33:0/3685839106 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f979c199220 con 0x7f9784039650 2024-04-23T18:17:55.574 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi122' with addr '172.21.15.122' 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.582+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 msgr2=0x7f978403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.582+0000 7f97a2354700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 0x7f978403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9790006fd0 tx=0x7f9790006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.582+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 msgr2=0x7f979c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.582+0000 7f97a2354700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f978c0056d0 tx=0x7f978c0057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 shutdown_connections 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9784039650 0x7f978403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 --2- 172.21.15.33:0/3685839106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f979c1084e0 0x7f979c197ec0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 >> 172.21.15.33:0/3685839106 conn(0x7f979c076ea0 msgr2=0x7f979c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 shutdown_connections 2024-04-23T18:17:55.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:55.583+0000 7f97a2354700 1 -- 172.21.15.33:0/3685839106 wait complete. 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:56.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:17:56.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-23T18:17:56.455 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:57.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:56 smithi033 ceph-mon[29274]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:17:57.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:56 smithi033 ceph-mon[29274]: Added host smithi122 2024-04-23T18:17:57.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:56 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:57.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:56 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:57.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:57 smithi033 ceph-mon[29274]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- 172.21.15.33:0/882661513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 msgr2=0x7fd448108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 --2- 172.21.15.33:0/882661513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448108930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd438009910 tx=0x7fd438009c20 comp rx=0 tx=0).stop 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- 172.21.15.33:0/882661513 shutdown_connections 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 --2- 172.21.15.33:0/882661513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448108930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- 172.21.15.33:0/882661513 >> 172.21.15.33:0/882661513 conn(0x7fd448077ac0 msgr2=0x7fd448077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- 172.21.15.33:0/882661513 shutdown_connections 2024-04-23T18:17:57.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- 172.21.15.33:0/882661513 wait complete. 2024-04-23T18:17:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 Processor -- start 2024-04-23T18:17:57.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.946+0000 7fd44e355700 1 -- start start 2024-04-23T18:17:57.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd44e355700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd44e355700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd44810c9c0 con 0x7fd448108550 2024-04-23T18:17:57.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd447fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd447fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48358/0 (socket says 172.21.15.33:48358) 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd447fff700 1 -- 172.21.15.33:0/780540057 learned_addr learned my addr 172.21.15.33:0/780540057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd447fff700 1 -- 172.21.15.33:0/780540057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd448198380 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.947+0000 7fd447fff700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd4380056d0 tx=0x7fd4380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd438026070 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd448198610 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd43800bc00 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd438003920 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44819c6c0 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.948+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd438029070 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.949+0000 7fd4457fa700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 0x7fd43003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.949+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd43801e070 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.949+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd448103cf0 con 0x7fd448108550 2024-04-23T18:17:57.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.949+0000 7fd4477fe700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 0x7fd43003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:17:57.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.949+0000 7fd4477fe700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 0x7fd43003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006fd0 tx=0x7fd43c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:17:57.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:57.954+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd438030070 con 0x7fd448108550 2024-04-23T18:17:58.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.170+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd448199150 con 0x7fd430039650 2024-04-23T18:17:58.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.171+0000 7fd4457fa700 1 -- 172.21.15.33:0/780540057 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fd448199150 con 0x7fd430039650 2024-04-23T18:17:58.165 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:17:58.165 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}] 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 msgr2=0x7fd43003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 0x7fd43003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd43c006fd0 tx=0x7fd43c006e40 comp rx=0 tx=0).stop 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 msgr2=0x7fd448197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd4380056d0 tx=0x7fd4380057b0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 shutdown_connections 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd430039650 0x7fd43003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 --2- 172.21.15.33:0/780540057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd448108550 0x7fd448197e40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:17:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 >> 172.21.15.33:0/780540057 conn(0x7fd448077ac0 msgr2=0x7fd4480fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:17:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 shutdown_connections 2024-04-23T18:17:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:17:58.173+0000 7fd44e355700 1 -- 172.21.15.33:0/780540057 wait complete. 2024-04-23T18:17:58.879 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-23T18:17:58.879 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-23T18:17:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:17:59.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:59 smithi033 ceph-mon[29274]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:17:59.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:17:59.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:17:59 smithi033 ceph-mon[29274]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:18:00.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 -- 172.21.15.33:0/2668720284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 msgr2=0x7f8114108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 --2- 172.21.15.33:0/2668720284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114108930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8104009910 tx=0x7f8104009c20 comp rx=0 tx=0).stop 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 -- 172.21.15.33:0/2668720284 shutdown_connections 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 --2- 172.21.15.33:0/2668720284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114108930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 -- 172.21.15.33:0/2668720284 >> 172.21.15.33:0/2668720284 conn(0x7f81140fdf70 msgr2=0x7f8114100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 -- 172.21.15.33:0/2668720284 shutdown_connections 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.519+0000 7f811a54d700 1 -- 172.21.15.33:0/2668720284 wait complete. 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f811a54d700 1 Processor -- start 2024-04-23T18:18:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f811a54d700 1 -- start start 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f811a54d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f811a54d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f811410c9c0 con 0x7f8114108550 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f8113fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f8113fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48372/0 (socket says 172.21.15.33:48372) 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.520+0000 7f8113fff700 1 -- 172.21.15.33:0/3139118730 learned_addr learned my addr 172.21.15.33:0/3139118730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f8113fff700 1 -- 172.21.15.33:0/3139118730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141983a0 con 0x7f8114108550 2024-04-23T18:18:00.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f8113fff700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81040056d0 tx=0x7f81040057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104026070 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810400bc00 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8104003920 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8114198630 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.521+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811419c730 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.522+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8104029070 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.522+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114103bc0 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.522+0000 7f81117fa700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 0x7f80fc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.522+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f810401e070 con 0x7f8114108550 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.522+0000 7f81137fe700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 0x7f80fc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:00.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.523+0000 7f81137fe700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 0x7f80fc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8108006fd0 tx=0x7f8108006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:00.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.527+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8104013070 con 0x7f8114108550 2024-04-23T18:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:00.739+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f81141991c0 con 0x7f8114108550 2024-04-23T18:18:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.135+0000 7f81117fa700 1 -- 172.21.15.33:0/3139118730 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8104030090 con 0x7f8114108550 2024-04-23T18:18:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 msgr2=0x7f80fc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 0x7f80fc03bb70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8108006fd0 tx=0x7f8108006e40 comp rx=0 tx=0).stop 2024-04-23T18:18:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 msgr2=0x7f8114197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81040056d0 tx=0x7f81040057b0 comp rx=0 tx=0).stop 2024-04-23T18:18:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 shutdown_connections 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.137+0000 7f811a54d700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80fc0396b0 0x7f80fc03bb70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.138+0000 7f811a54d700 1 --2- 172.21.15.33:0/3139118730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8114108550 0x7f8114197e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.138+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 >> 172.21.15.33:0/3139118730 conn(0x7f81140fdf70 msgr2=0x7f81140fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.138+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 shutdown_connections 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:01.138+0000 7f811a54d700 1 -- 172.21.15.33:0/3139118730 wait complete. 2024-04-23T18:18:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-04-23T18:18:01.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:01 smithi033 ceph-mon[29274]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:01.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:01 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3139118730' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-23T18:18:01.801 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-04-23T18:18:01.801 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-04-23T18:18:01.801 INFO:tasks.cephadm:Adding mon.c on smithi122 2024-04-23T18:18:01.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi106:172.21.15.106=b;smithi122:172.21.15.122=c' 2024-04-23T18:18:02.046 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:02.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3139118730' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-23T18:18:02.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:02.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:18:02.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:02 smithi033 ceph-mon[29274]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:03.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 -- 172.21.15.122:0/2148824206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 msgr2=0x7f6248103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 --2- 172.21.15.122:0/2148824206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248103a50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f624000b0a0 tx=0x7f624000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 -- 172.21.15.122:0/2148824206 shutdown_connections 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 --2- 172.21.15.122:0/2148824206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248103a50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 -- 172.21.15.122:0/2148824206 >> 172.21.15.122:0/2148824206 conn(0x7f62480fce60 msgr2=0x7f62480ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 -- 172.21.15.122:0/2148824206 shutdown_connections 2024-04-23T18:18:03.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.564+0000 7f624cbba700 1 -- 172.21.15.122:0/2148824206 wait complete. 2024-04-23T18:18:03.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624cbba700 1 Processor -- start 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624cbba700 1 -- start start 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624cbba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624cbba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f624810c9a0 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.122:44712/0 (socket says 172.21.15.122:44712) 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624659c700 1 -- 172.21.15.122:0/533221234 learned_addr learned my addr 172.21.15.122:0/533221234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.565+0000 7f624659c700 1 -- 172.21.15.122:0/533221234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481983d0 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f624659c700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6240015040 tx=0x7f6240009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6240021020 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6248198660 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240004000 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6240017040 con 0x7f6248103670 2024-04-23T18:18:03.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.566+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624819c710 con 0x7f6248103670 2024-04-23T18:18:03.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.567+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6240024020 con 0x7f6248103670 2024-04-23T18:18:03.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.567+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624806b170 con 0x7f6248103670 2024-04-23T18:18:03.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.567+0000 7f62377fe700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 0x7f623003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:03.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.567+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f624000f050 con 0x7f6248103670 2024-04-23T18:18:03.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.567+0000 7f6245d9b700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 0x7f623003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:03.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.568+0000 7f6245d9b700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 0x7f623003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f623c006fd0 tx=0x7f623c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.573+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f624002b020 con 0x7f6248103670 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:03.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.780+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi106:172.21.15.106=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}) -- 0x7f62481991a0 con 0x7f6230039650 2024-04-23T18:18:03.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.791+0000 7f62377fe700 1 -- 172.21.15.122:0/533221234 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f62481991a0 con 0x7f6230039650 2024-04-23T18:18:03.785 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mon update... 2024-04-23T18:18:03.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 msgr2=0x7f623003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 0x7f623003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f623c006fd0 tx=0x7f623c006e40 comp rx=0 tx=0).stop 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 msgr2=0x7f6248197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6240015040 tx=0x7f6240009800 comp rx=0 tx=0).stop 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 shutdown_connections 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f6230039650 0x7f623003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 --2- 172.21.15.122:0/533221234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6248103670 0x7f6248197e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 >> 172.21.15.122:0/533221234 conn(0x7f62480fce60 msgr2=0x7f6248104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 shutdown_connections 2024-04-23T18:18:03.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:03.793+0000 7f624cbba700 1 -- 172.21.15.122:0/533221234 wait complete. 2024-04-23T18:18:04.346 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.b.service 2024-04-23T18:18:04.349 DEBUG:teuthology.orchestra.run.smithi122:mon.c> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.c.service 2024-04-23T18:18:04.351 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:18:04.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi106:172.21.15.106=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi106:172.21.15.106=b;smithi122:172.21.15.122=c;count:3 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:05.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:05.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:05.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:04 smithi033 ceph-mon[29274]: Deploying daemon mon.c on smithi122 2024-04-23T18:18:05.411 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:06.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:05 smithi033 ceph-mon[29274]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:08.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 -- 172.21.15.122:0/2007512736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 msgr2=0x7fd3c0101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 --2- 172.21.15.122:0/2007512736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009910 tx=0x7fd3b0009c20 comp rx=0 tx=0).stop 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 -- 172.21.15.122:0/2007512736 shutdown_connections 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 --2- 172.21.15.122:0/2007512736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 -- 172.21.15.122:0/2007512736 >> 172.21.15.122:0/2007512736 conn(0x7fd3c00fce40 msgr2=0x7fd3c00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 -- 172.21.15.122:0/2007512736 shutdown_connections 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.109+0000 7fd3c637c700 1 -- 172.21.15.122:0/2007512736 wait complete. 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3c637c700 1 Processor -- start 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3c637c700 1 -- start start 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3c637c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3c637c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c010b830 con 0x7fd3c01015b0 2024-04-23T18:18:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.122:55144/0 (socket says 172.21.15.122:55144) 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.110+0000 7fd3bffff700 1 -- 172.21.15.122:0/300345901 learned_addr learned my addr 172.21.15.122:0/300345901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3bffff700 1 -- 172.21.15.122:0/300345901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c0199240 con 0x7fd3c01015b0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3bffff700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00056d0 tx=0x7fd3b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3b0026070 con 0x7fd3c01015b0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3b000bba0 con 0x7fd3c01015b0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c0199530 con 0x7fd3c01015b0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3b0003920 con 0x7fd3c01015b0 2024-04-23T18:18:08.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.111+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c01999f0 con 0x7fd3c01015b0 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.112+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd3b0029070 con 0x7fd3c01015b0 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.112+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c0102a90 con 0x7fd3c01015b0 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.112+0000 7fd3bd7fa700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 0x7fd3a803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.112+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd3b001e070 con 0x7fd3c01015b0 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.112+0000 7fd3bf7fe700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 0x7fd3a803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.113+0000 7fd3bf7fe700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 0x7fd3a803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006fd0 tx=0x7fd3b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:08.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.118+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3b0013070 con 0x7fd3c01015b0 2024-04-23T18:18:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:07 smithi033 ceph-mon[29274]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:08.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.411+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd3c00640a0 con 0x7fd3c01015b0 2024-04-23T18:18:08.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.411+0000 7fd3bd7fa700 1 -- 172.21.15.122:0/300345901 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd3b0003b90 con 0x7fd3c01015b0 2024-04-23T18:18:08.405 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:18:08.406 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:16:22.571495Z","created":"2024-04-23T18:16:22.571495Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-23T18:18:08.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 msgr2=0x7fd3a803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:08.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 0x7fd3a803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4006fd0 tx=0x7fd3b4006e40 comp rx=0 tx=0).stop 2024-04-23T18:18:08.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 msgr2=0x7fd3c0198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00056d0 tx=0x7fd3b00057b0 comp rx=0 tx=0).stop 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 shutdown_connections 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd3a8039650 0x7fd3a803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 --2- 172.21.15.122:0/300345901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3c01015b0 0x7fd3c0198d00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 >> 172.21.15.122:0/300345901 conn(0x7fd3c00fce40 msgr2=0x7fd3c0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 shutdown_connections 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:08.414+0000 7fd3c637c700 1 -- 172.21.15.122:0/300345901 wait complete. 2024-04-23T18:18:08.408 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 1 2024-04-23T18:18:08.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:08 smithi122 ceph-mon[32546]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-23T18:18:08.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:08 smithi122 bash[32518]: 3aafede04df2fd57a19b4fbb2aa3c07b0982dd2b9fbb502a00e6a4690a747917 2024-04-23T18:18:08.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:08 smithi122 systemd[1]: Started Ceph mon.c for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:18:10.131 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:18:10.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:18:10.372 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:18:13.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:18:13.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: mon.a calling monitor election 2024-04-23T18:18:13.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: mon.c calling monitor election 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: monmap epoch 2 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: last_changed 2024-04-23T18:18:08.490086+0000 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: min_mon_release 19 (squid) 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: election_strategy: 1 2024-04-23T18:18:13.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: fsmap 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: mgrmap e13: x(active, since 49s) 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:13.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: mon.a calling monitor election 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: mon.c calling monitor election 2024-04-23T18:18:13.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: monmap epoch 2 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: last_changed 2024-04-23T18:18:08.490086+0000 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: min_mon_release 19 (squid) 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: election_strategy: 1 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: fsmap 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:13.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: mgrmap e13: x(active, since 49s) 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:13.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:14.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 -- 172.21.15.122:0/3851704683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c80057d0 msgr2=0x7f16c8007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 --2- 172.21.15.122:0/3851704683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c80057d0 0x7f16c8007c60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16dc101380 tx=0x7f16d800feb0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 -- 172.21.15.122:0/3851704683 shutdown_connections 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 --2- 172.21.15.122:0/3851704683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16c80057d0 0x7f16c8007c60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 -- 172.21.15.122:0/3851704683 >> 172.21.15.122:0/3851704683 conn(0x7f16dc0fcf80 msgr2=0x7f16dc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 -- 172.21.15.122:0/3851704683 shutdown_connections 2024-04-23T18:18:14.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.954+0000 7f16e4717700 1 -- 172.21.15.122:0/3851704683 wait complete. 2024-04-23T18:18:14.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:14 smithi122 ceph-mon[32546]: Deploying daemon mon.b on smithi106 2024-04-23T18:18:14.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 Processor -- start 2024-04-23T18:18:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 -- start start 2024-04-23T18:18:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 0x7f16dc19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc1082f0 con 0x7f16dc19a6c0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e4717700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f16dc06bb80 con 0x7f16c80057d0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39498/0 (socket says 172.21.15.122:39498) 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 -- 172.21.15.122:0/3718405213 learned_addr learned my addr 172.21.15.122:0/3718405213 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e1cb2700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 0x7f16dc19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 -- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 msgr2=0x7f16dc19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 -- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 msgr2=0x7f16dc19a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.955+0000 7f16e24b3700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e1cb2700 1 -- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 msgr2=0x7f16dc19a180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e1cb2700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e1cb2700 1 -- 172.21.15.122:0/3718405213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16dc19ad00 con 0x7f16dc19a6c0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e1cb2700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 0x7f16dc19ea50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00b4d0 tx=0x7f16cc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f16cc011630 con 0x7f16dc19a6c0 2024-04-23T18:18:14.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16cc011c80 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16dc19ef90 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f16cc010da0 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.956+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16dc19f4f0 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.957+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f16cc018020 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.957+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16dc102b90 con 0x7f16dc19a6c0 2024-04-23T18:18:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.957+0000 7f16d37fe700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 0x7f16c003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.957+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f16cc04c150 con 0x7f16dc19a6c0 2024-04-23T18:18:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.958+0000 7f16e24b3700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 0x7f16c003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.958+0000 7f16e24b3700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 0x7f16c003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16d800fa60 tx=0x7f16d8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:14.963+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16cc01f020 con 0x7f16dc19a6c0 2024-04-23T18:18:14.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:14 smithi033 ceph-mon[29274]: Deploying daemon mon.b on smithi106 2024-04-23T18:18:14.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.267+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f16dc19f800 con 0x7f16dc19a6c0 2024-04-23T18:18:15.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.267+0000 7f16d37fe700 1 -- 172.21.15.122:0/3718405213 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f16cc018570 con 0x7f16dc19a6c0 2024-04-23T18:18:15.261 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:18:15.261 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:18:08.490086Z","created":"2024-04-23T18:16:22.571495Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 msgr2=0x7f16c003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 0x7f16c003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f16d800fa60 tx=0x7f16d8005960 comp rx=0 tx=0).stop 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 msgr2=0x7f16dc19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 0x7f16dc19ea50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f16cc00b4d0 tx=0x7f16cc00b890 comp rx=0 tx=0).stop 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 shutdown_connections 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f16c0039440 0x7f16c003b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f16c80057d0 0x7f16dc19a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 --2- 172.21.15.122:0/3718405213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16dc19a6c0 0x7f16dc19ea50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 >> 172.21.15.122:0/3718405213 conn(0x7f16dc0fcf80 msgr2=0x7f16dc0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:15.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 shutdown_connections 2024-04-23T18:18:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:15.269+0000 7f16e4717700 1 -- 172.21.15.122:0/3718405213 wait complete. 2024-04-23T18:18:15.264 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 2 2024-04-23T18:18:15.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:15 smithi122 ceph-mon[32546]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:15.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:15 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/3718405213' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:18:15.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:15 smithi033 ceph-mon[29274]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:15.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:15 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/3718405213' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:18:16.827 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-23T18:18:16.827 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-23T18:18:17.076 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:18:17.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:17 smithi106 systemd[1]: Started Ceph mon.b for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:18:22.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:18:22.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: mon.c calling monitor election 2024-04-23T18:18:22.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: mon.a calling monitor election 2024-04-23T18:18:22.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: monmap epoch 3 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:22.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: min_mon_release 19 (squid) 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: election_strategy: 1 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: fsmap 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: mgrmap e13: x(active, since 58s) 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:22 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:18:22.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: mon.c calling monitor election 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: mon.a calling monitor election 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: monmap epoch 3 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: min_mon_release 19 (squid) 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: election_strategy: 1 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: fsmap 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: mgrmap e13: x(active, since 58s) 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:22 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 -- 172.21.15.122:0/514135835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 msgr2=0x7f1e64005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/514135835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e64005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806a4a0 tx=0x7f1e70009e80 comp rx=0 tx=0).stop 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 -- 172.21.15.122:0/514135835 shutdown_connections 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/514135835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e64005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 -- 172.21.15.122:0/514135835 >> 172.21.15.122:0/514135835 conn(0x7f1e780fcf20 msgr2=0x7f1e780ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 -- 172.21.15.122:0/514135835 shutdown_connections 2024-04-23T18:18:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.911+0000 7f1e7f6da700 1 -- 172.21.15.122:0/514135835 wait complete. 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 Processor -- start 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 -- start start 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e78102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7810a430 con 0x7f1e640035c0 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7810a2b0 con 0x7f1e78101350 2024-04-23T18:18:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.912+0000 7f1e7f6da700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7810a5b0 con 0x7f1e78101d10 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36696/0 (socket says 172.21.15.122:36696) 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 -- 172.21.15.122:0/3873443483 learned_addr learned my addr 172.21.15.122:0/3873443483 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 msgr2=0x7f1e781aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 msgr2=0x7f1e781aab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7dc77700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e78102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7cc75700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7cc75700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 msgr2=0x7f1e781017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7cc75700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 msgr2=0x7f1e781017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7cc75700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7cc75700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 msgr2=0x7f1e781aab30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:18:22.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 msgr2=0x7f1e781017d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 -- 172.21.15.122:0/3873443483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781ab230 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.913+0000 7f1e7d476700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e78102f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e70002230 tx=0x7f1e70004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.914+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70003c00 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.914+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781ab4c0 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.914+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e781aba50 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.914+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e70026070 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.914+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70030060 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.915+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7806aca0 con 0x7f1e640035c0 2024-04-23T18:18:22.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.915+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1e70015040 con 0x7f1e640035c0 2024-04-23T18:18:22.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.915+0000 7f1e6e7fc700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 0x7f1e5c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:22.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.915+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1e7004e600 con 0x7f1e640035c0 2024-04-23T18:18:22.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.915+0000 7f1e7cc75700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 0x7f1e5c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:22.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.916+0000 7f1e7cc75700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 0x7f1e5c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e7819c2c0 tx=0x7f1e68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:22.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:22.921+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e70029b50 con 0x7f1e640035c0 2024-04-23T18:18:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.224+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1e781028d0 con 0x7f1e640035c0 2024-04-23T18:18:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.224+0000 7f1e6e7fc700 1 -- 172.21.15.122:0/3873443483 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f1e700175a0 con 0x7f1e640035c0 2024-04-23T18:18:23.219 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-23T18:18:23.220 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","modified":"2024-04-23T18:18:17.267332Z","created":"2024-04-23T18:16:22.571495Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-23T18:18:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.226+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 msgr2=0x7f1e5c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.226+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 0x7f1e5c03bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1e7819c2c0 tx=0x7f1e68008040 comp rx=0 tx=0).stop 2024-04-23T18:18:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 msgr2=0x7f1e78102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e78102f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e70002230 tx=0x7f1e70004420 comp rx=0 tx=0).stop 2024-04-23T18:18:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 shutdown_connections 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e5c039ae0 0x7f1e5c03bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e78101d10 0x7f1e781aab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e640035c0 0x7f1e78102f40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 --2- 172.21.15.122:0/3873443483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e78101350 0x7f1e781017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 >> 172.21.15.122:0/3873443483 conn(0x7f1e780fcf20 msgr2=0x7f1e78107670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 shutdown_connections 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:23.227+0000 7f1e7f6da700 1 -- 172.21.15.122:0/3873443483 wait complete. 2024-04-23T18:18:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 3 2024-04-23T18:18:23.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:23 smithi122 ceph-mon[32546]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:23.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:23 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:23.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/3873443483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:18:23.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:23 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:23.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:23 smithi033 ceph-mon[29274]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:23.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:23 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:23.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/3873443483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:18:23.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:23 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:23.949 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-23T18:18:23.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-23T18:18:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: mon.c calling monitor election 2024-04-23T18:18:24.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: mon.a calling monitor election 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: monmap epoch 3 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:24.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: min_mon_release 19 (squid) 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: election_strategy: 1 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: fsmap 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: mgrmap e13: x(active, since 58s) 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: overall HEALTH_OK 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:24.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:24.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:24.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:24.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/3873443483' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-23T18:18:24.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mon.b calling monitor election 2024-04-23T18:18:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mon.b calling monitor election 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mon.a calling monitor election 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mon.c calling monitor election 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: monmap epoch 3 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: min_mon_release 19 (squid) 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: election_strategy: 1 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:25.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: fsmap 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: mgrmap e13: x(active, since 60s) 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: overall HEALTH_OK 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.373 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:25.374 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mon.b calling monitor election 2024-04-23T18:18:25.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:25.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mon.b calling monitor election 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mon.a calling monitor election 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mon.c calling monitor election 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: monmap epoch 3 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: min_mon_release 19 (squid) 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: election_strategy: 1 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:25.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: fsmap 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: mgrmap e13: x(active, since 60s) 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:25.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mon.b calling monitor election 2024-04-23T18:18:25.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:25.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mon.b calling monitor election 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mon.a calling monitor election 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mon.c calling monitor election 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: monmap epoch 3 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: last_changed 2024-04-23T18:18:17.267332+0000 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: created 2024-04-23T18:16:22.571495+0000 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: min_mon_release 19 (squid) 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: election_strategy: 1 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-23T18:18:25.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: fsmap 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: osdmap e4: 0 total, 0 up, 0 in 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: mgrmap e13: x(active, since 60s) 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:25.476 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 -- 172.21.15.33:0/2691082542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec41084e0 msgr2=0x7faec41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 --2- 172.21.15.33:0/2691082542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec41084e0 0x7faec41088c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009910 tx=0x7faeb8009c20 comp rx=0 tx=0).stop 2024-04-23T18:18:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 -- 172.21.15.33:0/2691082542 shutdown_connections 2024-04-23T18:18:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 --2- 172.21.15.33:0/2691082542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec41084e0 0x7faec41088c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 -- 172.21.15.33:0/2691082542 >> 172.21.15.33:0/2691082542 conn(0x7faec4076ea0 msgr2=0x7faec40772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 -- 172.21.15.33:0/2691082542 shutdown_connections 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.830+0000 7faecb040700 1 -- 172.21.15.33:0/2691082542 wait complete. 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 Processor -- start 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 -- start start 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faec41084e0 0x7faec41980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 0x7faec419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faec410c920 con 0x7faec419cf00 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faec410c7a0 con 0x7faec41084e0 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faecb040700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faec410caa0 con 0x7faec4198630 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faec95dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faec95dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59010/0 (socket says 172.21.15.33:59010) 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.831+0000 7faec95dd700 1 -- 172.21.15.33:0/3485075034 learned_addr learned my addr 172.21.15.33:0/3485075034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec8ddc700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faec41084e0 0x7faec41980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec3fff700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 0x7faec419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 -- 172.21.15.33:0/3485075034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 msgr2=0x7faec419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 0x7faec419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 -- 172.21.15.33:0/3485075034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faec41084e0 msgr2=0x7faec41980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faec41084e0 0x7faec41980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 -- 172.21.15.33:0/3485075034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec41a4c70 con 0x7faec419cf00 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec3fff700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 0x7faec419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec95dd700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faebc00ba50 tx=0x7faebc00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc019070 con 0x7faec419cf00 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faebc014440 con 0x7faec419cf00 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec41a4ea0 con 0x7faec419cf00 2024-04-23T18:18:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc007590 con 0x7faec419cf00 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.832+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41a52f0 con 0x7faec419cf00 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.833+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faebc01c070 con 0x7faec419cf00 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.833+0000 7faec1ffb700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 0x7faeb403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.833+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faebc04bee0 con 0x7faec419cf00 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.833+0000 7faec8ddc700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 0x7faeb403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.833+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec40640a0 con 0x7faec419cf00 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.834+0000 7faec8ddc700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 0x7faeb403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:25.839+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faebc00e0a0 con 0x7faec419cf00 2024-04-23T18:18:26.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.061+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7faec41a5610 con 0x7faec419cf00 2024-04-23T18:18:26.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.061+0000 7faec1ffb700 1 -- 172.21.15.33:0/3485075034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7faebc014910 con 0x7faec419cf00 2024-04-23T18:18:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:26.055 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-04-23T18:18:26.055 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:18:26.055 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 msgr2=0x7faeb403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 0x7faeb403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faeb8000c00 tx=0x7faeb8005fb0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 msgr2=0x7faec419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faebc00ba50 tx=0x7faebc00be10 comp rx=0 tx=0).stop 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 shutdown_connections 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faeb4039ae0 0x7faeb403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faec419cf00 0x7faec419d3b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faec41084e0 0x7faec41980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 --2- 172.21.15.33:0/3485075034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faec4198630 0x7faec419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:26.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 >> 172.21.15.33:0/3485075034 conn(0x7faec4076ea0 msgr2=0x7faec40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:26.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 shutdown_connections 2024-04-23T18:18:26.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:26.063+0000 7faecb040700 1 -- 172.21.15.33:0/3485075034 wait complete. 2024-04-23T18:18:26.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:26 smithi033 ceph-mon[29274]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:18:26.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:26 smithi033 ceph-mon[29274]: Reconfiguring daemon mon.a on smithi033 2024-04-23T18:18:26.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:26.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:26 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3485075034' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:26.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:26 smithi122 ceph-mon[32546]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:18:26.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:26 smithi122 ceph-mon[32546]: Reconfiguring daemon mon.a on smithi033 2024-04-23T18:18:26.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:26.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:26 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3485075034' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:26.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:26 smithi106 ceph-mon[32716]: Reconfiguring mon.a (unknown last config time)... 2024-04-23T18:18:26.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:26 smithi106 ceph-mon[32716]: Reconfiguring daemon mon.a on smithi033 2024-04-23T18:18:26.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:18:26.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:26 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3485075034' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:27.088 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-23T18:18:27.088 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:18:27.088 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:18:27.115 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:18:27.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:18:27.179 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:18:27.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:18:27.211 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:18:27.211 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:18:27.277 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:18:27.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-04-23T18:18:27.309 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:18:27.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:18:27.374 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:27 smithi122 ceph-mon[32546]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:27.378 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-04-23T18:18:27.378 INFO:tasks.cephadm:Adding mgr.y on smithi106 2024-04-23T18:18:27.378 INFO:tasks.cephadm:Adding mgr.z on smithi122 2024-04-23T18:18:27.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi106=y;smithi122=z' 2024-04-23T18:18:27.388 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:27 smithi033 ceph-mon[29274]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:27.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:27 smithi106 ceph-mon[32716]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:27.654 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:18:28.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:18:28.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:28.553 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:28.553 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:28.553 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:28 smithi106 ceph-mon[32716]: Reconfiguring daemon mon.b on smithi106 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:28 smithi122 ceph-mon[32546]: Reconfiguring daemon mon.b on smithi106 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: Reconfiguring mon.b (monmap changed)... 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:28 smithi033 ceph-mon[29274]: Reconfiguring daemon mon.b on smithi106 2024-04-23T18:18:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 -- 172.21.15.122:0/2311665815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441025d0 msgr2=0x7f55441029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 --2- 172.21.15.122:0/2311665815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441025d0 0x7f55441029f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5538009910 tx=0x7f5538009c20 comp rx=0 tx=0).stop 2024-04-23T18:18:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 -- 172.21.15.122:0/2311665815 shutdown_connections 2024-04-23T18:18:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 --2- 172.21.15.122:0/2311665815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441025d0 0x7f55441029f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 -- 172.21.15.122:0/2311665815 >> 172.21.15.122:0/2311665815 conn(0x7f55440fdbb0 msgr2=0x7f55440fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 -- 172.21.15.122:0/2311665815 shutdown_connections 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.161+0000 7f554aabd700 1 -- 172.21.15.122:0/2311665815 wait complete. 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 Processor -- start 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 -- start start 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 0x7f5544197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441986f0 0x7f554419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5544108510 con 0x7f55441986f0 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5544108390 con 0x7f55441025d0 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554aabd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5544108690 con 0x7f5544198270 2024-04-23T18:18:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49998/0 (socket says 172.21.15.122:49998) 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 -- 172.21.15.122:0/1467811752 learned_addr learned my addr 172.21.15.122:0/1467811752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f554905a700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441986f0 0x7f554419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5548859700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 0x7f5544197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 -- 172.21.15.122:0/1467811752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 msgr2=0x7f5544197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 0x7f5544197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 -- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441986f0 msgr2=0x7f554419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441986f0 0x7f554419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5543fff700 1 -- 172.21.15.122:0/1467811752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55441a0150 con 0x7f5544198270 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.162+0000 7f5548859700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 0x7f5544197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f5543fff700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f553000b780 tx=0x7f553000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530004230 con 0x7f5544198270 2024-04-23T18:18:29.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55441a0440 con 0x7f5544198270 2024-04-23T18:18:29.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55300043d0 con 0x7f5544198270 2024-04-23T18:18:29.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530011ad0 con 0x7f5544198270 2024-04-23T18:18:29.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.163+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55441a0a50 con 0x7f5544198270 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.164+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5530011c90 con 0x7f5544198270 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.164+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55440640a0 con 0x7f5544198270 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.164+0000 7f5541ffb700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 0x7f553403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.164+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f553004c3c0 con 0x7f5544198270 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.164+0000 7f5548859700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 0x7f553403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:29.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.165+0000 7f5548859700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 0x7f553403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5538005190 tx=0x7f553801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:29.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.169+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5530020500 con 0x7f5544198270 2024-04-23T18:18:29.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.395+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi106=y;smithi122=z", "target": ["mon-mgr", ""]}) -- 0x7f5544062f10 con 0x7f5534039ae0 2024-04-23T18:18:29.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.404+0000 7f5541ffb700 1 -- 172.21.15.122:0/1467811752 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5544062f10 con 0x7f5534039ae0 2024-04-23T18:18:29.398 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mgr update... 2024-04-23T18:18:29.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 msgr2=0x7f553403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:29.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 0x7f553403bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5538005190 tx=0x7f553801a040 comp rx=0 tx=0).stop 2024-04-23T18:18:29.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 msgr2=0x7f554419d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:29.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f553000b780 tx=0x7f553000ba90 comp rx=0 tx=0).stop 2024-04-23T18:18:29.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 shutdown_connections 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5534039ae0 0x7f553403bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55441986f0 0x7f554419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55441025d0 0x7f5544197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 --2- 172.21.15.122:0/1467811752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5544198270 0x7f554419d2a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 >> 172.21.15.122:0/1467811752 conn(0x7f55440fdbb0 msgr2=0x7f55440fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 shutdown_connections 2024-04-23T18:18:29.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:18:29.406+0000 7f554aabd700 1 -- 172.21.15.122:0/1467811752 wait complete. 2024-04-23T18:18:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:18:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:30.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:30.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:30.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: Reconfiguring daemon mon.c on smithi122 2024-04-23T18:18:30.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: Reconfiguring daemon mon.c on smithi122 2024-04-23T18:18:30.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: Reconfiguring mon.c (monmap changed)... 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: Reconfiguring daemon mon.c on smithi122 2024-04-23T18:18:30.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:30.444 DEBUG:teuthology.orchestra.run.smithi106:mgr.y> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.y.service 2024-04-23T18:18:30.447 DEBUG:teuthology.orchestra.run.smithi122:mgr.z> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z.service 2024-04-23T18:18:30.449 INFO:tasks.cephadm:Deploying OSDs... 2024-04-23T18:18:30.449 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:18:30.449 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:18:30.469 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:18:30.469 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-23 18:17:37.053531777 +0000 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-23 18:12:36.493227076 +0000 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-23 18:12:36.493227076 +0000 2024-04-23T18:18:30.528 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-23 18:12:36.493227076 +0000 2024-04-23T18:18:30.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:18:30.596 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-23T18:18:30.596 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-23T18:18:30.596 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00022583 s, 2.3 MB/s 2024-04-23T18:18:30.598 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:18:30.658 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-23 18:17:37.083531302 +0000 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-23 18:12:36.848221411 +0000 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-23 18:12:36.848221411 +0000 2024-04-23T18:18:30.716 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-23 18:12:36.848221411 +0000 2024-04-23T18:18:30.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:18:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-23T18:18:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-23T18:18:30.783 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000223659 s, 2.3 MB/s 2024-04-23T18:18:30.784 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:18:30.843 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:30.900 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-23 18:17:37.109530892 +0000 2024-04-23T18:18:30.901 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-23 18:12:37.224215412 +0000 2024-04-23T18:18:30.901 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-23 18:12:37.224215412 +0000 2024-04-23T18:18:30.901 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-23 18:12:37.224215412 +0000 2024-04-23T18:18:30.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:18:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-23T18:18:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-23T18:18:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000237657 s, 2.2 MB/s 2024-04-23T18:18:30.965 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:18:31.021 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-23 18:17:37.137530449 +0000 2024-04-23T18:18:31.076 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-23 18:12:37.573209842 +0000 2024-04-23T18:18:31.077 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-23 18:12:37.573209842 +0000 2024-04-23T18:18:31.077 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-23 18:12:37.573209842 +0000 2024-04-23T18:18:31.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:18:31.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:30 smithi033 ceph-mon[29274]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi106=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:31.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:30 smithi033 ceph-mon[29274]: Saving service mgr spec with placement smithi033=x;smithi106=y;smithi122=z;count:3 2024-04-23T18:18:31.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.138 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:31.140 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-23T18:18:31.140 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-23T18:18:31.140 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000183775 s, 2.8 MB/s 2024-04-23T18:18:31.141 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:18:31.198 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-23T18:18:31.198 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:18:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:30 smithi122 ceph-mon[32546]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi106=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:30 smithi122 ceph-mon[32546]: Saving service mgr spec with placement smithi033=x;smithi106=y;smithi122=z;count:3 2024-04-23T18:18:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:31.217 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:18:31.217 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-23 18:18:16.728245439 +0000 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-23 18:12:59.058215668 +0000 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-23 18:12:59.058215668 +0000 2024-04-23T18:18:31.275 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-23 18:12:59.058215668 +0000 2024-04-23T18:18:31.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:30 smithi106 ceph-mon[32716]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi106=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:30 smithi106 ceph-mon[32716]: Saving service mgr spec with placement smithi033=x;smithi106=y;smithi122=z;count:3 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:31.341 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-23T18:18:31.341 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-23T18:18:31.341 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000280877 s, 1.8 MB/s 2024-04-23T18:18:31.342 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:18:31.399 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-23T18:18:31.455 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-23 18:18:16.729245416 +0000 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-23 18:12:59.399207696 +0000 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-23 18:12:59.399207696 +0000 2024-04-23T18:18:31.456 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-23 18:12:59.399207696 +0000 2024-04-23T18:18:31.456 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:18:31.519 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-23T18:18:31.519 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-23T18:18:31.519 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000274708 s, 1.9 MB/s 2024-04-23T18:18:31.520 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:18:31.576 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.631 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-23 18:18:16.729245416 +0000 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-23 18:12:59.740199724 +0000 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-23 18:12:59.740199724 +0000 2024-04-23T18:18:31.632 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-23 18:12:59.740199724 +0000 2024-04-23T18:18:31.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:18:31.694 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-23T18:18:31.694 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-23T18:18:31.694 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000184817 s, 2.8 MB/s 2024-04-23T18:18:31.695 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:18:31.751 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-23 18:18:16.729245416 +0000 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-23 18:13:00.085191658 +0000 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-23 18:13:00.085191658 +0000 2024-04-23T18:18:31.806 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-23 18:13:00.085191658 +0000 2024-04-23T18:18:31.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:18:31.868 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-23T18:18:31.868 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-23T18:18:31.868 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000172187 s, 3.0 MB/s 2024-04-23T18:18:31.869 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:18:31.925 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-23T18:18:31.925 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-23T18:18:31.940 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-23T18:18:31.941 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-23 18:18:07.556290508 +0000 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-23 18:13:07.262022206 +0000 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-23 18:13:07.262022206 +0000 2024-04-23T18:18:31.996 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-23 18:13:07.262022206 +0000 2024-04-23T18:18:31.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-23T18:18:32.059 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-23T18:18:32.059 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-23T18:18:32.059 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000187351 s, 2.7 MB/s 2024-04-23T18:18:32.060 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-23T18:18:32.115 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:31 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.170 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-23 18:18:07.557290486 +0000 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-23 18:13:07.610014575 +0000 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-23 18:13:07.610014575 +0000 2024-04-23T18:18:32.171 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-23 18:13:07.610014575 +0000 2024-04-23T18:18:32.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:31 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-23T18:18:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-23T18:18:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000178332 s, 2.9 MB/s 2024-04-23T18:18:32.225 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:32.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:31 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:32.281 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-23T18:18:32.335 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-23 18:18:07.557290486 +0000 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-23 18:13:07.947007185 +0000 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-23 18:13:07.947007185 +0000 2024-04-23T18:18:32.336 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-23 18:13:07.947007185 +0000 2024-04-23T18:18:32.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-23T18:18:32.398 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-23T18:18:32.398 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-23T18:18:32.398 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000191217 s, 2.7 MB/s 2024-04-23T18:18:32.399 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-23T18:18:32.454 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-23T18:18:32.508 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-23 18:18:07.558290465 +0000 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-23 18:13:08.268000146 +0000 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-23 18:13:08.268000146 +0000 2024-04-23T18:18:32.509 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-23 18:13:08.268000146 +0000 2024-04-23T18:18:32.509 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-23T18:18:32.571 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-23T18:18:32.571 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-23T18:18:32.571 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000172029 s, 3.0 MB/s 2024-04-23T18:18:32.572 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-23T18:18:32.627 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-04-23T18:18:32.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:18:32.881 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:32.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:32 smithi033 ceph-mon[29274]: Deploying daemon mgr.z on smithi122 2024-04-23T18:18:33.172 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:32 smithi122 ceph-mon[32546]: Deploying daemon mgr.z on smithi122 2024-04-23T18:18:33.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:32 smithi106 ceph-mon[32716]: Deploying daemon mgr.z on smithi122 2024-04-23T18:18:33.449 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 systemd[1]: Starting Ceph mgr.z for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:18:33.878 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 podman[34390]: 2024-04-23 18:18:33.593866918 +0000 UTC m=+0.109877614 container create 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-23T18:18:33.878 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 podman[34390]: 2024-04-23 18:18:33.502073388 +0000 UTC m=+0.018084115 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:18:33.878 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 podman[34390]: 2024-04-23 18:18:33.812594285 +0000 UTC m=+0.328604990 container init 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:18:33.878 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 podman[34390]: 2024-04-23 18:18:33.815173954 +0000 UTC m=+0.331184649 container start 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:18:33.878 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 ceph-mgr[34409]: -- 172.21.15.122:0/2699029768 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd6782a5a0 con 0x55cd677fe800 2024-04-23T18:18:34.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:33 smithi122 ceph-mon[32546]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:34.148 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 bash[34390]: 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 2024-04-23T18:18:34.148 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 systemd[1]: Started Ceph mgr.z for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:18:34.148 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:33.969+0000 7efe34f2e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:18:34.148 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:34.154+0000 7efe34f2e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:18:34.189 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:33 smithi033 ceph-mon[29274]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:34.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:33 smithi106 ceph-mon[32716]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:34.449 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:34.255+0000 7efe34f2e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:18:34.951 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:34.562+0000 7efe34f2e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:18:34.951 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:34.646+0000 7efe34f2e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:18:34.987 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125148 s, 838 MB/s 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 2024-04-23T18:18:34.988 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:35.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:35.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:35.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:35.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-04-23T18:18:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:18:35.980 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:35 smithi106 ceph-mon[32716]: Deploying daemon mgr.y on smithi106 2024-04-23T18:18:35.980 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:35 smithi106 ceph-mon[32716]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:36.195 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:35 smithi122 ceph-mon[32546]: Deploying daemon mgr.y on smithi106 2024-04-23T18:18:36.195 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:35 smithi122 ceph-mon[32546]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:36.195 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:35.862+0000 7efe34f2e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:18:36.195 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.032+0000 7efe34f2e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:18:36.195 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.201+0000 7efe34f2e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:18:36.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:35 smithi033 ceph-mon[29274]: Deploying daemon mgr.y on smithi106 2024-04-23T18:18:36.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:35 smithi033 ceph-mon[29274]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:36.621 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:36.509+0000 7faa1b773200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:18:36.783 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.490+0000 7efe34f2e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:18:36.783 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.582+0000 7efe34f2e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:18:36.783 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.789+0000 7efe34f2e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:36.698+0000 7faa1b773200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:18:37.121 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:36.806+0000 7faa1b773200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:18:37.199 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:36.990+0000 7efe34f2e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/2698365751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc103580 msgr2=0x7fb0dc109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/2698365751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc103580 0x7fb0dc109e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400b0d0 tx=0x7fb0d400b3e0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/2698365751 shutdown_connections 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/2698365751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc103580 0x7fb0dc109e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/2698365751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/2698365751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/2698365751 >> 172.21.15.33:0/2698365751 conn(0x7fb0dc0fcfa0 msgr2=0x7fb0dc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/2698365751 shutdown_connections 2024-04-23T18:18:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.433+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/2698365751 wait complete. 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 Processor -- start 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 -- start start 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc105970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc103580 0x7fb0dc104500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc10d4d0 con 0x7fb0dc102be0 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc10d350 con 0x7fb0dc103580 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0e2a2a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc10d650 con 0x7fb0dc1019e0 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.434+0000 7fb0db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33070/0 (socket says 172.21.15.33:33070) 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0db7fe700 1 -- 172.21.15.33:0/3924405434 learned_addr learned my addr 172.21.15.33:0/3924405434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0e0fc7700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc103580 0x7fb0dc104500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 -- 172.21.15.33:0/3924405434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc103580 msgr2=0x7fb0dc104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc103580 0x7fb0dc104500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 -- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 msgr2=0x7fb0dc103fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 -- 172.21.15.33:0/3924405434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc104db0 con 0x7fb0dc1019e0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0db7fe700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0dbfff700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc105970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00d740 tx=0x7fb0cc00da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc016070 con 0x7fb0dc1019e0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0cc00f690 con 0x7fb0dc1019e0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc1a2d70 con 0x7fb0dc1019e0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc003b50 con 0x7fb0dc1019e0 2024-04-23T18:18:37.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.435+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1a31f0 con 0x7fb0dc1019e0 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.436+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb0cc003cf0 con 0x7fb0dc1019e0 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.436+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0dc069730 con 0x7fb0dc1019e0 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.436+0000 7fb0d97fa700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 0x7fb0c403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.436+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0cc04c7e0 con 0x7fb0dc1019e0 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.436+0000 7fb0db7fe700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 0x7fb0c403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:18:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.437+0000 7fb0db7fe700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 0x7fb0c403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00105c0 tx=0x7fb0d000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:18:37.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.442+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0cc00fb60 con 0x7fb0dc1019e0 2024-04-23T18:18:37.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:37.131+0000 7faa1b773200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:18:37.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:37.225+0000 7faa1b773200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:18:37.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:37.660+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb0dc0ff130 con 0x7fb0c4039890 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:37.618+0000 7efe34f2e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:38.337 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:38.047+0000 7efe34f2e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:18:38.338 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:38.142+0000 7efe34f2e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:18:38.338 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:38.344+0000 7efe34f2e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:38.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:38.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:38.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:38.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.802 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:38.469+0000 7faa1b773200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:18:38.802 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:38.640+0000 7faa1b773200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:18:38.802 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:38.808+0000 7faa1b773200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='client.24118 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-23T18:18:38.803 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-23T18:18:38.804 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:39.191 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:38.867+0000 7efe34f2e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:18:39.192 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:38.958+0000 7efe34f2e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:39.096+0000 7faa1b773200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:18:39.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:39.190+0000 7faa1b773200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:18:39.449 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:39.198+0000 7efe34f2e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:18:39.449 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:39.329+0000 7efe34f2e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:18:39.449 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:39.423+0000 7efe34f2e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:18:39.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:39 smithi033 ceph-mon[29274]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:39.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:39 smithi033 ceph-mon[29274]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:18:39.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:39 smithi033 ceph-mon[29274]: Reconfiguring daemon mgr.x on smithi033 2024-04-23T18:18:39.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-mon[32546]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-mon[32546]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-mon[32546]: Reconfiguring daemon mgr.x on smithi033 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:39.538+0000 7efe34f2e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:18:39.829 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:39.835+0000 7efe34f2e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-mon[32716]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-mon[32716]: Reconfiguring mgr.x (unknown last config time)... 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-mon[32716]: Reconfiguring daemon mgr.x on smithi033 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:39.392+0000 7faa1b773200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:18:39.871 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:39.585+0000 7faa1b773200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:18:40.598 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:40.179+0000 7faa1b773200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:18:40.699 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:40.429+0000 7efe34f2e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:18:40.699 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:18:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:18:40.644+0000 7efe34f2e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:18:40.871 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:40.605+0000 7faa1b773200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:18:40.871 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:40.701+0000 7faa1b773200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:18:41.333 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:40.903+0000 7faa1b773200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:18:41.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:41.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: Standby manager daemon z started 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:18:41.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:41 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:41.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:41.496+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fb0cc010d00 con 0x7fb0dc1019e0 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:41.448+0000 7faa1b773200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:41.543+0000 7faa1b773200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:41.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: Standby manager daemon z started 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:18:41.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: Standby manager daemon z started 2024-04-23T18:18:41.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:18:41.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:41.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:18:41.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:41 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2134522712' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:42.115 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:41.782+0000 7faa1b773200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:18:42.115 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:41.913+0000 7faa1b773200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:18:42.115 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:42.007+0000 7faa1b773200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:18:42.115 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:42.121+0000 7faa1b773200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]: dispatch 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]': finished 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:18:42.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:18:42.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1964696786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]: dispatch 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]': finished 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:18:42.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:18:42.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1964696786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]: dispatch 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2871290830' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c"}]': finished 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: osdmap e5: 1 total, 0 up, 1 in 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:18:42.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1964696786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:18:42.871 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:42.418+0000 7faa1b773200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:18:43.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:43.015+0000 7faa1b773200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:18:43.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:18:43.227+0000 7faa1b773200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:18:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:43.506+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mon.1 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fb0cc019d70 con 0x7fb0dc1019e0 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: Standby manager daemon y started 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:18:43.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:43 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: Standby manager daemon y started 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:18:43.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:43 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:43.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: Standby manager daemon y started 2024-04-23T18:18:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:18:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:18:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:18:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:43 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/2809786156' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:18:44.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:44 smithi106 ceph-mon[32716]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-23T18:18:44.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:44 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:44.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:44 smithi122 ceph-mon[32546]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-23T18:18:44.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:44 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:44.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:44 smithi033 ceph-mon[29274]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-23T18:18:44.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:44 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:18:45.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:45 smithi033 ceph-mon[29274]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:45.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:45 smithi106 ceph-mon[32716]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:45.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:45 smithi122 ceph-mon[32546]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:47.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:47 smithi033 ceph-mon[29274]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:47.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:47 smithi106 ceph-mon[32716]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:47.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:47 smithi122 ceph-mon[32546]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-23T18:18:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:49.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:49 smithi033 ceph-mon[29274]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:49.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:49 smithi033 ceph-mon[29274]: Deploying daemon osd.0 on smithi033 2024-04-23T18:18:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:49 smithi106 ceph-mon[32716]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:49 smithi106 ceph-mon[32716]: Deploying daemon osd.0 on smithi033 2024-04-23T18:18:49.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:49 smithi122 ceph-mon[32546]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:49.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:49 smithi122 ceph-mon[32546]: Deploying daemon osd.0 on smithi033 2024-04-23T18:18:51.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:51 smithi033 ceph-mon[29274]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:51.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:51 smithi106 ceph-mon[32716]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:51.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:51 smithi122 ceph-mon[32546]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:53.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:53 smithi033 ceph-mon[29274]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:53.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:53 smithi106 ceph-mon[32716]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:53.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:53 smithi122 ceph-mon[32546]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:54.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:54.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:54.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:54.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:54.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:54.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:55.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:55 smithi122 ceph-mon[32546]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:55.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:55 smithi033 ceph-mon[29274]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:55 smithi106 ceph-mon[32716]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.201+0000 7fb0d97fa700 1 -- 172.21.15.33:0/3924405434 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb0dc0ff130 con 0x7fb0c4039890 2024-04-23T18:18:57.196 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-04-23T18:18:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 msgr2=0x7fb0c403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 0x7fb0c403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00105c0 tx=0x7fb0d000b480 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 msgr2=0x7fb0dc105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc105970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00d740 tx=0x7fb0cc00da50 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 shutdown_connections 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb0c4039890 0x7fb0c403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0dc103580 0x7fb0dc104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0dc1019e0 0x7fb0dc105970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 --2- 172.21.15.33:0/3924405434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0dc102be0 0x7fb0dc103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 >> 172.21.15.33:0/3924405434 conn(0x7fb0dc0fcfa0 msgr2=0x7fb0dc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:18:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 shutdown_connections 2024-04-23T18:18:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:18:57.205+0000 7fb0e2a2a700 1 -- 172.21.15.33:0/3924405434 wait complete. 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:18:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:18:57.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:18:57.865 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.0.service 2024-04-23T18:18:57.867 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/vg_nvme/lv_4... 2024-04-23T18:18:57.867 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:18:58.109 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:18:58.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:58 smithi033 ceph-mon[29274]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:58.545 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:58 smithi106 ceph-mon[32716]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:58 smithi122 ceph-mon[32546]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-23T18:18:59.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:59 smithi106 ceph-mon[32716]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:59.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:59 smithi106 ceph-mon[32716]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:59.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:59 smithi106 ceph-mon[32716]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:59.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:59 smithi106 ceph-mon[32716]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:18:59.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:18:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-mon[29274]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-mon[29274]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-mon[29274]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-mon[29274]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:18:59.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:59 smithi122 ceph-mon[32546]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:18:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:59 smithi122 ceph-mon[32546]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-23T18:18:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:59 smithi122 ceph-mon[32546]: osdmap e6: 1 total, 0 up, 1 in 2024-04-23T18:18:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:59 smithi122 ceph-mon[32546]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:18:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:18:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113368 s, 925 MB/s 2024-04-23T18:18:59.920 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-23T18:19:00.110 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0[43690]: 2024-04-23T18:18:59.805+0000 7f685b54d700 -1 osd.0 0 waiting for initial osdmap 2024-04-23T18:19:00.110 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:18:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0[43690]: 2024-04-23T18:18:59.811+0000 7f6853338700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:00.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:00 smithi033 ceph-mon[29274]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:19:00.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:00 smithi033 ceph-mon[29274]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:19:00.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:00 smithi033 ceph-mon[29274]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' 2024-04-23T18:19:00.491 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:00 smithi106 ceph-mon[32716]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:19:00.491 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:00 smithi106 ceph-mon[32716]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:19:00.491 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.491 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.491 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:00 smithi106 ceph-mon[32716]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' 2024-04-23T18:19:00.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-04-23T18:19:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:00 smithi122 ceph-mon[32546]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:19:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:00 smithi122 ceph-mon[32546]: osdmap e7: 1 total, 0 up, 1 in 2024-04-23T18:19:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:00 smithi122 ceph-mon[32546]: from='osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407]' entity='osd.0' 2024-04-23T18:19:00.763 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] boot 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:19:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi033 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:01.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] boot 2024-04-23T18:19:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi033 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:01.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: osd.0 [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] boot 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: osdmap e8: 1 total, 1 up, 1 in 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi033 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 17093M 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:01.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 -- 172.21.15.106:0/3371082442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1958103540 msgr2=0x7f1958109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 --2- 172.21.15.106:0/3371082442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1958103540 0x7f1958109df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f195000b0d0 tx=0x7f195000b3e0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 -- 172.21.15.106:0/3371082442 shutdown_connections 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 --2- 172.21.15.106:0/3371082442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1958103540 0x7f1958109df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 --2- 172.21.15.106:0/3371082442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19581019a0 0x7f1958101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 --2- 172.21.15.106:0/3371082442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958102ba0 0x7f1958103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 -- 172.21.15.106:0/3371082442 >> 172.21.15.106:0/3371082442 conn(0x7f19580fcf40 msgr2=0x7f19580ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 -- 172.21.15.106:0/3371082442 shutdown_connections 2024-04-23T18:19:02.263 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.267+0000 7f195e224700 1 -- 172.21.15.106:0/3371082442 wait complete. 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 Processor -- start 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 -- start start 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1958102ba0 0x7f195819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958103540 0x7f19581a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d530 con 0x7f1958102ba0 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d3b0 con 0x7f19581019a0 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f195e224700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f195810d6b0 con 0x7f1958103540 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:53412/0 (socket says 172.21.15.106:53412) 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 -- 172.21.15.106:0/1526878327 learned_addr learned my addr 172.21.15.106:0/1526878327 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f1957fff700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958103540 0x7f19581a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:02.264 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f1956ffd700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1958102ba0 0x7f195819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 -- 172.21.15.106:0/1526878327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958103540 msgr2=0x7f19581a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958103540 0x7f19581a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 -- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1958102ba0 msgr2=0x7f195819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1958102ba0 0x7f195819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.268+0000 7f19577fe700 1 -- 172.21.15.106:0/1526878327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19581a0990 con 0x7f19581019a0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f19577fe700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f194800b500 tx=0x7f194800b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948015020 con 0x7f19581019a0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1948011ce0 con 0x7f19581019a0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948010d60 con 0x7f19581019a0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19581a0c80 con 0x7f19581019a0 2024-04-23T18:19:02.265 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.269+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19581a11c0 con 0x7f19581019a0 2024-04-23T18:19:02.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.270+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1958069770 con 0x7f19581019a0 2024-04-23T18:19:02.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.271+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1948018020 con 0x7f19581019a0 2024-04-23T18:19:02.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.272+0000 7f1954ff9700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:02.266 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.272+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f19480ce660 con 0x7f19581019a0 2024-04-23T18:19:02.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.272+0000 7f1956ffd700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:02.267 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.272+0000 7f1956ffd700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 0x7f19400a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f194c006fd0 tx=0x7f194c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:02.269 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.276+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f194801f020 con 0x7f19581019a0 2024-04-23T18:19:02.476 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:02.482+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1958062e80 con 0x7f19400a3c30 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: osdmap e9: 1 total, 1 up, 1 in 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/4224233634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]': finished 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:19:05.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:05 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/4224233634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]': finished 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:19:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:05 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/4224233634' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]: dispatch 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3bf75bc-fda4-44d5-be29-dbb282a8a56a"}]': finished 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: osdmap e10: 2 total, 1 up, 2 in 2024-04-23T18:19:05.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:05 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:06.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:06 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/1526194804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:06.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:06 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/1526194804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:06.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:06 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/1526194804' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:07 smithi106 ceph-mon[32716]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:07 smithi122 ceph-mon[32546]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:07.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:07 smithi033 ceph-mon[29274]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:09.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:09 smithi106 ceph-mon[32716]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:09.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:09 smithi122 ceph-mon[32546]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:09.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:09 smithi033 ceph-mon[29274]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:11.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:11 smithi106 ceph-mon[32716]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:11.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:11 smithi122 ceph-mon[32546]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:11 smithi033 ceph-mon[29274]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:12.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:19:12.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:12.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:19:12.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:12.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-23T18:19:12.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:13.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:13 smithi106 ceph-mon[32716]: Deploying daemon osd.1 on smithi106 2024-04-23T18:19:13.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:13 smithi106 ceph-mon[32716]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:13 smithi122 ceph-mon[32546]: Deploying daemon osd.1 on smithi106 2024-04-23T18:19:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:13 smithi122 ceph-mon[32546]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:13.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:13 smithi033 ceph-mon[29274]: Deploying daemon osd.1 on smithi106 2024-04-23T18:19:13.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:13 smithi033 ceph-mon[29274]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:15.571 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:15 smithi106 ceph-mon[32716]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:15.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:15 smithi122 ceph-mon[32546]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:15.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:15 smithi033 ceph-mon[29274]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:17 smithi122 ceph-mon[32546]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:17.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:17 smithi106 ceph-mon[32716]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:17.712 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:17.712 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:17.712 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:17.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:17 smithi033 ceph-mon[29274]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:17.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:17.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:17.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:19.518 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:19 smithi106 ceph-mon[32716]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:19.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:19 smithi122 ceph-mon[32546]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:19.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:19 smithi033 ceph-mon[29274]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:20.196 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.202+0000 7f1954ff9700 1 -- 172.21.15.106:0/1526878327 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1958062e80 con 0x7f19400a3c30 2024-04-23T18:19:20.196 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-04-23T18:19:20.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 msgr2=0x7f19400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:20.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 0x7f19400a60f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f194c006fd0 tx=0x7f194c008040 comp rx=0 tx=0).stop 2024-04-23T18:19:20.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 msgr2=0x7f195806b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:20.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f194800b500 tx=0x7f194800b8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:20.198 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 shutdown_connections 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f19400a3c30 0x7f19400a60f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1958103540 0x7f19581a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19581019a0 0x7f195806b2a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 --2- 172.21.15.106:0/1526878327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1958102ba0 0x7f195819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.204+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 >> 172.21.15.106:0/1526878327 conn(0x7f19580fcf40 msgr2=0x7f1958105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.205+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 shutdown_connections 2024-04-23T18:19:20.199 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:19:20.205+0000 7f195e224700 1 -- 172.21.15.106:0/1526878327 wait complete. 2024-04-23T18:19:21.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:21 smithi106 ceph-mon[32716]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:21.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.261 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.1.service 2024-04-23T18:19:21.263 INFO:tasks.cephadm:Deploying osd.2 on smithi122 with /dev/vg_nvme/lv_4... 2024-04-23T18:19:21.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-23T18:19:21.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:21 smithi122 ceph-mon[32546]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:21.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:21 smithi033 ceph-mon[29274]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:19:22.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:22 smithi122 ceph-mon[32546]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:22.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:22 smithi122 ceph-mon[32546]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:22.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:22 smithi033 ceph-mon[29274]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:22.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:22 smithi033 ceph-mon[29274]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:22.554 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:22 smithi106 ceph-mon[32716]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:22.554 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:22 smithi106 ceph-mon[32716]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111151 s, 943 MB/s 2024-04-23T18:19:23.243 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:23 smithi033 ceph-mon[29274]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:23 smithi122 ceph-mon[32546]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: osdmap e11: 2 total, 1 up, 2 in 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: from='osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-mon[32716]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:19:23.871 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-04-23T18:19:24.088 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-23T18:19:23.682+0000 7fd8367cd700 -1 osd.1 0 waiting for initial osdmap 2024-04-23T18:19:24.088 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-23T18:19:23.685+0000 7fd82e5b8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:24.111 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:19:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:24 smithi106 ceph-mon[32716]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:19:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:24 smithi106 ceph-mon[32716]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:19:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:24 smithi106 ceph-mon[32716]: from='osd.1 ' entity='osd.1' 2024-04-23T18:19:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:24 smithi033 ceph-mon[29274]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:19:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:24 smithi033 ceph-mon[29274]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:19:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:24 smithi033 ceph-mon[29274]: from='osd.1 ' entity='osd.1' 2024-04-23T18:19:24.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:24 smithi122 ceph-mon[32546]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:19:24.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:24 smithi122 ceph-mon[32546]: osdmap e12: 2 total, 1 up, 2 in 2024-04-23T18:19:24.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:24.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:24 smithi122 ceph-mon[32546]: from='osd.1 ' entity='osd.1' 2024-04-23T18:19:25.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:19:25.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:19:25.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:25.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.278 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi106 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] boot 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:19:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:19:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi106 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] boot 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:25.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 -- 172.21.15.122:0/940752910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 msgr2=0x7fab341020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 --2- 172.21.15.122:0/940752910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab341020e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fab280099c0 tx=0x7fab28009cd0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 -- 172.21.15.122:0/940752910 shutdown_connections 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 --2- 172.21.15.122:0/940752910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 --2- 172.21.15.122:0/940752910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 0x7fab34103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 --2- 172.21.15.122:0/940752910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab341020e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 -- 172.21.15.122:0/940752910 >> 172.21.15.122:0/940752910 conn(0x7fab340fc6b0 msgr2=0x7fab340feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:25.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 -- 172.21.15.122:0/940752910 shutdown_connections 2024-04-23T18:19:25.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.504+0000 7fab39eee700 1 -- 172.21.15.122:0/940752910 wait complete. 2024-04-23T18:19:25.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.505+0000 7fab39eee700 1 Processor -- start 2024-04-23T18:19:25.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.505+0000 7fab39eee700 1 -- start start 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 0x7fab3419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410d590 con 0x7fab34102620 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410d410 con 0x7fab34102f40 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab3410d710 con 0x7fab34101c80 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52196/0 (socket says 172.21.15.122:52196) 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 -- 172.21.15.122:0/2384591347 learned_addr learned my addr 172.21.15.122:0/2384591347 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab33fff700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 0x7fab3419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab32ffd700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:25.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 -- 172.21.15.122:0/2384591347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 msgr2=0x7fab3419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 0x7fab3419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 -- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 msgr2=0x7fab34198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 -- 172.21.15.122:0/2384591347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419f870 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab32ffd700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab337fe700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b500 tx=0x7fab2400b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24015020 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419fb60 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab24011ce0 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24010d60 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.506+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a00a0 con 0x7fab34101c80 2024-04-23T18:19:25.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.507+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab34069760 con 0x7fab34101c80 2024-04-23T18:19:25.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.508+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fab24018020 con 0x7fab34101c80 2024-04-23T18:19:25.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.509+0000 7fab30ff9700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 0x7fab1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:25.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.509+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fab240ce3c0 con 0x7fab34101c80 2024-04-23T18:19:25.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.509+0000 7fab32ffd700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 0x7fab1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:25.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.510+0000 7fab32ffd700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 0x7fab1c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab28006010 tx=0x7fab28009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:25.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.514+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab34069760 con 0x7fab34101c80 2024-04-23T18:19:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:19:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:19:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-23T18:19:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi106 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: osd.1 [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] boot 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: osdmap e13: 2 total, 2 up, 2 in 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:25.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:25.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:25.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:25.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:25.723+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fab34100bd0 con 0x7fab1c0a3bb0 2024-04-23T18:19:26.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:26 smithi122 ceph-mon[32546]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:19:26.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:26 smithi122 ceph-mon[32546]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:26.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:26.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:26.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:26 smithi033 ceph-mon[29274]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:19:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:26 smithi033 ceph-mon[29274]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:26.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:27.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:26 smithi106 ceph-mon[32716]: osdmap e14: 2 total, 2 up, 2 in 2024-04-23T18:19:27.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:26 smithi106 ceph-mon[32716]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:27.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:27.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:27.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:27.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:27 smithi122 ceph-mon[32546]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:27.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:27 smithi033 ceph-mon[29274]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:28.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:27 smithi106 ceph-mon[32716]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:28.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/2385922912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:28.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:28 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:28.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:28 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]': finished 2024-04-23T18:19:28.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:28 smithi033 ceph-mon[29274]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:19:28.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:29.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/2385922912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:29.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:28 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:29.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:28 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]': finished 2024-04-23T18:19:29.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:28 smithi106 ceph-mon[32716]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:19:29.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:29.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/2385922912' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:29.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:28 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]: dispatch 2024-04-23T18:19:29.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:28 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77497994-8bdd-45c2-8521-089e1016f545"}]': finished 2024-04-23T18:19:29.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:28 smithi122 ceph-mon[32546]: osdmap e15: 3 total, 2 up, 3 in 2024-04-23T18:19:29.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:29 smithi033 ceph-mon[29274]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:29 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/3970747464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:29 smithi106 ceph-mon[32716]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:29 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/3970747464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:29 smithi122 ceph-mon[32546]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:29 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/3970747464' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:31.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:31 smithi033 ceph-mon[29274]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:32.091 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:31 smithi122 ceph-mon[32546]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:31 smithi106 ceph-mon[32716]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:33.906 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:33 smithi122 ceph-mon[32546]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:33.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:33 smithi033 ceph-mon[29274]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:34.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:33 smithi106 ceph-mon[32716]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:35 smithi033 ceph-mon[29274]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:35 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:19:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:35 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:36.106 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:35 smithi122 ceph-mon[32546]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:36.106 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:35 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:19:36.106 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:35 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:36.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:35 smithi106 ceph-mon[32716]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:36.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:35 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-23T18:19:36.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:35 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:36.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:36 smithi033 ceph-mon[29274]: Deploying daemon osd.2 on smithi122 2024-04-23T18:19:37.107 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:36 smithi122 ceph-mon[32546]: Deploying daemon osd.2 on smithi122 2024-04-23T18:19:37.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:36 smithi106 ceph-mon[32716]: Deploying daemon osd.2 on smithi122 2024-04-23T18:19:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:37 smithi122 ceph-mon[32546]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:38.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:37 smithi106 ceph-mon[32716]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:38.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:37 smithi033 ceph-mon[29274]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:39.916 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:39 smithi122 ceph-mon[32546]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:40.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:39 smithi106 ceph-mon[32716]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:40.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:39 smithi033 ceph-mon[29274]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:41.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:41 smithi122 ceph-mon[32546]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:41.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:41.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:41.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:41.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:41 smithi033 ceph-mon[29274]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:42.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:41 smithi106 ceph-mon[32716]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:42.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:42.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:42.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:43.844 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:43 smithi122 ceph-mon[32546]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:43.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.939+0000 7fab30ff9700 1 -- 172.21.15.122:0/2384591347 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab34100bd0 con 0x7fab1c0a3bb0 2024-04-23T18:19:43.933 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 2 on host 'smithi122' 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 msgr2=0x7fab1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 0x7fab1c0a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fab28006010 tx=0x7fab28009f10 comp rx=0 tx=0).stop 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 msgr2=0x7fab34197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab2400b500 tx=0x7fab2400b8c0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 shutdown_connections 2024-04-23T18:19:43.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fab1c0a3bb0 0x7fab1c0a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab34102f40 0x7fab3419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101c80 0x7fab34197c20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 --2- 172.21.15.122:0/2384591347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab34102620 0x7fab34198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 >> 172.21.15.122:0/2384591347 conn(0x7fab340fc6b0 msgr2=0x7fab34106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 shutdown_connections 2024-04-23T18:19:43.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:19:43.942+0000 7fab39eee700 1 -- 172.21.15.122:0/2384591347 wait complete. 2024-04-23T18:19:43.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:43 smithi033 ceph-mon[29274]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:44.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:43 smithi106 ceph-mon[32716]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:44.515 DEBUG:teuthology.orchestra.run.smithi122:osd.2> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.2.service 2024-04-23T18:19:44.517 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-04-23T18:19:44.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:19:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:44.821 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T18:19:44.641+0000 7f9cd7f3e7c0 -1 osd.2 0 log_to_monitors true 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:45.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:44 smithi033 ceph-mon[29274]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:45.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:44 smithi122 ceph-mon[32546]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:44 smithi106 ceph-mon[32716]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:45 smithi033 ceph-mon[29274]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:45 smithi122 ceph-mon[32546]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: osdmap e16: 3 total, 2 up, 3 in 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: from='osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:45 smithi106 ceph-mon[32716]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111039 s, 944 MB/s 2024-04-23T18:19:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-23T18:19:46.699 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T18:19:46.465+0000 7f9ccd5a3700 -1 osd.2 0 waiting for initial osdmap 2024-04-23T18:19:46.699 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T18:19:46.471+0000 7f9cc538e700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:19:47.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:19:47.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:47.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:47.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:47.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: osdmap e17: 3 total, 2 up, 3 in 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: from='osd.2 ' entity='osd.2' 2024-04-23T18:19:47.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:47.507 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] boot 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi122 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:48 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:19:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:19:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] boot 2024-04-23T18:19:48.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi122 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.873 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:48 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 -- 172.21.15.33:0/3241698083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 msgr2=0x7f7670108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 --2- 172.21.15.33:0/3241698083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670108250 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7658009990 tx=0x7f7658009ca0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 -- 172.21.15.33:0/3241698083 shutdown_connections 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 --2- 172.21.15.33:0/3241698083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 0x7f767010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 --2- 172.21.15.33:0/3241698083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 --2- 172.21.15.33:0/3241698083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670108250 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 -- 172.21.15.33:0/3241698083 >> 172.21.15.33:0/3241698083 conn(0x7f76700fcf40 msgr2=0x7f76700ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 -- 172.21.15.33:0/3241698083 shutdown_connections 2024-04-23T18:19:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.936+0000 7f7674f50700 1 -- 172.21.15.33:0/3241698083 wait complete. 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 Processor -- start 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 -- start start 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 0x7f76701a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f767010e530 con 0x7f7670105e60 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f767010e3b0 con 0x7f7670103530 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f7674f50700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f767010e6b0 con 0x7f7670108790 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f766dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f766dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42154/0 (socket says 172.21.15.33:42154) 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f766dd9b700 1 -- 172.21.15.33:0/1998458237 learned_addr learned my addr 172.21.15.33:0/1998458237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.937+0000 7f766e59c700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766ed9d700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 0x7f76701a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 -- 172.21.15.33:0/1998458237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 msgr2=0x7f76701a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 0x7f76701a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 -- 172.21.15.33:0/1998458237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 msgr2=0x7f7670198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 -- 172.21.15.33:0/1998458237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76701a08a0 con 0x7f7670105e60 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766e59c700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f766dd9b700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7658006010 tx=0x7f76580057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658027070 con 0x7f7670105e60 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7658004000 con 0x7f7670105e60 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76701a0b30 con 0x7f7670105e60 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658022ba0 con 0x7f7670105e60 2024-04-23T18:19:48.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.938+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701a0f80 con 0x7f7670105e60 2024-04-23T18:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.939+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7670069730 con 0x7f7670105e60 2024-04-23T18:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.941+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f765802a070 con 0x7f7670105e60 2024-04-23T18:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.941+0000 7f76677fe700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.941+0000 7f766e59c700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:19:48.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.941+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f76580d0760 con 0x7f7670105e60 2024-04-23T18:19:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.943+0000 7f766e59c700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 0x7f765c0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f767019a130 tx=0x7f76600091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:19:48.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:48.945+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7658031020 con 0x7f7670105e60 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: osd.2 [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] boot 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: osdmap e18: 3 total, 3 up, 3 in 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi122 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:48.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:48 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:49.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:49.163+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7670062e80 con 0x7f765c0a3bb0 2024-04-23T18:19:49.849 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:49.850 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:49.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41450]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41450]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41450]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41450]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:49.872 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41453]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48384]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48384]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48384]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48384]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48389]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48389]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48389]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:49.915 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48389]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:49.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: pgmap v80: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: osdmap e19: 3 total, 3 up, 3 in 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:19:49.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41760]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41760]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41760]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41760]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41763]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41763]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41763]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.199 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41763]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41766]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41769]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41769]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41769]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:49 smithi122 sudo[41769]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48393]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48393]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48399]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:49 smithi033 sudo[48399]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41456]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41456]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:49 smithi106 sudo[41456]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 sudo[41456]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 sudo[41459]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 sudo[41459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 sudo[41459]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-23T18:19:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 sudo[41459]: pam_unix(sudo:session): session closed for user root 2024-04-23T18:19:50.771 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:50.771 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:50.771 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:50.771 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.773 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:50 smithi033 ceph-mon[29274]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:50 smithi106 ceph-mon[32716]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:19:50.918+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f765809a600 con 0x7f7670105e60 2024-04-23T18:19:50.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:19:50.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-23T18:19:50.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: osdmap e20: 3 total, 3 up, 3 in 2024-04-23T18:19:50.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:19:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:19:50.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:19:50.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:50 smithi122 ceph-mon[32546]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-23T18:19:51.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:51 smithi106 ceph-mon[32716]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:51.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:51 smithi106 ceph-mon[32716]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:51.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:51 smithi106 ceph-mon[32716]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:51.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:51 smithi122 ceph-mon[32546]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:51.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:51 smithi122 ceph-mon[32546]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:51.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:51 smithi122 ceph-mon[32546]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:51.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:51 smithi033 ceph-mon[29274]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:51.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:51 smithi033 ceph-mon[29274]: osdmap e21: 3 total, 3 up, 3 in 2024-04-23T18:19:51.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:51 smithi033 ceph-mon[29274]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-23T18:19:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:52 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]: dispatch 2024-04-23T18:19:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:52 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]': finished 2024-04-23T18:19:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:52 smithi106 ceph-mon[32716]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:52.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:52 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]: dispatch 2024-04-23T18:19:52.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:52 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]': finished 2024-04-23T18:19:52.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:52 smithi122 ceph-mon[32546]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:52.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:52.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]: dispatch 2024-04-23T18:19:52.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2804439068' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "70af3b74-4df3-4c39-8562-3b0052a1319f"}]': finished 2024-04-23T18:19:52.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:52 smithi033 ceph-mon[29274]: osdmap e22: 4 total, 3 up, 4 in 2024-04-23T18:19:52.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:19:53.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:53 smithi106 ceph-mon[32716]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:53.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2557423277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:53.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:53 smithi122 ceph-mon[32546]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:53.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2557423277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:53.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:53 smithi033 ceph-mon[29274]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:53.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2557423277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:19:55.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:55 smithi106 ceph-mon[32716]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:55.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:55 smithi122 ceph-mon[32546]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:55.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:55 smithi033 ceph-mon[29274]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:57.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:57 smithi106 ceph-mon[32716]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:57.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:57 smithi033 ceph-mon[29274]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:57.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:57 smithi122 ceph-mon[32546]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:59 smithi106 ceph-mon[32716]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:19:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:59.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:59 smithi033 ceph-mon[29274]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:59.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:59.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:19:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:19:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:59 smithi122 ceph-mon[32546]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:19:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-23T18:19:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:19:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:00.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:00 smithi122 ceph-mon[32546]: Deploying daemon osd.3 on smithi033 2024-04-23T18:20:00.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:00 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:20:00.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:00 smithi033 ceph-mon[29274]: Deploying daemon osd.3 on smithi033 2024-04-23T18:20:00.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:00 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:20:01.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:00 smithi106 ceph-mon[32716]: Deploying daemon osd.3 on smithi033 2024-04-23T18:20:01.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:00 smithi106 ceph-mon[32716]: overall HEALTH_OK 2024-04-23T18:20:01.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:01 smithi122 ceph-mon[32546]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:01.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:01 smithi033 ceph-mon[29274]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:02.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:01 smithi106 ceph-mon[32716]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:03.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:03 smithi122 ceph-mon[32546]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:03.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:03 smithi033 ceph-mon[29274]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:04.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:03 smithi106 ceph-mon[32716]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:06.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:05 smithi122 ceph-mon[32546]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:06.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:05 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:05 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:05 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:06.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:05 smithi033 ceph-mon[29274]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:06.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:05 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:05 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.233 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:05 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:06.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:05 smithi106 ceph-mon[32716]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:06.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:05 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:05 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:06.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:05 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:08.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.724+0000 7f76677fe700 1 -- 172.21.15.33:0/1998458237 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7670062e80 con 0x7f765c0a3bb0 2024-04-23T18:20:08.719 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-04-23T18:20:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 msgr2=0x7f765c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 0x7f765c0a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f767019a130 tx=0x7f76600091b0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 msgr2=0x7f7670199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:08.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7658006010 tx=0x7f76580057a0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 shutdown_connections 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f765c0a3bb0 0x7f765c0a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7670108790 0x7f76701a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7670103530 0x7f7670198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 --2- 172.21.15.33:0/1998458237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7670105e60 0x7f7670199190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 >> 172.21.15.33:0/1998458237 conn(0x7f76700fcf40 msgr2=0x7f7670105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 shutdown_connections 2024-04-23T18:20:08.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:20:08.727+0000 7f7674f50700 1 -- 172.21.15.33:0/1998458237 wait complete. 2024-04-23T18:20:09.376 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.3.service 2024-04-23T18:20:09.378 INFO:tasks.cephadm:Deploying osd.4 on smithi106 with /dev/vg_nvme/lv_3... 2024-04-23T18:20:09.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:20:09.620 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:09 smithi106 ceph-mon[32716]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:09 smithi106 ceph-mon[32716]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:09 smithi033 ceph-mon[29274]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:09.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:09.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:09.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:09 smithi033 ceph-mon[29274]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:20:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:09 smithi122 ceph-mon[32546]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:09 smithi122 ceph-mon[32546]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-23T18:20:10.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:10 smithi033 ceph-mon[29274]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:20:10.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:10 smithi033 ceph-mon[29274]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:20:10.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:10 smithi033 ceph-mon[29274]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:20:10.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:10 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:10.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:10 smithi106 ceph-mon[32716]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:20:10.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:10 smithi106 ceph-mon[32716]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:20:10.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:10 smithi106 ceph-mon[32716]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:20:10.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:10 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:11.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:10 smithi122 ceph-mon[32546]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-23T18:20:11.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:10 smithi122 ceph-mon[32546]: osdmap e23: 4 total, 3 up, 4 in 2024-04-23T18:20:11.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:10 smithi122 ceph-mon[32546]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:20:11.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:10 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:11.468 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:20:11.469 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:20:11.469 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-23T18:20:11.469 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-23T18:20:11.469 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128154 s, 818 MB/s 2024-04-23T18:20:11.469 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-23T18:20:11.481 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T18:20:11.154+0000 7f6a286c0700 -1 osd.3 0 waiting for initial osdmap 2024-04-23T18:20:11.481 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T18:20:11.160+0000 7f6a204ab700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:11.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:11 smithi033 ceph-mon[29274]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' 2024-04-23T18:20:12.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:12.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:20:12.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:20:12.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:11 smithi106 ceph-mon[32716]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' 2024-04-23T18:20:12.111 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: osdmap e24: 4 total, 3 up, 4 in 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:11 smithi122 ceph-mon[32546]: from='osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347]' entity='osd.3' 2024-04-23T18:20:12.347 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] boot 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:12.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:12.735 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] boot 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:12.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: osd.3 [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] boot 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: osdmap e25: 4 total, 4 up, 4 in 2024-04-23T18:20:13.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:13.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 -- 172.21.15.106:0/2673093826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c103550 msgr2=0x7f677c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 --2- 172.21.15.106:0/2673093826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c103550 0x7f677c109e00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f677400b0d0 tx=0x7f677400b3e0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 -- 172.21.15.106:0/2673093826 shutdown_connections 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 --2- 172.21.15.106:0/2673093826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c103550 0x7f677c109e00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 --2- 172.21.15.106:0/2673093826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c1019b0 0x7f677c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 --2- 172.21.15.106:0/2673093826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c102bb0 0x7f677c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 -- 172.21.15.106:0/2673093826 >> 172.21.15.106:0/2673093826 conn(0x7f677c0fcf70 msgr2=0x7f677c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:13.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 -- 172.21.15.106:0/2673093826 shutdown_connections 2024-04-23T18:20:13.905 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.910+0000 7f67810f6700 1 -- 172.21.15.106:0/2673093826 wait complete. 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 Processor -- start 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 -- start start 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c102bb0 0x7f677c19b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c103550 0x7f677c1a2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d540 con 0x7f677c103550 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d3c0 con 0x7f677c1019b0 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f67810f6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f677c10d6c0 con 0x7f677c102bb0 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f677ad9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f677ad9d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48364/0 (socket says 172.21.15.106:48364) 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.911+0000 7f677ad9d700 1 -- 172.21.15.106:0/731487798 learned_addr learned my addr 172.21.15.106:0/731487798 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677a59c700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c102bb0 0x7f677c19b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677b59e700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c103550 0x7f677c1a2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 -- 172.21.15.106:0/731487798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c102bb0 msgr2=0x7f677c19b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c102bb0 0x7f677c19b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 -- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c103550 msgr2=0x7f677c1a2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c103550 0x7f677c1a2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 -- 172.21.15.106:0/731487798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677c1a2bd0 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f677ad9d700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f676c00b500 tx=0x7f676c00b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c015020 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c011ce0 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677c1a2ec0 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c010d50 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.912+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677c1a3340 con 0x7f677c1019b0 2024-04-23T18:20:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.913+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677c1a35d0 con 0x7f677c1019b0 2024-04-23T18:20:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.914+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f676c018020 con 0x7f677c1019b0 2024-04-23T18:20:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.915+0000 7f6763fff700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 0x7f67640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.915+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f676c0ce9e0 con 0x7f677c1019b0 2024-04-23T18:20:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.915+0000 7f677a59c700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 0x7f67640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.915+0000 7f677a59c700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 0x7f67640a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:13.913 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:13.920+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f676c01f020 con 0x7f677c1019b0 2024-04-23T18:20:14.121 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:14.127+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f677c06b8d0 con 0x7f67640a3c40 2024-04-23T18:20:14.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:13 smithi106 ceph-mon[32716]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:14.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:13 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi033 2024-04-23T18:20:14.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:13 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-23T18:20:14.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:13 smithi106 ceph-mon[32716]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:20:14.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:13 smithi122 ceph-mon[32546]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:14.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:13 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi033 2024-04-23T18:20:14.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:13 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-23T18:20:14.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:13 smithi122 ceph-mon[32546]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:20:14.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:13 smithi033 ceph-mon[29274]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-23T18:20:14.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:13 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi033 2024-04-23T18:20:14.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:13 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 8546M 2024-04-23T18:20:14.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:13 smithi033 ceph-mon[29274]: osdmap e26: 4 total, 4 up, 4 in 2024-04-23T18:20:15.017 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:14 smithi122 ceph-mon[32546]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:20:15.018 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:15.018 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:15.018 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:15.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:14 smithi106 ceph-mon[32716]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:20:15.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:15.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:15.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:15.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:14 smithi033 ceph-mon[29274]: osdmap e27: 4 total, 4 up, 4 in 2024-04-23T18:20:15.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:15.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:15.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:16.096 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:15 smithi106 ceph-mon[32716]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:16.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:15 smithi106 ceph-mon[32716]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:16.097 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:15 smithi106 ceph-mon[32716]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:20:16.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:15 smithi122 ceph-mon[32546]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:16.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:15 smithi122 ceph-mon[32546]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:16.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:15 smithi122 ceph-mon[32546]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:20:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:15 smithi033 ceph-mon[29274]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:15 smithi033 ceph-mon[29274]: from='client.24221 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:16.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:15 smithi033 ceph-mon[29274]: osdmap e28: 4 total, 4 up, 4 in 2024-04-23T18:20:16.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:16 smithi122 ceph-mon[32546]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:20:17.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:16 smithi106 ceph-mon[32716]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:20:17.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:16 smithi033 ceph-mon[29274]: osdmap e29: 4 total, 4 up, 4 in 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/2495431958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]': finished 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/4258548014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/2495431958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]': finished 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/4258548014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:18.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/2495431958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]: dispatch 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b8f450e4-1ae9-4833-90de-3668d806c829"}]': finished 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: osdmap e30: 5 total, 4 up, 5 in 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:18.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/4258548014' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:20.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:19 smithi106 ceph-mon[32716]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-23T18:20:20.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:19 smithi122 ceph-mon[32546]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-23T18:20:20.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:19 smithi033 ceph-mon[29274]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-23T18:20:22.019 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:21 smithi106 ceph-mon[32716]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:20:22.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:21 smithi122 ceph-mon[32546]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:20:22.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:21 smithi033 ceph-mon[29274]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:20:24.044 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:23 smithi106 ceph-mon[32716]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:20:24.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:23 smithi122 ceph-mon[32546]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:20:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:23 smithi033 ceph-mon[29274]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-04-23T18:20:25.018 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:20:25.019 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:20:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:25.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-23T18:20:25.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:26.070 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:25 smithi106 ceph-mon[32716]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:26.071 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:25 smithi106 ceph-mon[32716]: Deploying daemon osd.4 on smithi106 2024-04-23T18:20:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:25 smithi122 ceph-mon[32546]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:25 smithi122 ceph-mon[32546]: Deploying daemon osd.4 on smithi106 2024-04-23T18:20:26.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:25 smithi033 ceph-mon[29274]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:26.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:25 smithi033 ceph-mon[29274]: Deploying daemon osd.4 on smithi106 2024-04-23T18:20:27.971 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:27 smithi106 ceph-mon[32716]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:27 smithi122 ceph-mon[32546]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:28.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:27 smithi033 ceph-mon[29274]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:30.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:29 smithi106 ceph-mon[32716]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:30.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:30.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:29 smithi122 ceph-mon[32546]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:30.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:29 smithi033 ceph-mon[29274]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:30.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:32.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:31 smithi033 ceph-mon[29274]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:31 smithi106 ceph-mon[32716]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:31 smithi122 ceph-mon[32546]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:33.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.495+0000 7f6763fff700 1 -- 172.21.15.106:0/731487798 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f677c06b8d0 con 0x7f67640a3c40 2024-04-23T18:20:33.488 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 4 on host 'smithi106' 2024-04-23T18:20:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 msgr2=0x7f67640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 0x7f67640a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).stop 2024-04-23T18:20:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 msgr2=0x7f677c19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f676c00b500 tx=0x7f676c00b8c0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 shutdown_connections 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f67640a3c40 0x7f67640a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677c103550 0x7f677c1a2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f677c1019b0 0x7f677c19af70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 --2- 172.21.15.106:0/731487798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f677c102bb0 0x7f677c19b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 >> 172.21.15.106:0/731487798 conn(0x7f677c0fcf70 msgr2=0x7f677c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 shutdown_connections 2024-04-23T18:20:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:20:33.497+0000 7f67810f6700 1 -- 172.21.15.106:0/731487798 wait complete. 2024-04-23T18:20:33.590 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:33.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:33.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:33.591 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.210 DEBUG:teuthology.orchestra.run.smithi106:osd.4> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.4.service 2024-04-23T18:20:34.212 INFO:tasks.cephadm:Deploying osd.5 on smithi122 with /dev/vg_nvme/lv_3... 2024-04-23T18:20:34.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-23T18:20:34.453 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:20:34.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:34.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:34 smithi122 ceph-mon[32546]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:34.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:34 smithi122 ceph-mon[32546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:34 smithi033 ceph-mon[29274]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:34.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:34 smithi033 ceph-mon[29274]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:34.745 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.746 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:34.746 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:34.746 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:34 smithi106 ceph-mon[32716]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:34.746 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:34 smithi106 ceph-mon[32716]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:35.665 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-mon[32716]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:35.666 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T18:20:35.501+0000 7f55c933e700 -1 osd.4 0 waiting for initial osdmap 2024-04-23T18:20:35.666 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 18:20:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T18:20:35.507+0000 7f55c5131700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:20:35.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:35.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:20:35.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:35.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:20:35.773 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:35.773 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:35 smithi122 ceph-mon[32546]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: from='osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: osdmap e31: 5 total, 4 up, 5 in 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:35 smithi033 ceph-mon[29274]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:20:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-23T18:20:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-23T18:20:36.338 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-23T18:20:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-23T18:20:36.338 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011033 s, 950 MB/s 2024-04-23T18:20:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-23T18:20:36.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:36 smithi106 ceph-mon[32716]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:20:36.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:36 smithi106 ceph-mon[32716]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:20:36.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:36 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:36 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:36 smithi122 ceph-mon[32546]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:20:36.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:36 smithi122 ceph-mon[32546]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:20:36.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:36 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:36 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:36 smithi033 ceph-mon[29274]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:20:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:36 smithi033 ceph-mon[29274]: osdmap e32: 5 total, 4 up, 5 in 2024-04-23T18:20:36.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:36 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:36 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:36.983 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-04-23T18:20:37.223 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:20:37.660 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] boot 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.661 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:37.662 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:37.662 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] boot 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:37.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:37.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 123 MiB used, 357 GiB / 358 GiB avail 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: osd.4 [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] boot 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: osdmap e33: 5 total, 5 up, 5 in 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:38.657 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:38 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi106 2024-04-23T18:20:38.657 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:38 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-23T18:20:38.657 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:38 smithi122 ceph-mon[32546]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:38.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 -- 172.21.15.122:0/3260120585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0102ba0 msgr2=0x7fb8e0103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 --2- 172.21.15.122:0/3260120585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0102ba0 0x7fb8e0103000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0009990 tx=0x7fb8d0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 -- 172.21.15.122:0/3260120585 shutdown_connections 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 --2- 172.21.15.122:0/3260120585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0103540 0x7fb8e0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 --2- 172.21.15.122:0/3260120585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 0x7fb8e0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 --2- 172.21.15.122:0/3260120585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0102ba0 0x7fb8e0103000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 -- 172.21.15.122:0/3260120585 >> 172.21.15.122:0/3260120585 conn(0x7fb8e00fcf40 msgr2=0x7fb8e00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 -- 172.21.15.122:0/3260120585 shutdown_connections 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.867+0000 7fb8e7375700 1 -- 172.21.15.122:0/3260120585 wait complete. 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 Processor -- start 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 -- start start 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 0x7fb8e006b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0102ba0 0x7fb8e019fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010d4b0 con 0x7fb8e0102ba0 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010d330 con 0x7fb8e01019a0 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e7375700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e010d630 con 0x7fb8e0103540 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33118/0 (socket says 172.21.15.122:33118) 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 -- 172.21.15.122:0/2107877903 learned_addr learned my addr 172.21.15.122:0/2107877903 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e4910700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0102ba0 0x7fb8e019fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5111700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 0x7fb8e006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 -- 172.21.15.122:0/2107877903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 msgr2=0x7fb8e006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 0x7fb8e006b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 -- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0102ba0 msgr2=0x7fb8e019fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0102ba0 0x7fb8e019fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 -- 172.21.15.122:0/2107877903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e01a2af0 con 0x7fb8e0103540 2024-04-23T18:20:38.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.868+0000 7fb8e5912700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8007d70 tx=0x7fb8d800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.869+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8016020 con 0x7fb8e0103540 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.869+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d800ad40 con 0x7fb8e0103540 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.869+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e01a2de0 con 0x7fb8e0103540 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.869+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d80115e0 con 0x7fb8e0103540 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.869+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a32a0 con 0x7fb8e0103540 2024-04-23T18:20:38.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.870+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e0069730 con 0x7fb8e0103540 2024-04-23T18:20:38.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.871+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb8d8019020 con 0x7fb8e0103540 2024-04-23T18:20:38.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.871+0000 7fb8d67fc700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 0x7fb8cc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:20:38.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.872+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fb8d80cf340 con 0x7fb8e0103540 2024-04-23T18:20:38.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.872+0000 7fb8e5111700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 0x7fb8cc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:20:38.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.872+0000 7fb8e5111700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 0x7fb8cc0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc006fd0 tx=0x7fb8dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:20:38.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:38 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi106 2024-04-23T18:20:38.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:38 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-23T18:20:38.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:38 smithi106 ceph-mon[32716]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:38.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:38.876+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8d8020020 con 0x7fb8e0103540 2024-04-23T18:20:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:38 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi106 2024-04-23T18:20:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:38 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-23T18:20:38.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:38 smithi033 ceph-mon[29274]: osdmap e34: 5 total, 5 up, 5 in 2024-04-23T18:20:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:39.086+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb8e0062e80 con 0x7fb8cc0a3c40 2024-04-23T18:20:39.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:39 smithi122 ceph-mon[32546]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:39.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:39 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:39.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:39 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:39.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:39 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:39 smithi106 ceph-mon[32716]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:39 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:39 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:39.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:39 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:39.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:39 smithi033 ceph-mon[29274]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:39.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:20:39.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:20:39.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:39 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:40.854 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:40 smithi122 ceph-mon[32546]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:40.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:40 smithi106 ceph-mon[32716]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:40.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:40 smithi033 ceph-mon[29274]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:20:41.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:41 smithi106 ceph-mon[32716]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:41.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:41 smithi122 ceph-mon[32546]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:41.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:41 smithi033 ceph-mon[29274]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/328313497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]': finished 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:42.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/3887225815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:42.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/328313497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]': finished 2024-04-23T18:20:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/3887225815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:42.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/328313497' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]: dispatch 2024-04-23T18:20:42.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3bd5fd8-f114-476a-be25-de2b536d9601"}]': finished 2024-04-23T18:20:42.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: osdmap e35: 6 total, 5 up, 6 in 2024-04-23T18:20:42.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:20:42.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/3887225815' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:20:43.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:43 smithi033 ceph-mon[29274]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:44.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:43 smithi106 ceph-mon[32716]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:43 smithi122 ceph-mon[32546]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:45.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:45 smithi122 ceph-mon[32546]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:45.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:45 smithi033 ceph-mon[29274]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:46.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:45 smithi106 ceph-mon[32716]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:47.750 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:47 smithi122 ceph-mon[32546]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:47.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:47 smithi033 ceph-mon[29274]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:48.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:47 smithi106 ceph-mon[32716]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:49 smithi033 ceph-mon[29274]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:50.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:49 smithi122 ceph-mon[32546]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:50.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:50.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:49 smithi106 ceph-mon[32716]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-23T18:20:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:50.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:50 smithi033 ceph-mon[29274]: Deploying daemon osd.5 on smithi122 2024-04-23T18:20:51.052 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:50 smithi122 ceph-mon[32546]: Deploying daemon osd.5 on smithi122 2024-04-23T18:20:51.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:50 smithi106 ceph-mon[32716]: Deploying daemon osd.5 on smithi122 2024-04-23T18:20:51.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:51 smithi033 ceph-mon[29274]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:52.046 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:51 smithi122 ceph-mon[32546]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:52.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:51 smithi106 ceph-mon[32716]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:53.812 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:53 smithi122 ceph-mon[32546]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:53.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:53 smithi033 ceph-mon[29274]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:54.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:53 smithi106 ceph-mon[32716]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:55.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:55 smithi033 ceph-mon[29274]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:55.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:55.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:55.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:56.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:55 smithi122 ceph-mon[32546]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:56.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:56.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:56.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:56.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:55 smithi106 ceph-mon[32716]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:57.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:57.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:57.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:57.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:57.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:57.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:58.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:58.116 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:58.116 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:58.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:20:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:20:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.506+0000 7fb8d67fc700 1 -- 172.21.15.122:0/2107877903 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8e0062e80 con 0x7fb8cc0a3c40 2024-04-23T18:20:58.499 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 5 on host 'smithi122' 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 msgr2=0x7fb8cc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 0x7fb8cc0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc006fd0 tx=0x7fb8dc008040 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 msgr2=0x7fb8e01a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8007d70 tx=0x7fb8d800a340 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 shutdown_connections 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb8cc0a3c40 0x7fb8cc0a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8e0103540 0x7fb8e01a2310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8e01019a0 0x7fb8e006b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 --2- 172.21.15.122:0/2107877903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8e0102ba0 0x7fb8e019fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 >> 172.21.15.122:0/2107877903 conn(0x7fb8e00fcf40 msgr2=0x7fb8e0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 shutdown_connections 2024-04-23T18:20:58.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:20:58.508+0000 7fb8e7375700 1 -- 172.21.15.122:0/2107877903 wait complete. 2024-04-23T18:20:59.137 DEBUG:teuthology.orchestra.run.smithi122:osd.5> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.5.service 2024-04-23T18:20:59.139 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-04-23T18:20:59.140 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:20:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:59.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:20:59 smithi033 ceph-mon[29274]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:59.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:20:59 smithi106 ceph-mon[32716]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:20:59.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:20:59 smithi122 ceph-mon[32546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-23T18:21:00.608 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-mon[32546]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:21:00.608 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-mon[32546]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:00.609 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-mon[32546]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:21:00.609 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:00.609 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-mon[32546]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:00.609 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T18:21:00.481+0000 7ff2e2d05700 -1 osd.5 0 waiting for initial osdmap 2024-04-23T18:21:00.609 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 18:21:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T18:21:00.488+0000 7ff2deaf8700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:21:00.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:00 smithi033 ceph-mon[29274]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:21:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:00 smithi033 ceph-mon[29274]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:00 smithi033 ceph-mon[29274]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:21:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:00 smithi033 ceph-mon[29274]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:00.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:00 smithi106 ceph-mon[32716]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-23T18:21:00.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:00 smithi106 ceph-mon[32716]: from='osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:00.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:00 smithi106 ceph-mon[32716]: osdmap e36: 6 total, 5 up, 6 in 2024-04-23T18:21:00.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:00.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:00 smithi106 ceph-mon[32716]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112256 s, 934 MB/s 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 2024-04-23T18:21:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] boot 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:21:01.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] boot 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:21:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 150 MiB used, 447 GiB / 447 GiB avail 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: osdmap e37: 6 total, 5 up, 6 in 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: osd.5 [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] boot 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: osdmap e38: 6 total, 6 up, 6 in 2024-04-23T18:21:01.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:21:02.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-04-23T18:21:02.264 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:02.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:02.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:02 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:02.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:02.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:02 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: osdmap e39: 6 total, 6 up, 6 in 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:02 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:03.647 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:03 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi122 2024-04-23T18:21:03.647 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:03 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-23T18:21:03.647 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:03 smithi033 ceph-mon[29274]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:03.648 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:03 smithi033 ceph-mon[29274]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:21:03.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:03 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi122 2024-04-23T18:21:03.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:03 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-23T18:21:03.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:03 smithi122 ceph-mon[32546]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:03.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:03 smithi122 ceph-mon[32546]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:21:03.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:03 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi122 2024-04-23T18:21:03.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:03 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-23T18:21:03.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:03 smithi106 ceph-mon[32716]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:03.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:03 smithi106 ceph-mon[32716]: osdmap e40: 6 total, 6 up, 6 in 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- 172.21.15.33:0/210231973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 msgr2=0x7f1424103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 --2- 172.21.15.33:0/210231973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424103020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f14140099c0 tx=0x7f1414009cd0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- 172.21.15.33:0/210231973 shutdown_connections 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 --2- 172.21.15.33:0/210231973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 0x7f1424109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 --2- 172.21.15.33:0/210231973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 --2- 172.21.15.33:0/210231973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424103020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- 172.21.15.33:0/210231973 >> 172.21.15.33:0/210231973 conn(0x7f14240fcfa0 msgr2=0x7f14240ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- 172.21.15.33:0/210231973 shutdown_connections 2024-04-23T18:21:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- 172.21.15.33:0/210231973 wait complete. 2024-04-23T18:21:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 Processor -- start 2024-04-23T18:21:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.888+0000 7f142b2c8700 1 -- start start 2024-04-23T18:21:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 0x7f14241a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f142410d430 con 0x7f1424102bc0 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f142410d2b0 con 0x7f14241019c0 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f142b2c8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f142410d5b0 con 0x7f1424103560 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56516/0 (socket says 172.21.15.33:56516) 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 -- 172.21.15.33:0/1919667048 learned_addr learned my addr 172.21.15.33:0/1919667048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1429064700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1429865700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 0x7f14241a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 -- 172.21.15.33:0/1919667048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 msgr2=0x7f14241a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 0x7f14241a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 -- 172.21.15.33:0/1919667048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 msgr2=0x7f1424198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 -- 172.21.15.33:0/1919667048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14241a0830 con 0x7f1424102bc0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1429064700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f1428863700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f141400b580 tx=0x7f141400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.889+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414027070 con 0x7f1424102bc0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.890+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14241a0ac0 con 0x7f1424102bc0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.890+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141400fe30 con 0x7f1424102bc0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.890+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14140173f0 con 0x7f1424102bc0 2024-04-23T18:21:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.890+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14241a0f20 con 0x7f1424102bc0 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.891+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1424069730 con 0x7f1424102bc0 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.892+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f141402a070 con 0x7f1424102bc0 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.892+0000 7f141a7fc700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 0x7f14100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.892+0000 7f1429064700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 0x7f14100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.892+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4533+0+0 (secure 0 0 0) 0x7f14140d0170 con 0x7f1424102bc0 2024-04-23T18:21:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.893+0000 7f1429064700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 0x7f14100a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1424102a20 tx=0x7f1420009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:03.897+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1414013070 con 0x7f1424102bc0 2024-04-23T18:21:04.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:04.115+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f14240ff160 con 0x7f14100a3c40 2024-04-23T18:21:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:04 smithi106 ceph-mon[32716]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:21:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:04.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:04 smithi033 ceph-mon[29274]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:21:04.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:04.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:04.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:04.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:04 smithi122 ceph-mon[32546]: osdmap e41: 6 total, 6 up, 6 in 2024-04-23T18:21:04.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:04.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:04.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:05.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:05 smithi106 ceph-mon[32716]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:05.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:05 smithi106 ceph-mon[32716]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:05.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:05 smithi106 ceph-mon[32716]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:21:05.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:05 smithi033 ceph-mon[29274]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:05.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:05 smithi033 ceph-mon[29274]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:05.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:05 smithi033 ceph-mon[29274]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:21:05.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:05 smithi122 ceph-mon[32546]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:05.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:05 smithi122 ceph-mon[32546]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:05.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:05 smithi122 ceph-mon[32546]: osdmap e42: 6 total, 6 up, 6 in 2024-04-23T18:21:07.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:07 smithi106 ceph-mon[32716]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-23T18:21:07.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:07 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]: dispatch 2024-04-23T18:21:07.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:07 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]': finished 2024-04-23T18:21:07.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:07 smithi106 ceph-mon[32716]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:21:07.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:07.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:07 smithi122 ceph-mon[32546]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-23T18:21:07.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:07 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]: dispatch 2024-04-23T18:21:07.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:07 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]': finished 2024-04-23T18:21:07.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:07 smithi122 ceph-mon[32546]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:21:07.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:07.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:07 smithi033 ceph-mon[29274]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-23T18:21:07.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:07 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]: dispatch 2024-04-23T18:21:07.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:07 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/280101188' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c84631e0-7950-46a1-9e3a-24f56cab53f5"}]': finished 2024-04-23T18:21:07.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:07 smithi033 ceph-mon[29274]: osdmap e43: 7 total, 6 up, 7 in 2024-04-23T18:21:07.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:08.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:08 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1222546714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:08.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:08 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1222546714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:08.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:08 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1222546714' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:09.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:09 smithi106 ceph-mon[32716]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:21:09.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:09 smithi122 ceph-mon[32546]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:21:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:09 smithi033 ceph-mon[29274]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:21:11.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:11 smithi033 ceph-mon[29274]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-23T18:21:11.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:11 smithi106 ceph-mon[32716]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-23T18:21:11.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:11 smithi122 ceph-mon[32546]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-04-23T18:21:13.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:13 smithi033 ceph-mon[29274]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:13.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:13 smithi106 ceph-mon[32716]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:13.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:13 smithi122 ceph-mon[32546]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-23T18:21:15.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:15 smithi033 ceph-mon[29274]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T18:21:15.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:15 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:21:15.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:15 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:15.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:15 smithi033 ceph-mon[29274]: Deploying daemon osd.6 on smithi033 2024-04-23T18:21:15.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:15 smithi106 ceph-mon[32716]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T18:21:15.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:15 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:21:15.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:15 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:15.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:15 smithi106 ceph-mon[32716]: Deploying daemon osd.6 on smithi033 2024-04-23T18:21:15.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:15 smithi122 ceph-mon[32546]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-23T18:21:15.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:15 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-23T18:21:15.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:15 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:15.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:15 smithi122 ceph-mon[32546]: Deploying daemon osd.6 on smithi033 2024-04-23T18:21:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:17 smithi033 ceph-mon[29274]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:17 smithi106 ceph-mon[32716]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:17.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:17 smithi122 ceph-mon[32546]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:19.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:19 smithi033 ceph-mon[29274]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:19.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:19 smithi106 ceph-mon[32716]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:19.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:19 smithi122 ceph-mon[32546]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:21 smithi122 ceph-mon[32546]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:21.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:21 smithi033 ceph-mon[29274]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:21.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:21.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:21 smithi106 ceph-mon[32716]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:21.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:21.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:23.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:23 smithi033 ceph-mon[29274]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:23.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:23 smithi122 ceph-mon[32546]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:23.773 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-04-23T18:21:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.778+0000 7f141a7fc700 1 -- 172.21.15.33:0/1919667048 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14240ff160 con 0x7f14100a3c40 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.780+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 msgr2=0x7f14100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.780+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 0x7f14100a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1424102a20 tx=0x7f1420009260 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.780+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 msgr2=0x7f1424199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.780+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f141400b580 tx=0x7f141400fa00 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.780+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 shutdown_connections 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14100a3c40 0x7f14100a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1424103560 0x7f14241a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14241019c0 0x7f1424198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 --2- 172.21.15.33:0/1919667048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1424102bc0 0x7f1424199180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 >> 172.21.15.33:0/1919667048 conn(0x7f14240fcfa0 msgr2=0x7f14240fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:23.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 shutdown_connections 2024-04-23T18:21:23.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:21:23.781+0000 7f142b2c8700 1 -- 172.21.15.33:0/1919667048 wait complete. 2024-04-23T18:21:23.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:23 smithi106 ceph-mon[32716]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:24.435 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.6.service 2024-04-23T18:21:24.437 INFO:tasks.cephadm:Deploying osd.7 on smithi106 with /dev/vg_nvme/lv_2... 2024-04-23T18:21:24.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:21:24.681 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:21:24.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:24.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:24.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:24 smithi033 ceph-mon[29274]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:21:24.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:24.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:24 smithi106 ceph-mon[32716]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:21:25.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:24 smithi122 ceph-mon[32546]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-23T18:21:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:25.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:25 smithi033 ceph-mon[29274]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:25.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:25 smithi033 ceph-mon[29274]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:21:25.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:25 smithi033 ceph-mon[29274]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:21:25.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:25 smithi033 ceph-mon[29274]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:21:25.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:26.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:25 smithi106 ceph-mon[32716]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:26.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:25 smithi106 ceph-mon[32716]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:21:26.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:25 smithi106 ceph-mon[32716]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:21:26.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:25 smithi106 ceph-mon[32716]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:21:26.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:25 smithi122 ceph-mon[32546]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:25 smithi122 ceph-mon[32546]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-23T18:21:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:25 smithi122 ceph-mon[32546]: osdmap e44: 7 total, 6 up, 7 in 2024-04-23T18:21:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:25 smithi122 ceph-mon[32546]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:21:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:26.411 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T18:21:26.180+0000 7fbe06346700 -1 osd.6 0 waiting for initial osdmap 2024-04-23T18:21:26.412 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T18:21:26.189+0000 7fbdfe131700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:21:26.645 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:21:26.645 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:21:26.645 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-23T18:21:26.646 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-23T18:21:26.646 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117001 s, 896 MB/s 2024-04-23T18:21:26.646 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-23T18:21:27.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:26 smithi106 ceph-mon[32716]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:21:27.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:26 smithi106 ceph-mon[32716]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:21:27.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.028 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:26 smithi106 ceph-mon[32716]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' 2024-04-23T18:21:27.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:26 smithi122 ceph-mon[32546]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:21:27.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:26 smithi122 ceph-mon[32546]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:21:27.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:26 smithi122 ceph-mon[32546]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' 2024-04-23T18:21:27.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-mon[29274]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:21:27.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-mon[29274]: osdmap e45: 7 total, 6 up, 7 in 2024-04-23T18:21:27.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:26 smithi033 ceph-mon[29274]: from='osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411]' entity='osd.6' 2024-04-23T18:21:27.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-04-23T18:21:27.562 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] boot 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:21:27.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:27 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] boot 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:21:27.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:27 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: osd.6 [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] boot 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: osdmap e46: 7 total, 7 up, 7 in 2024-04-23T18:21:28.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:27 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi033 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 -- 172.21.15.106:0/3059029209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc102ba0 msgr2=0x7fdddc103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 --2- 172.21.15.106:0/3059029209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc102ba0 0x7fdddc103000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fddd00099c0 tx=0x7fddd0009cd0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 -- 172.21.15.106:0/3059029209 shutdown_connections 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 --2- 172.21.15.106:0/3059029209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc103540 0x7fdddc109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 --2- 172.21.15.106:0/3059029209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc1019a0 0x7fdddc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 --2- 172.21.15.106:0/3059029209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc102ba0 0x7fdddc103000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 -- 172.21.15.106:0/3059029209 >> 172.21.15.106:0/3059029209 conn(0x7fdddc0fcf40 msgr2=0x7fdddc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 -- 172.21.15.106:0/3059029209 shutdown_connections 2024-04-23T18:21:29.312 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.316+0000 7fdde2329700 1 -- 172.21.15.106:0/3059029209 wait complete. 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 Processor -- start 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 -- start start 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 0x7fdddc1058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 0x7fdddc103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d4d0 con 0x7fdddc102ba0 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d350 con 0x7fdddc103540 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde2329700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdddc10d650 con 0x7fdddc1019a0 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde08c6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde08c6700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50538/0 (socket says 172.21.15.106:50538) 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdde08c6700 1 -- 172.21.15.106:0/3777003038 learned_addr learned my addr 172.21.15.106:0/3777003038 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.317+0000 7fdddbfff700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 0x7fdddc1058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:29.313 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdddb7fe700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 0x7fdddc103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 -- 172.21.15.106:0/3777003038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 msgr2=0x7fdddc1058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 0x7fdddc1058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 -- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 msgr2=0x7fdddc103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 0x7fdddc103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 -- 172.21.15.106:0/3777003038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdddc104d90 con 0x7fdddc103540 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdddbfff700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 0x7fdddc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdddb7fe700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 0x7fdddc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde08c6700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddd400ada0 tx=0x7fddd400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4017020 con 0x7fdddc103540 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdddc1a2e10 con 0x7fdddc103540 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd400ce30 con 0x7fdddc103540 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4005420 con 0x7fdddc103540 2024-04-23T18:21:29.314 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.318+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdddc1a3260 con 0x7fdddc103540 2024-04-23T18:21:29.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.319+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdddc069760 con 0x7fdddc103540 2024-04-23T18:21:29.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.320+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fddd4004ad0 con 0x7fdddc103540 2024-04-23T18:21:29.315 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.321+0000 7fddd97fa700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 0x7fddc40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:29.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.321+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7fddd40cf4f0 con 0x7fdddc103540 2024-04-23T18:21:29.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.321+0000 7fdddbfff700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 0x7fddc40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:29.316 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.322+0000 7fdddbfff700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 0x7fddc40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdddc102a00 tx=0x7fddcc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:29.318 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.325+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdddc069760 con 0x7fdddc103540 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi033 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:21:29.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:29.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi033 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 5697M 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:29.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: osdmap e47: 7 total, 7 up, 7 in 2024-04-23T18:21:29.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:29.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:29.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:29.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:29.521 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:29.528+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdddc0ff150 con 0x7fddc40a3c40 2024-04-23T18:21:30.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:30 smithi106 ceph-mon[32716]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:30.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:30.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:30.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:30 smithi122 ceph-mon[32546]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:30.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:30 smithi033 ceph-mon[29274]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:31.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:31 smithi106 ceph-mon[32716]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:31.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:31 smithi122 ceph-mon[32546]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:31.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:31 smithi033 ceph-mon[29274]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/1905031228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]': finished 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:33.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/485273181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:33.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:33.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/1905031228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]': finished 2024-04-23T18:21:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:21:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:33.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/485273181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/1905031228' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]: dispatch 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a454eca-15b1-4305-bf15-24a47be985d4"}]': finished 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: osdmap e48: 8 total, 7 up, 8 in 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/485273181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:35.423 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:35 smithi106 ceph-mon[32716]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:35.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:35 smithi122 ceph-mon[32546]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:35.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:35 smithi033 ceph-mon[29274]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:37.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:37 smithi122 ceph-mon[32546]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:37.454 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:37 smithi106 ceph-mon[32716]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:37.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:37 smithi033 ceph-mon[29274]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:39.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:39 smithi122 ceph-mon[32546]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:39.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:39 smithi033 ceph-mon[29274]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:39.537 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:39 smithi106 ceph-mon[32716]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:40.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:40.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:40.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:40 smithi122 ceph-mon[32546]: Deploying daemon osd.7 on smithi106 2024-04-23T18:21:40.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:40.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:40.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:40 smithi033 ceph-mon[29274]: Deploying daemon osd.7 on smithi106 2024-04-23T18:21:40.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-23T18:21:40.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:40.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:40 smithi106 ceph-mon[32716]: Deploying daemon osd.7 on smithi106 2024-04-23T18:21:41.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:41 smithi106 ceph-mon[32716]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:41.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:41 smithi122 ceph-mon[32546]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:41.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:41 smithi033 ceph-mon[29274]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:43.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:43 smithi106 ceph-mon[32716]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:43.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:43 smithi122 ceph-mon[32546]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:43.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:43 smithi033 ceph-mon[29274]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:45.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:45 smithi106 ceph-mon[32716]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:45.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:45 smithi122 ceph-mon[32546]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:45.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:45 smithi033 ceph-mon[29274]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:46 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:46.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:46 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:46.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:46.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:46 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:47.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:47 smithi106 ceph-mon[32716]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:47.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:47 smithi122 ceph-mon[32546]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:47.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:47 smithi033 ceph-mon[29274]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:48.740 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 7 on host 'smithi106' 2024-04-23T18:21:48.740 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.746+0000 7fddd97fa700 1 -- 172.21.15.106:0/3777003038 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdddc0ff150 con 0x7fddc40a3c40 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 msgr2=0x7fddc40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 0x7fddc40a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdddc102a00 tx=0x7fddcc008040 comp rx=0 tx=0).stop 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 msgr2=0x7fdddc1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddd400ada0 tx=0x7fddd400c330 comp rx=0 tx=0).stop 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 shutdown_connections 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fddc40a3c40 0x7fddc40a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdddc103540 0x7fdddc1044c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdddc1019a0 0x7fdddc1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 --2- 172.21.15.106:0/3777003038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdddc102ba0 0x7fdddc103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:48.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.748+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 >> 172.21.15.106:0/3777003038 conn(0x7fdddc0fcf40 msgr2=0x7fdddc0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:48.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.749+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 shutdown_connections 2024-04-23T18:21:48.743 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:21:48.749+0000 7fdde2329700 1 -- 172.21.15.106:0/3777003038 wait complete. 2024-04-23T18:21:49.373 DEBUG:teuthology.orchestra.run.smithi106:osd.7> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.7.service 2024-04-23T18:21:49.375 INFO:tasks.cephadm:Deploying osd.8 on smithi122 with /dev/vg_nvme/lv_2... 2024-04-23T18:21:49.376 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-23T18:21:49.629 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:49.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:49 smithi033 ceph-mon[29274]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:50.036 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:49 smithi122 ceph-mon[32546]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:50.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:49 smithi106 ceph-mon[32716]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-23T18:21:51.016 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:50 smithi122 ceph-mon[32546]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:51.016 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:50 smithi122 ceph-mon[32546]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:21:51.016 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:51.016 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:50 smithi122 ceph-mon[32546]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.016 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:50 smithi122 ceph-mon[32546]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:50 smithi106 ceph-mon[32716]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:51.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:50 smithi106 ceph-mon[32716]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:21:51.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:51.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:50 smithi106 ceph-mon[32716]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.100 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:50 smithi106 ceph-mon[32716]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:50 smithi033 ceph-mon[29274]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-23T18:21:51.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:50 smithi033 ceph-mon[29274]: osdmap e49: 8 total, 7 up, 8 in 2024-04-23T18:21:51.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:51.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:50 smithi033 ceph-mon[29274]: from='osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:50 smithi033 ceph-mon[29274]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:21:51.371 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T18:21:51.256+0000 7f2d814a3700 -1 osd.7 0 waiting for initial osdmap 2024-04-23T18:21:51.371 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T18:21:51.265+0000 7f2d7928e700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111682 s, 939 MB/s 2024-04-23T18:21:51.688 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-23T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-mon[32716]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-mon[32716]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-mon[32716]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:51 smithi106 ceph-mon[32716]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:52.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:51 smithi122 ceph-mon[32546]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:52.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:51 smithi122 ceph-mon[32546]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:21:52.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:51 smithi122 ceph-mon[32546]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:52.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:51 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:52.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:51 smithi122 ceph-mon[32546]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:52.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:51 smithi033 ceph-mon[29274]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-04-23T18:21:52.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:51 smithi033 ceph-mon[29274]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:21:52.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:51 smithi033 ceph-mon[29274]: osdmap e50: 8 total, 7 up, 8 in 2024-04-23T18:21:52.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:51 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:52.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:51 smithi033 ceph-mon[29274]: from='osd.7 ' entity='osd.7' 2024-04-23T18:21:52.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-04-23T18:21:52.569 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] boot 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: osdmap e51: 8 total, 8 up, 8 in 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:52 smithi106 ceph-mon[32716]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] boot 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: osdmap e51: 8 total, 8 up, 8 in 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.200 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:52 smithi122 ceph-mon[32546]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: osd.7 [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] boot 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: osdmap e51: 8 total, 8 up, 8 in 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:21:53.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:52 smithi033 ceph-mon[29274]: osdmap e52: 8 total, 8 up, 8 in 2024-04-23T18:21:54.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 -- 172.21.15.122:0/3341256768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c1019a0 msgr2=0x7f408c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 --2- 172.21.15.122:0/3341256768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c1019a0 0x7f408c101dc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f407c009990 tx=0x7f407c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 -- 172.21.15.122:0/3341256768 shutdown_connections 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 --2- 172.21.15.122:0/3341256768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 0x7f408c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 --2- 172.21.15.122:0/3341256768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c1019a0 0x7f408c101dc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 --2- 172.21.15.122:0/3341256768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c102ba0 0x7f408c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 -- 172.21.15.122:0/3341256768 >> 172.21.15.122:0/3341256768 conn(0x7f408c0fcf40 msgr2=0x7f408c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 -- 172.21.15.122:0/3341256768 shutdown_connections 2024-04-23T18:21:54.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.052+0000 7f4091c00700 1 -- 172.21.15.122:0/3341256768 wait complete. 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 Processor -- start 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 -- start start 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c1019a0 0x7f408c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 0x7f408c1a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:54.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f408c10d430 con 0x7f408c103540 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f408c10d2b0 con 0x7f408c1019a0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4091c00700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f408c10d5b0 con 0x7f408c102ba0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4083fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4083fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49688/0 (socket says 172.21.15.122:49688) 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f4083fff700 1 -- 172.21.15.122:0/2501651301 learned_addr learned my addr 172.21.15.122:0/2501651301 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.053+0000 7f408b7fe700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c1019a0 0x7f408c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f408bfff700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 0x7f408c1a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 -- 172.21.15.122:0/2501651301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c1019a0 msgr2=0x7f408c06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c1019a0 0x7f408c06b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 -- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 msgr2=0x7f408c1a0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 0x7f408c1a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 -- 172.21.15.122:0/2501651301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408c1a08f0 con 0x7f408c102ba0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4083fff700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f407400e9c0 tx=0x7f407400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:54.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f407400cbd0 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408c1a0be0 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f407400cd70 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4074005610 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.054+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408c1a1120 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.055+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408c069730 con 0x7f408c102ba0 2024-04-23T18:21:54.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.057+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4074004750 con 0x7f408c102ba0 2024-04-23T18:21:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.057+0000 7f40897fa700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 0x7f40780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:21:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.058+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f40740cfdb0 con 0x7f408c102ba0 2024-04-23T18:21:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.058+0000 7f408b7fe700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 0x7f40780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:21:54.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.058+0000 7f408b7fe700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 0x7f40780a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b2e0 tx=0x7f407c00b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:21:54.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.061+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4074053ff0 con 0x7f408c102ba0 2024-04-23T18:21:54.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:53 smithi106 ceph-mon[32716]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:54.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:53 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi106 2024-04-23T18:21:54.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:53 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-23T18:21:54.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:53 smithi106 ceph-mon[32716]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:54.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:53 smithi122 ceph-mon[32546]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:54.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:53 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi106 2024-04-23T18:21:54.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:53 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-23T18:21:54.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:53 smithi122 ceph-mon[32546]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:53 smithi033 ceph-mon[29274]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:53 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi106 2024-04-23T18:21:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:53 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-23T18:21:54.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:53 smithi033 ceph-mon[29274]: osdmap e53: 8 total, 8 up, 8 in 2024-04-23T18:21:54.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:21:54.271+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f408c062e80 con 0x7f40780a3c40 2024-04-23T18:21:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:54 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:55.128 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:55.128 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:55.128 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:54 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:21:55.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:21:55.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:54 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:21:56.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:55 smithi106 ceph-mon[32716]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:55 smithi106 ceph-mon[32716]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:56.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:55 smithi106 ceph-mon[32716]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:56.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:55 smithi122 ceph-mon[32546]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:56.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:55 smithi122 ceph-mon[32546]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:56.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:55 smithi122 ceph-mon[32546]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:55 smithi033 ceph-mon[29274]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:56.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:55 smithi033 ceph-mon[29274]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:21:56.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:55 smithi033 ceph-mon[29274]: osdmap e54: 8 total, 8 up, 8 in 2024-04-23T18:21:57.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:56 smithi106 ceph-mon[32716]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:57.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:56 smithi122 ceph-mon[32546]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:57.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:56 smithi033 ceph-mon[29274]: osdmap e55: 8 total, 8 up, 8 in 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/539386097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]': finished 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: osdmap e56: 9 total, 8 up, 9 in 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:58.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:57 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/1800351338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/539386097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]': finished 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: osdmap e56: 9 total, 8 up, 9 in 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:58.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:57 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/1800351338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/539386097' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]: dispatch 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "574d7ce1-62ac-4720-9f2a-d40cdf230ffc"}]': finished 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: osdmap e56: 9 total, 8 up, 9 in 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:21:58.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:57 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/1800351338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:00.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:21:59 smithi106 ceph-mon[32716]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:00.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:21:59 smithi122 ceph-mon[32546]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:00.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:21:59 smithi033 ceph-mon[29274]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:02.122 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:01 smithi122 ceph-mon[32546]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:02.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:01 smithi033 ceph-mon[29274]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:02.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:01 smithi106 ceph-mon[32716]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:04.098 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:03 smithi122 ceph-mon[32546]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:04.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:03 smithi033 ceph-mon[29274]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:04.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:03 smithi106 ceph-mon[32716]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:05.088 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:22:05.088 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:05.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:22:05.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:05.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-23T18:22:05.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:05.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:05 smithi122 ceph-mon[32546]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:05.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:05 smithi122 ceph-mon[32546]: Deploying daemon osd.8 on smithi122 2024-04-23T18:22:06.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:05 smithi033 ceph-mon[29274]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:06.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:05 smithi033 ceph-mon[29274]: Deploying daemon osd.8 on smithi122 2024-04-23T18:22:06.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:05 smithi106 ceph-mon[32716]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:06.291 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:05 smithi106 ceph-mon[32716]: Deploying daemon osd.8 on smithi122 2024-04-23T18:22:08.111 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:07 smithi122 ceph-mon[32546]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:08.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:07 smithi033 ceph-mon[29274]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:07 smithi106 ceph-mon[32716]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:10.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:09 smithi122 ceph-mon[32546]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:10.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:09 smithi033 ceph-mon[29274]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:10.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:09 smithi106 ceph-mon[32716]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:11.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:11 smithi106 ceph-mon[32716]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:11.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:11.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:11 smithi122 ceph-mon[32546]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:11.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:11 smithi033 ceph-mon[29274]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:11.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:13.402 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:13 smithi122 ceph-mon[32546]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:13.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:13 smithi033 ceph-mon[29274]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:13.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:13 smithi106 ceph-mon[32716]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:14.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.139+0000 7f40897fa700 1 -- 172.21.15.122:0/2501651301 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f408c062e80 con 0x7f40780a3c40 2024-04-23T18:22:14.135 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 8 on host 'smithi122' 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 msgr2=0x7f40780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 0x7f40780a6100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b2e0 tx=0x7f407c00b7f0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 msgr2=0x7f408c19d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f407400e9c0 tx=0x7f407400ecd0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 shutdown_connections 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40780a3c40 0x7f40780a6100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f408c103540 0x7f408c1a0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f408c1019a0 0x7f408c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 --2- 172.21.15.122:0/2501651301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f408c102ba0 0x7f408c19d9d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:14.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 >> 172.21.15.122:0/2501651301 conn(0x7f408c0fcf40 msgr2=0x7f408c106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:14.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 shutdown_connections 2024-04-23T18:22:14.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:22:14.142+0000 7f4091c00700 1 -- 172.21.15.122:0/2501651301 wait complete. 2024-04-23T18:22:14.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:14.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:14.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:14.783 DEBUG:teuthology.orchestra.run.smithi122:osd.8> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.8.service 2024-04-23T18:22:14.785 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-04-23T18:22:14.786 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:22:15.036 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:15.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-mon[32546]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:15.449 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T18:22:15.285+0000 7f978e9c1700 -1 osd.8 0 waiting for initial osdmap 2024-04-23T18:22:15.450 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 18:22:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T18:22:15.295+0000 7f978a7b4700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:22:15.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:15.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:22:15.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:22:15.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:15.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:15.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:15 smithi033 ceph-mon[29274]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:15.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:15.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-23T18:22:15.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: osdmap e57: 9 total, 8 up, 9 in 2024-04-23T18:22:15.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:15.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: from='osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:15.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:15 smithi106 ceph-mon[32716]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:22:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] boot 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: osdmap e59: 9 total, 9 up, 9 in 2024-04-23T18:22:16.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:16 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] boot 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: osdmap e59: 9 total, 9 up, 9 in 2024-04-23T18:22:16.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:16 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: osdmap e58: 9 total, 8 up, 9 in 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: osd.8 [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] boot 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: osdmap e59: 9 total, 9 up, 9 in 2024-04-23T18:22:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:16 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:22:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:22:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:22:17.136 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-23T18:22:17.136 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-23T18:22:17.137 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111955 s, 937 MB/s 2024-04-23T18:22:17.137 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-23T18:22:17.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:17 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:22:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:17 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:22:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:17 smithi122 ceph-mon[32546]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:17 smithi122 ceph-mon[32546]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:22:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:17 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:22:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:17 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:22:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:17 smithi033 ceph-mon[29274]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:17 smithi033 ceph-mon[29274]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:22:17.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-04-23T18:22:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:17 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:22:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:17 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:22:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:17 smithi106 ceph-mon[32716]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-04-23T18:22:17.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:17 smithi106 ceph-mon[32716]: osdmap e60: 9 total, 9 up, 9 in 2024-04-23T18:22:18.114 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi122 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:18.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi122 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: osdmap e61: 9 total, 9 up, 9 in 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi122 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:18.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:18.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:18.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 -- 172.21.15.33:0/1025730377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8108790 msgr2=0x7fb6a810ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 --2- 172.21.15.33:0/1025730377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8108790 0x7fb6a810ac00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009990 tx=0x7fb6a0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 -- 172.21.15.33:0/1025730377 shutdown_connections 2024-04-23T18:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 --2- 172.21.15.33:0/1025730377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8108790 0x7fb6a810ac00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 --2- 172.21.15.33:0/1025730377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8103530 0x7fb6a8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 --2- 172.21.15.33:0/1025730377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 0x7fb6a8108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 -- 172.21.15.33:0/1025730377 >> 172.21.15.33:0/1025730377 conn(0x7fb6a80fcf40 msgr2=0x7fb6a80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 -- 172.21.15.33:0/1025730377 shutdown_connections 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.864+0000 7fb6af588700 1 -- 172.21.15.33:0/1025730377 wait complete. 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 Processor -- start 2024-04-23T18:22:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 -- start start 2024-04-23T18:22:19.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:19.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 0x7fb6a81014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:19.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8108790 0x7fb6a8101a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:19.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a810e550 con 0x7fb6a8103530 2024-04-23T18:22:19.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a810e3d0 con 0x7fb6a8108790 2024-04-23T18:22:19.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6af588700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a810e6d0 con 0x7fb6a8105e60 2024-04-23T18:22:19.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6ad324700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:19.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6ad324700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41994/0 (socket says 172.21.15.33:41994) 2024-04-23T18:22:19.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6ad324700 1 -- 172.21.15.33:0/3145389164 learned_addr learned my addr 172.21.15.33:0/3145389164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:22:19.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6adb25700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8108790 0x7fb6a8101a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.865+0000 7fb6acb23700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 0x7fb6a81014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 -- 172.21.15.33:0/3145389164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 msgr2=0x7fb6a81014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 0x7fb6a81014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 -- 172.21.15.33:0/3145389164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8108790 msgr2=0x7fb6a8101a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8108790 0x7fb6a8101a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:19.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 -- 172.21.15.33:0/3145389164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a81022d0 con 0x7fb6a8103530 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6ad324700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb698007e20 tx=0x7fb69800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698016020 con 0x7fb6a8103530 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb69800ac90 con 0x7fb6a8103530 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a81a2d20 con 0x7fb6a8103530 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6980115e0 con 0x7fb6a8103530 2024-04-23T18:22:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.866+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a81a31b0 con 0x7fb6a8103530 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.867+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a802df80 con 0x7fb6a8103530 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.868+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb698019020 con 0x7fb6a8103530 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.869+0000 7fb6a67fc700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 0x7fb6900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.869+0000 7fb6acb23700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 0x7fb6900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.869+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7fb6980cf450 con 0x7fb6a8103530 2024-04-23T18:22:19.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.869+0000 7fb6acb23700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 0x7fb6900a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005950 tx=0x7fb69c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:19.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:19.873+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb698020070 con 0x7fb6a8103530 2024-04-23T18:22:19.870 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:19 smithi033 ceph-mon[29274]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:19.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:19 smithi033 ceph-mon[29274]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:22:20.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:20.092+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb6a8062e80 con 0x7fb6900a3cc0 2024-04-23T18:22:20.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:19 smithi106 ceph-mon[32716]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:20.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:19 smithi106 ceph-mon[32716]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:22:20.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:19 smithi122 ceph-mon[32546]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:20.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:19 smithi122 ceph-mon[32546]: osdmap e62: 9 total, 9 up, 9 in 2024-04-23T18:22:20.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:20 smithi033 ceph-mon[29274]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:22:20.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:20 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:20.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:20 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:20.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:20 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:21.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:20 smithi106 ceph-mon[32716]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:22:21.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:20 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:21.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:20 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:21.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:20 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:21.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:20 smithi122 ceph-mon[32546]: osdmap e63: 9 total, 9 up, 9 in 2024-04-23T18:22:21.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:20 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:21.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:20 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:21.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:20 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:21.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:21 smithi033 ceph-mon[29274]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:21.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:21 smithi033 ceph-mon[29274]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:22.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:21 smithi106 ceph-mon[32716]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:22.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:21 smithi106 ceph-mon[32716]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:22.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:21 smithi122 ceph-mon[32546]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:22.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:21 smithi122 ceph-mon[32546]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:24.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:22:24.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]: dispatch 2024-04-23T18:22:24.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]': finished 2024-04-23T18:22:24.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: osdmap e64: 10 total, 9 up, 10 in 2024-04-23T18:22:24.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:24.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4079503277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:24.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:22:24.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]: dispatch 2024-04-23T18:22:24.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]': finished 2024-04-23T18:22:24.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: osdmap e64: 10 total, 9 up, 10 in 2024-04-23T18:22:24.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:24.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4079503277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:24.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-23T18:22:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]: dispatch 2024-04-23T18:22:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3660678311' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0449b46b-87c2-4174-b0fc-623efb4f453d"}]': finished 2024-04-23T18:22:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: osdmap e64: 10 total, 9 up, 10 in 2024-04-23T18:22:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:24.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4079503277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:26.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:25 smithi106 ceph-mon[32716]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:22:26.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:25 smithi122 ceph-mon[32546]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:22:26.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:25 smithi033 ceph-mon[29274]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-23T18:22:28.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:27 smithi106 ceph-mon[32716]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-23T18:22:28.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:27 smithi033 ceph-mon[29274]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-23T18:22:28.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:27 smithi122 ceph-mon[32546]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-23T18:22:30.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:29 smithi106 ceph-mon[32716]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-23T18:22:30.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:29 smithi122 ceph-mon[32546]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-23T18:22:30.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:29 smithi033 ceph-mon[29274]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-23T18:22:31.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:22:31.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:22:31.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:31.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-23T18:22:31.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:32.080 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:31 smithi033 ceph-mon[29274]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:22:32.080 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:31 smithi033 ceph-mon[29274]: Deploying daemon osd.9 on smithi033 2024-04-23T18:22:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:31 smithi106 ceph-mon[32716]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:22:32.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:31 smithi106 ceph-mon[32716]: Deploying daemon osd.9 on smithi033 2024-04-23T18:22:32.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:31 smithi122 ceph-mon[32546]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-23T18:22:32.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:31 smithi122 ceph-mon[32546]: Deploying daemon osd.9 on smithi033 2024-04-23T18:22:33.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:33 smithi033 ceph-mon[29274]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:34.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:33 smithi106 ceph-mon[32716]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:34.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:33 smithi122 ceph-mon[32546]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:35.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:35 smithi033 ceph-mon[29274]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:35 smithi106 ceph-mon[32716]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:36.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:35 smithi122 ceph-mon[32546]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:37 smithi122 ceph-mon[32546]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:37.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:37 smithi033 ceph-mon[29274]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:37.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:37.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:37.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:38.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:37 smithi106 ceph-mon[32716]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:38.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:38.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:38.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:39.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:39 smithi033 ceph-mon[29274]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:39.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:39 smithi122 ceph-mon[32546]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:39 smithi106 ceph-mon[32716]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:40.767 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-04-23T18:22:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.773+0000 7fb6a67fc700 1 -- 172.21.15.33:0/3145389164 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb6a8062e80 con 0x7fb6900a3cc0 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 msgr2=0x7fb6900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 0x7fb6900a6180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb69c005950 tx=0x7fb69c00b410 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 msgr2=0x7fb6a8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb698007e20 tx=0x7fb69800a340 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 shutdown_connections 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb6900a3cc0 0x7fb6900a6180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6a8108790 0x7fb6a8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6a8103530 0x7fb6a8102e90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 --2- 172.21.15.33:0/3145389164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6a8105e60 0x7fb6a81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.776+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 >> 172.21.15.33:0/3145389164 conn(0x7fb6a80fcf40 msgr2=0x7fb6a8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.777+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 shutdown_connections 2024-04-23T18:22:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:22:40.777+0000 7fb6af588700 1 -- 172.21.15.33:0/3145389164 wait complete. 2024-04-23T18:22:40.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:40.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:40.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:40.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:40.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:40.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:40 smithi033 ceph-mon[29274]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:41.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:41.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:40 smithi106 ceph-mon[32716]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:41.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:41.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:40 smithi122 ceph-mon[32546]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-23T18:22:41.415 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.9.service 2024-04-23T18:22:41.417 INFO:tasks.cephadm:Deploying osd.10 on smithi106 with /dev/vg_nvme/lv_1... 2024-04-23T18:22:41.418 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:22:41.661 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:22:41.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:41.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:41 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:22:41.987 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:41.988 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:41 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: osdmap e65: 10 total, 9 up, 10 in 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:42.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:41 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:22:42.481 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T18:22:42.175+0000 7f4807051700 -1 osd.9 0 waiting for initial osdmap 2024-04-23T18:22:42.481 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T18:22:42.185+0000 7f47fee3c700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:22:42.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-mon[29274]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:22:42.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-mon[29274]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:42.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:42.902 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:42.902 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:42 smithi033 ceph-mon[29274]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' 2024-04-23T18:22:43.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:42 smithi106 ceph-mon[32716]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:22:43.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:42 smithi106 ceph-mon[32716]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:43.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:42 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:43.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:42 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:43.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:42 smithi106 ceph-mon[32716]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' 2024-04-23T18:22:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:42 smithi122 ceph-mon[32546]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-23T18:22:43.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:42 smithi122 ceph-mon[32546]: osdmap e66: 10 total, 9 up, 10 in 2024-04-23T18:22:43.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:42 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:43.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:42 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:43.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:42 smithi122 ceph-mon[32546]: from='osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333]' entity='osd.9' 2024-04-23T18:22:43.959 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:22:43.959 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:22:43.960 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-23T18:22:43.960 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-23T18:22:43.960 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113096 s, 927 MB/s 2024-04-23T18:22:43.960 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-23T18:22:44.013 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] boot 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: osdmap e67: 10 total, 10 up, 10 in 2024-04-23T18:22:44.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:43 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] boot 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: osdmap e67: 10 total, 10 up, 10 in 2024-04-23T18:22:44.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:43 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: osd.9 [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] boot 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: osdmap e67: 10 total, 10 up, 10 in 2024-04-23T18:22:44.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:43 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:22:44.713 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-04-23T18:22:44.967 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.b/config 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi033 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:45.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi033 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:45.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:45.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: osdmap e68: 10 total, 10 up, 10 in 2024-04-23T18:22:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi033 2024-04-23T18:22:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 4273M 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:45.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:45.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:22:45.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:22:46.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3924260834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc103530 msgr2=0x7f69bc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:46.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3924260834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc103530 0x7f69bc105920 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b0a0 tx=0x7f69ac00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3924260834 shutdown_connections 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3924260834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 0x7f69bc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3924260834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc103530 0x7f69bc105920 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3924260834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc105e60 0x7f69bc108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3924260834 >> 172.21.15.106:0/3924260834 conn(0x7f69bc0fcf40 msgr2=0x7f69bc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3924260834 shutdown_connections 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3924260834 wait complete. 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.641+0000 7f69c1eeb700 1 Processor -- start 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 -- start start 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc103530 0x7f69bc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 0x7f69bc1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10e550 con 0x7f69bc103530 2024-04-23T18:22:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10e3d0 con 0x7f69bc105e60 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69c1eeb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc10e6d0 con 0x7f69bc108790 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41030/0 (socket says 172.21.15.106:41030) 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 -- 172.21.15.106:0/3784370103 learned_addr learned my addr 172.21.15.106:0/3784370103 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69bb7fe700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc103530 0x7f69bc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69bbfff700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 0x7f69bc1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 -- 172.21.15.106:0/3784370103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 msgr2=0x7f69bc1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 0x7f69bc1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 -- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc103530 msgr2=0x7f69bc19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc103530 0x7f69bc19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 -- 172.21.15.106:0/3784370103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc1a2aa0 con 0x7f69bc105e60 2024-04-23T18:22:46.638 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.642+0000 7f69baffd700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c920 tx=0x7f69a400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:46.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.643+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4007b00 con 0x7f69bc105e60 2024-04-23T18:22:46.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.643+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc1a2d90 con 0x7f69bc105e60 2024-04-23T18:22:46.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.643+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69a4004ce0 con 0x7f69bc105e60 2024-04-23T18:22:46.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.643+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a4005580 con 0x7f69bc105e60 2024-04-23T18:22:46.639 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.643+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc1a3270 con 0x7f69bc105e60 2024-04-23T18:22:46.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.644+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc069730 con 0x7f69bc105e60 2024-04-23T18:22:46.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.645+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f69a4004750 con 0x7f69bc105e60 2024-04-23T18:22:46.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.646+0000 7f69b8ff9700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 0x7f69a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:22:46.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.646+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f69a40cfb70 con 0x7f69bc105e60 2024-04-23T18:22:46.640 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.646+0000 7f69bb7fe700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 0x7f69a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:22:46.641 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.646+0000 7f69bb7fe700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 0x7f69a80a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009fd0 tx=0x7f69ac00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:22:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.650+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69a4014030 con 0x7f69bc105e60 2024-04-23T18:22:46.847 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:22:46.854+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f69bc062e80 con 0x7f69a80a3c40 2024-04-23T18:22:47.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:47 smithi122 ceph-mon[32546]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:47.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:47 smithi122 ceph-mon[32546]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:47.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:47 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:47.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:47 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:47.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:47 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:47.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:47 smithi106 ceph-mon[32716]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:47.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:47 smithi106 ceph-mon[32716]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:47.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:47 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:47.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:47 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:47.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:47 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:47 smithi033 ceph-mon[29274]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:47 smithi033 ceph-mon[29274]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:22:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:22:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:22:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:47 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:49.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:49 smithi106 ceph-mon[32716]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:49.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:49 smithi122 ceph-mon[32546]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:49.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:49 smithi033 ceph-mon[29274]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:50.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:50 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/2731288862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:50 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:50 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]': finished 2024-04-23T18:22:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:50 smithi106 ceph-mon[32716]: osdmap e69: 11 total, 10 up, 11 in 2024-04-23T18:22:50.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:50 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:50.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:50 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/2731288862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:50 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:50 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]': finished 2024-04-23T18:22:50.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:50 smithi122 ceph-mon[32546]: osdmap e69: 11 total, 10 up, 11 in 2024-04-23T18:22:50.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:50 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:50.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:50 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/2731288862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:50 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]: dispatch 2024-04-23T18:22:50.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:50 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fab0d97-f3b4-484d-8f60-12b5327a1a5e"}]': finished 2024-04-23T18:22:50.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:50 smithi033 ceph-mon[29274]: osdmap e69: 11 total, 10 up, 11 in 2024-04-23T18:22:50.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:50 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:22:51.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:51 smithi122 ceph-mon[32546]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:51.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:51 smithi122 ceph-mon[32546]: from='client.? 172.21.15.106:0/366906814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:51.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:51 smithi033 ceph-mon[29274]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:51.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:51 smithi033 ceph-mon[29274]: from='client.? 172.21.15.106:0/366906814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:51.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:51 smithi106 ceph-mon[32716]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:51.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:51 smithi106 ceph-mon[32716]: from='client.? 172.21.15.106:0/366906814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:22:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:53 smithi122 ceph-mon[32546]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:53.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:53 smithi033 ceph-mon[29274]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:53.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:53 smithi106 ceph-mon[32716]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:55.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:55 smithi106 ceph-mon[32716]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:55.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:55 smithi122 ceph-mon[32546]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:55.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:55 smithi033 ceph-mon[29274]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:57.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:57 smithi122 ceph-mon[32546]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:57.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:57 smithi033 ceph-mon[29274]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:57.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:57 smithi106 ceph-mon[32716]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:58.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:58 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:58.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:58 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:58 smithi106 ceph-mon[32716]: Deploying daemon osd.10 on smithi106 2024-04-23T18:22:58.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:58 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:58.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:58 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:58 smithi033 ceph-mon[29274]: Deploying daemon osd.10 on smithi106 2024-04-23T18:22:58.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:58 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-23T18:22:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:58 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:22:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:58 smithi122 ceph-mon[32546]: Deploying daemon osd.10 on smithi106 2024-04-23T18:22:59.433 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:22:59 smithi106 ceph-mon[32716]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:59.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:22:59 smithi033 ceph-mon[29274]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:22:59.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:22:59 smithi122 ceph-mon[32546]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:01.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:01 smithi106 ceph-mon[32716]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:01.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:01 smithi033 ceph-mon[29274]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:01.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:01 smithi122 ceph-mon[32546]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:03 smithi106 ceph-mon[32716]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:03.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:03 smithi033 ceph-mon[29274]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:03.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:03 smithi122 ceph-mon[32546]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:05.405 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:05 smithi106 ceph-mon[32716]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:05.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:05 smithi033 ceph-mon[29274]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:05.501 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:05 smithi122 ceph-mon[32546]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:07.175 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.181+0000 7f69b8ff9700 1 -- 172.21.15.106:0/3784370103 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f69bc062e80 con 0x7f69a80a3c40 2024-04-23T18:23:07.175 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 10 on host 'smithi106' 2024-04-23T18:23:07.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 msgr2=0x7f69a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:07.176 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 0x7f69a80a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009fd0 tx=0x7f69ac00b7b0 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 msgr2=0x7f69bc19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c920 tx=0x7f69a400cc30 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 shutdown_connections 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f69a80a3c40 0x7f69a80a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69bc108790 0x7f69bc1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69bc103530 0x7f69bc19ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 --2- 172.21.15.106:0/3784370103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69bc105e60 0x7f69bc19b380 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 >> 172.21.15.106:0/3784370103 conn(0x7f69bc0fcf40 msgr2=0x7f69bc102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 shutdown_connections 2024-04-23T18:23:07.177 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-23T18:23:07.183+0000 7f69c1eeb700 1 -- 172.21.15.106:0/3784370103 wait complete. 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:07.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:07 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:07.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:07.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:07 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:07.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:07.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:07 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:07.808 DEBUG:teuthology.orchestra.run.smithi106:osd.10> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.10.service 2024-04-23T18:23:07.811 INFO:tasks.cephadm:Deploying osd.11 on smithi122 with /dev/vg_nvme/lv_1... 2024-04-23T18:23:07.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-23T18:23:08.056 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:23:08.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-mon[32716]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:23:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-mon[32716]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:23:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-mon[32716]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-mon[32716]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:08.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:08 smithi122 ceph-mon[32546]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:23:08.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:08 smithi122 ceph-mon[32546]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:23:08.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:08 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:08.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:08 smithi122 ceph-mon[32546]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:08.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:08 smithi122 ceph-mon[32546]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:08.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:08 smithi033 ceph-mon[29274]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-23T18:23:08.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:08 smithi033 ceph-mon[29274]: osdmap e70: 11 total, 10 up, 11 in 2024-04-23T18:23:08.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:08 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:08.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:08 smithi033 ceph-mon[29274]: from='osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:08.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:08 smithi033 ceph-mon[29274]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-23T18:23:09.016 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T18:23:08.744+0000 7f04baccb700 -1 osd.10 0 waiting for initial osdmap 2024-04-23T18:23:09.017 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 18:23:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T18:23:08.755+0000 7f04b2ab6700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:09 smithi122 ceph-mon[32546]: from='osd.10 ' entity='osd.10' 2024-04-23T18:23:09.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:09.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:23:09.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:23:09.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:09 smithi106 ceph-mon[32716]: from='osd.10 ' entity='osd.10' 2024-04-23T18:23:09.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:09.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-23T18:23:09.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: osdmap e71: 11 total, 10 up, 11 in 2024-04-23T18:23:09.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:09.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:09 smithi033 ceph-mon[29274]: from='osd.10 ' entity='osd.10' 2024-04-23T18:23:10.298 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-23T18:23:10.299 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-23T18:23:10.299 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-23T18:23:10.299 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-23T18:23:10.299 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110441 s, 949 MB/s 2024-04-23T18:23:10.299 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-23T18:23:10.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] boot 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: osdmap e72: 11 total, 11 up, 11 in 2024-04-23T18:23:10.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:10 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:10.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:23:10.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:23:10.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:10.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] boot 2024-04-23T18:23:10.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: osdmap e72: 11 total, 11 up, 11 in 2024-04-23T18:23:10.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:10 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:10.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:23:10.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:10.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: osd.10 [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] boot 2024-04-23T18:23:10.759 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: osdmap e72: 11 total, 11 up, 11 in 2024-04-23T18:23:10.759 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:10 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:23:11.066 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-04-23T18:23:11.313 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:11.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:11.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:23:11.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:11.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: osdmap e73: 11 total, 11 up, 11 in 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:11.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:12.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:12 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi106 2024-04-23T18:23:12.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:12 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-23T18:23:12.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:12 smithi122 ceph-mon[32546]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:23:12.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:12 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi106 2024-04-23T18:23:12.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:12 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-23T18:23:12.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:12 smithi033 ceph-mon[29274]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:23:12.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:12 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi106 2024-04-23T18:23:12.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:12 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-23T18:23:12.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:12 smithi106 ceph-mon[32716]: osdmap e74: 11 total, 11 up, 11 in 2024-04-23T18:23:13.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 -- 172.21.15.122:0/1386284417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac101990 msgr2=0x7f8eac101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 --2- 172.21.15.122:0/1386284417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac101990 0x7f8eac101db0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009990 tx=0x7f8ea8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 -- 172.21.15.122:0/1386284417 shutdown_connections 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 --2- 172.21.15.122:0/1386284417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac103530 0x7f8eac109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 --2- 172.21.15.122:0/1386284417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac101990 0x7f8eac101db0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 --2- 172.21.15.122:0/1386284417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.026+0000 7f8eb390d700 1 -- 172.21.15.122:0/1386284417 >> 172.21.15.122:0/1386284417 conn(0x7f8eac0fcf30 msgr2=0x7f8eac0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- 172.21.15.122:0/1386284417 shutdown_connections 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- 172.21.15.122:0/1386284417 wait complete. 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 Processor -- start 2024-04-23T18:23:13.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- start start 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac101990 0x7f8eac19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac10d420 con 0x7f8eac101990 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac10d2a0 con 0x7f8eac102b90 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.027+0000 7f8eb390d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8eac10d5a0 con 0x7f8eac103530 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50576/0 (socket says 172.21.15.122:50576) 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 -- 172.21.15.122:0/11135953 learned_addr learned my addr 172.21.15.122:0/11135953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb0ea8700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 -- 172.21.15.122:0/11135953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 msgr2=0x7f8eac19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 -- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac101990 msgr2=0x7f8eac19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb16a9700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac101990 0x7f8eac19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac101990 0x7f8eac19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 -- 172.21.15.122:0/11135953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac1a2a80 con 0x7f8eac103530 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb0ea8700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb1eaa700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007f10 tx=0x7f8ea400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4004bb0 con 0x7f8eac103530 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea4004d50 con 0x7f8eac103530 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eac1a2d70 con 0x7f8eac103530 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4005a30 con 0x7f8eac103530 2024-04-23T18:23:13.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.028+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eac1a3250 con 0x7f8eac103530 2024-04-23T18:23:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.029+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eac069760 con 0x7f8eac103530 2024-04-23T18:23:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.030+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8ea401a020 con 0x7f8eac103530 2024-04-23T18:23:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.031+0000 7f8ea27fc700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 0x7f8e980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.031+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f8ea40cf560 con 0x7f8eac103530 2024-04-23T18:23:13.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.031+0000 7f8eb16a9700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 0x7f8e980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:13.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.031+0000 7f8eb16a9700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 0x7f8e980a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006010 tx=0x7f8ea8003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:13.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.035+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8eac069760 con 0x7f8eac103530 2024-04-23T18:23:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:13.240+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8eac06b900 con 0x7f8e980a3c40 2024-04-23T18:23:13.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:13 smithi122 ceph-mon[32546]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:13 smithi122 ceph-mon[32546]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:23:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:23:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:23:13.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:13 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:13.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:13 smithi033 ceph-mon[29274]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:13.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:13 smithi033 ceph-mon[29274]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:23:13.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:23:13.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:23:13.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:13 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:13.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:13 smithi106 ceph-mon[32716]: pgmap v238: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:13.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:13 smithi106 ceph-mon[32716]: osdmap e75: 11 total, 11 up, 11 in 2024-04-23T18:23:13.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:13 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-23T18:23:13.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:13 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-23T18:23:13.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:13 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:14.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:14 smithi122 ceph-mon[32546]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:14.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:14 smithi122 ceph-mon[32546]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:23:14.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:14 smithi033 ceph-mon[29274]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:14.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:14 smithi033 ceph-mon[29274]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:23:14.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:14 smithi106 ceph-mon[32716]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:14.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:14 smithi106 ceph-mon[32716]: osdmap e76: 11 total, 11 up, 11 in 2024-04-23T18:23:15.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:15 smithi122 ceph-mon[32546]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:15.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:15 smithi033 ceph-mon[29274]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:15.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:15 smithi106 ceph-mon[32716]: pgmap v241: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-23T18:23:16.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:16 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/3018808464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:16 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:16 smithi122 ceph-mon[32546]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]': finished 2024-04-23T18:23:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:16 smithi122 ceph-mon[32546]: osdmap e77: 12 total, 11 up, 12 in 2024-04-23T18:23:16.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:16 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:16.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:16 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/3018808464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:16 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:16 smithi033 ceph-mon[29274]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]': finished 2024-04-23T18:23:16.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:16 smithi033 ceph-mon[29274]: osdmap e77: 12 total, 11 up, 12 in 2024-04-23T18:23:16.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:16 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:16.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:16 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/3018808464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:16 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]: dispatch 2024-04-23T18:23:16.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:16 smithi106 ceph-mon[32716]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "95bc8f26-44c0-429c-893b-566a272d0dbf"}]': finished 2024-04-23T18:23:16.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:16 smithi106 ceph-mon[32716]: osdmap e77: 12 total, 11 up, 12 in 2024-04-23T18:23:16.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:16 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:17 smithi122 ceph-mon[32546]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:23:17.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.122:0/65443124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:23:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:17 smithi033 ceph-mon[29274]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:23:17.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.122:0/65443124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:23:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:17 smithi106 ceph-mon[32716]: pgmap v243: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-23T18:23:17.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.122:0/65443124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-23T18:23:19.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:19 smithi122 ceph-mon[32546]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:19.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:19 smithi033 ceph-mon[29274]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:19.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:19 smithi106 ceph-mon[32716]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:21.568 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:21 smithi122 ceph-mon[32546]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:21.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:21 smithi033 ceph-mon[29274]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:21.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:21 smithi106 ceph-mon[32716]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:23.649 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:23 smithi122 ceph-mon[32546]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:23.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:23 smithi033 ceph-mon[29274]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:23.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:23 smithi106 ceph-mon[32716]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:24.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:23:24.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:24.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:24 smithi122 ceph-mon[32546]: Deploying daemon osd.11 on smithi122 2024-04-23T18:23:24.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:23:24.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:24.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:24 smithi033 ceph-mon[29274]: Deploying daemon osd.11 on smithi122 2024-04-23T18:23:24.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-23T18:23:24.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:24.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:24 smithi106 ceph-mon[32716]: Deploying daemon osd.11 on smithi122 2024-04-23T18:23:25.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:25 smithi122 ceph-mon[32546]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:25 smithi033 ceph-mon[29274]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:25.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:25 smithi106 ceph-mon[32716]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:27.670 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:27 smithi122 ceph-mon[32546]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:27.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:27 smithi033 ceph-mon[29274]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:27.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:27 smithi106 ceph-mon[32716]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:29.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:29 smithi122 ceph-mon[32546]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:29.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:29 smithi033 ceph-mon[29274]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:29.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:29 smithi106 ceph-mon[32716]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:30.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:30 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:30 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:30.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:30.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:30 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:31 smithi122 ceph-mon[32546]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:31 smithi033 ceph-mon[29274]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:31 smithi106 ceph-mon[32716]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:33.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.114+0000 7f8ea27fc700 1 -- 172.21.15.122:0/11135953 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8eac06b900 con 0x7f8e980a3c40 2024-04-23T18:23:33.107 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 11 on host 'smithi122' 2024-04-23T18:23:33.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 msgr2=0x7f8e980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 0x7f8e980a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8006010 tx=0x7f8ea8003860 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 msgr2=0x7f8eac1a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4007f10 tx=0x7f8ea400d3b0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 shutdown_connections 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8e980a3c40 0x7f8e980a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8eac103530 0x7f8eac1a23e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8eac101990 0x7f8eac19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 --2- 172.21.15.122:0/11135953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eac102b90 0x7f8eac19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 >> 172.21.15.122:0/11135953 conn(0x7f8eac0fcf30 msgr2=0x7f8eac06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 shutdown_connections 2024-04-23T18:23:33.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:33.116+0000 7f8eb390d700 1 -- 172.21.15.122:0/11135953 wait complete. 2024-04-23T18:23:33.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:33 smithi122 ceph-mon[32546]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:33 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:33 smithi122 ceph-mon[32546]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:33 smithi122 ceph-mon[32546]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:33.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:33 smithi033 ceph-mon[29274]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:33.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:33 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:33 smithi033 ceph-mon[29274]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:33.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:33 smithi033 ceph-mon[29274]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:33.790 DEBUG:teuthology.orchestra.run.smithi122:osd.11> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.11.service 2024-04-23T18:23:33.792 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-23T18:23:33.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:23:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:33 smithi106 ceph-mon[32716]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:33 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:33 smithi106 ceph-mon[32716]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:33 smithi106 ceph-mon[32716]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-23T18:23:34.053 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:35.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:35.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:23:35.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:35.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:34 smithi033 ceph-mon[29274]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:35.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:23:35.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:35.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:34 smithi122 ceph-mon[32546]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: osdmap e78: 12 total, 11 up, 12 in 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:35.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:34 smithi106 ceph-mon[32716]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-23T18:23:35.790 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T18:23:35.502+0000 7f5329f0c700 -1 osd.11 0 waiting for initial osdmap 2024-04-23T18:23:35.791 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T18:23:35.518+0000 7f5321cf7700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-23T18:23:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.867+0000 7f0dc0525700 1 -- 172.21.15.33:0/3739286527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8103ca0 msgr2=0x7f0db8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.867+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3739286527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8103ca0 0x7f0db8104100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0da80099c0 tx=0x7f0da8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 -- 172.21.15.33:0/3739286527 shutdown_connections 2024-04-23T18:23:35.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3739286527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8104640 0x7f0db810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3739286527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8102aa0 0x7f0db8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3739286527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8103ca0 0x7f0db8104100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 -- 172.21.15.33:0/3739286527 >> 172.21.15.33:0/3739286527 conn(0x7f0db80fe040 msgr2=0x7f0db8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 -- 172.21.15.33:0/3739286527 shutdown_connections 2024-04-23T18:23:35.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 -- 172.21.15.33:0/3739286527 wait complete. 2024-04-23T18:23:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.868+0000 7f0dc0525700 1 Processor -- start 2024-04-23T18:23:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 -- start start 2024-04-23T18:23:35.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8103ca0 0x7f0db81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8104640 0x7f0db81a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0db810e510 con 0x7f0db8102aa0 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0db810e390 con 0x7f0db8103ca0 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dc0525700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0db810e690 con 0x7f0db8104640 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41386/0 (socket says 172.21.15.33:41386) 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 -- 172.21.15.33:0/3750721600 learned_addr learned my addr 172.21.15.33:0/3750721600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbdac0700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8103ca0 0x7f0db81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbeac2700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8104640 0x7f0db81a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 -- 172.21.15.33:0/3750721600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8104640 msgr2=0x7f0db81a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8104640 0x7f0db81a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 -- 172.21.15.33:0/3750721600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8103ca0 msgr2=0x7f0db81990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8103ca0 0x7f0db81990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 -- 172.21.15.33:0/3750721600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0db81a07e0 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.869+0000 7f0dbe2c1700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0db400c9c0 tx=0x7f0db400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.870+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400ce60 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.870+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0db400f450 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.870+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0db81a0ad0 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.870+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4018660 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.870+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0db81a1010 con 0x7f0db8102aa0 2024-04-23T18:23:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.871+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0db8069730 con 0x7f0db8102aa0 2024-04-23T18:23:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.872+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0db400f920 con 0x7f0db8102aa0 2024-04-23T18:23:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.872+0000 7f0daf7fe700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 0x7f0da40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.872+0000 7f0dbdac0700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 0x7f0da40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.873+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7f0db40cfcd0 con 0x7f0db8102aa0 2024-04-23T18:23:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.873+0000 7f0dbdac0700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 0x7f0da40a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009990 tx=0x7f0da8005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:35.877+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0db4014030 con 0x7f0db8102aa0 2024-04-23T18:23:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.085+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0db802e070 con 0x7f0db8102aa0 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:35 smithi033 ceph-mon[29274]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.085+0000 7f0daf7fe700 1 -- 172.21.15.33:0/3750721600 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f0db409cc70 con 0x7f0db8102aa0 2024-04-23T18:23:36.080 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 msgr2=0x7f0da40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 0x7f0da40a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009990 tx=0x7f0da8005c20 comp rx=0 tx=0).stop 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 msgr2=0x7f0db8198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0db400c9c0 tx=0x7f0db400ccd0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 shutdown_connections 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0da40a3c40 0x7f0da40a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0db8104640 0x7f0db81a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0db8102aa0 0x7f0db8198b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 --2- 172.21.15.33:0/3750721600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0db8103ca0 0x7f0db81990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 >> 172.21.15.33:0/3750721600 conn(0x7f0db80fe040 msgr2=0x7f0db8106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 shutdown_connections 2024-04-23T18:23:36.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:36.088+0000 7f0dc0525700 1 -- 172.21.15.33:0/3750721600 wait complete. 2024-04-23T18:23:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:23:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:36.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:35 smithi122 ceph-mon[32546]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: osdmap e79: 12 total, 11 up, 12 in 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:36.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:35 smithi106 ceph-mon[32716]: from='osd.11 ' entity='osd.11' 2024-04-23T18:23:36.835 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1713896589,"num_in_osds":12,"osd_in_since":1713896596,"num_remapped_pgs":0} 2024-04-23T18:23:37.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: purged_snaps scrub starts 2024-04-23T18:23:37.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: purged_snaps scrub ok 2024-04-23T18:23:37.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3750721600' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:37.061 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] boot 2024-04-23T18:23:37.061 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: osdmap e80: 12 total, 12 up, 12 in 2024-04-23T18:23:37.061 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:36 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: purged_snaps scrub starts 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: purged_snaps scrub ok 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3750721600' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] boot 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: osdmap e80: 12 total, 12 up, 12 in 2024-04-23T18:23:37.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:36 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: purged_snaps scrub starts 2024-04-23T18:23:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: purged_snaps scrub ok 2024-04-23T18:23:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3750721600' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:37.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: osd.11 [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] boot 2024-04-23T18:23:37.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: osdmap e80: 12 total, 12 up, 12 in 2024-04-23T18:23:37.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:36 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:23:37.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-23T18:23:38.088 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:37 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:37 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:37 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:39.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: Detected new or changed devices on smithi122 2024-04-23T18:23:39.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-23T18:23:39.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:39.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:38 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: Detected new or changed devices on smithi122 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:38 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:39.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: Detected new or changed devices on smithi122 2024-04-23T18:23:39.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-23T18:23:39.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: osdmap e81: 12 total, 12 up, 12 in 2024-04-23T18:23:39.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:39.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:39.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:38 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3901675869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4103540 msgr2=0x7ff4a4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3901675869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4103540 0x7ff4a4109df0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00b0d0 tx=0x7ff49c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3901675869 shutdown_connections 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3901675869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4103540 0x7ff4a4109df0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3901675869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a41019a0 0x7ff4a4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3901675869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3901675869 >> 172.21.15.33:0/3901675869 conn(0x7ff4a40fcf40 msgr2=0x7ff4a40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3901675869 shutdown_connections 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.760+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3901675869 wait complete. 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 Processor -- start 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 -- start start 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 0x7ff4a4198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a4103540 0x7ff4a41a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d420 con 0x7ff4a4102ba0 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d2a0 con 0x7ff4a4103540 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4ab6cc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410d5a0 con 0x7ff4a41019a0 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4a8c67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4a8c67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43602/0 (socket says 172.21.15.33:43602) 2024-04-23T18:23:39.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4a8c67700 1 -- 172.21.15.33:0/3460987322 learned_addr learned my addr 172.21.15.33:0/3460987322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4a9468700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 0x7ff4a4198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.761+0000 7ff4a9c69700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a4103540 0x7ff4a41a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 -- 172.21.15.33:0/3460987322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 msgr2=0x7ff4a4198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 0x7ff4a4198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 -- 172.21.15.33:0/3460987322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a4103540 msgr2=0x7ff4a41a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a4103540 0x7ff4a41a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 -- 172.21.15.33:0/3460987322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a4103dc0 con 0x7ff4a4102ba0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a9468700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 0x7ff4a4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4a8c67700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d980 tx=0x7ff4a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0016070 con 0x7ff4a4102ba0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a41040b0 con 0x7ff4a4102ba0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4a0010460 con 0x7ff4a4102ba0 2024-04-23T18:23:39.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000f620 con 0x7ff4a4102ba0 2024-04-23T18:23:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.762+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a4104640 con 0x7ff4a4102ba0 2024-04-23T18:23:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.763+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a4069730 con 0x7ff4a4102ba0 2024-04-23T18:23:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.764+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff4a0019070 con 0x7ff4a4102ba0 2024-04-23T18:23:39.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.765+0000 7ff49a7fc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 0x7ff4900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.765+0000 7ff4a9468700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 0x7ff4900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.765+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff4a00cfec0 con 0x7ff4a4102ba0 2024-04-23T18:23:39.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.765+0000 7ff4a9468700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 0x7ff4900a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4102a00 tx=0x7ff494005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:39.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.769+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4a009cd40 con 0x7ff4a4102ba0 2024-04-23T18:23:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.979+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff4a4104ae0 con 0x7ff4a4102ba0 2024-04-23T18:23:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.980+0000 7ff49a7fc700 1 -- 172.21.15.33:0/3460987322 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff4a0098030 con 0x7ff4a4102ba0 2024-04-23T18:23:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 msgr2=0x7ff4900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 0x7ff4900a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4102a00 tx=0x7ff494005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 msgr2=0x7ff4a4199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d980 tx=0x7ff4a000dd40 comp rx=0 tx=0).stop 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 shutdown_connections 2024-04-23T18:23:39.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff4900a3c40 0x7ff4900a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a4103540 0x7ff4a41a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4a41019a0 0x7ff4a4198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 --2- 172.21.15.33:0/3460987322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4a4102ba0 0x7ff4a4199100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 >> 172.21.15.33:0/3460987322 conn(0x7ff4a40fcf40 msgr2=0x7ff4a40fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 shutdown_connections 2024-04-23T18:23:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:39.982+0000 7ff4ab6cc700 1 -- 172.21.15.33:0/3460987322 wait complete. 2024-04-23T18:23:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:39 smithi122 ceph-mon[32546]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:40.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:39 smithi033 ceph-mon[29274]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:40.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:39 smithi106 ceph-mon[32716]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:40.626 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1713896616,"num_in_osds":12,"osd_in_since":1713896596,"num_remapped_pgs":0} 2024-04-23T18:23:40.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:23:40.875 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:40 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3460987322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:40 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3460987322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:41.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:40 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3460987322' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-23T18:23:41.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:41 smithi033 ceph-mon[29274]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:41 smithi122 ceph-mon[32546]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:41 smithi106 ceph-mon[32716]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 -- 172.21.15.33:0/1438228235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 msgr2=0x7f93bc10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1438228235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc10f930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f93b400b0a0 tx=0x7f93b400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 -- 172.21.15.33:0/1438228235 shutdown_connections 2024-04-23T18:23:42.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1438228235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc10f930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1438228235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc108a60 0x7f93bc108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1438228235 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc06b1f0 0x7f93bc06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 -- 172.21.15.33:0/1438228235 >> 172.21.15.33:0/1438228235 conn(0x7f93bc0789e0 msgr2=0x7f93bc078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 -- 172.21.15.33:0/1438228235 shutdown_connections 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.528+0000 7f93c22eb700 1 -- 172.21.15.33:0/1438228235 wait complete. 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.529+0000 7f93c22eb700 1 Processor -- start 2024-04-23T18:23:42.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.529+0000 7f93c22eb700 1 -- start start 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc06b1f0 0x7f93bc199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc108a60 0x7f93bc19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc112ca0 con 0x7f93bc06bbb0 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc112b20 con 0x7f93bc108a60 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc112e20 con 0x7f93bc06b1f0 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43616/0 (socket says 172.21.15.33:43616) 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 -- 172.21.15.33:0/1305523409 learned_addr learned my addr 172.21.15.33:0/1305523409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c0888700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc108a60 0x7f93bc19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bbfff700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc06b1f0 0x7f93bc199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 -- 172.21.15.33:0/1305523409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc06b1f0 msgr2=0x7f93bc199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc06b1f0 0x7f93bc199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 -- 172.21.15.33:0/1305523409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc108a60 msgr2=0x7f93bc19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc108a60 0x7f93bc19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 -- 172.21.15.33:0/1305523409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc19e160 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93bb7fe700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d980 tx=0x7f93b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0017070 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc19e450 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93b0005020 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc1a5cd0 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.530+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0009b50 con 0x7f93bc06bbb0 2024-04-23T18:23:42.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.531+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc069730 con 0x7f93bc06bbb0 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.533+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f93b00051d0 con 0x7f93bc06bbb0 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.533+0000 7f93b97fa700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 0x7f93a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.533+0000 7f93bbfff700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 0x7f93a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:42.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.533+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f93b00d0290 con 0x7f93bc06bbb0 2024-04-23T18:23:42.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.534+0000 7f93bbfff700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 0x7f93a40a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93ac006fd0 tx=0x7f93ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:42.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.537+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93b0020020 con 0x7f93bc06bbb0 2024-04-23T18:23:42.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.744+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93bc02df80 con 0x7f93bc06bbb0 2024-04-23T18:23:42.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.745+0000 7f93b97fa700 1 -- 172.21.15.33:0/1305523409 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18481 (secure 0 0 0) 0x7f93b009d110 con 0x7f93bc06bbb0 2024-04-23T18:23:42.739 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:42.739 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:23:37.902951+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 msgr2=0x7f93a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 0x7f93a40a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f93ac006fd0 tx=0x7f93ac008040 comp rx=0 tx=0).stop 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 msgr2=0x7f93bc199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f93b000d980 tx=0x7f93b000dc90 comp rx=0 tx=0).stop 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 shutdown_connections 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f93a40a3c40 0x7f93a40a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93bc108a60 0x7f93bc19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93bc06b1f0 0x7f93bc199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 --2- 172.21.15.33:0/1305523409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93bc06bbb0 0x7f93bc199690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:42.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 >> 172.21.15.33:0/1305523409 conn(0x7f93bc0789e0 msgr2=0x7f93bc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:42.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 shutdown_connections 2024-04-23T18:23:42.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:42.748+0000 7f93c22eb700 1 -- 172.21.15.33:0/1305523409 wait complete. 2024-04-23T18:23:43.104 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:42 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1305523409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:43.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:42 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1305523409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:43.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:42 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1305523409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:43.420 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:23:43.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-23T18:23:43.669 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:23:44.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:43 smithi033 ceph-mon[29274]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:44.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:43 smithi122 ceph-mon[32546]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:44.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:43 smithi106 ceph-mon[32716]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 -- 172.21.15.33:0/2852290187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8102ac0 msgr2=0x7f5ea8102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 --2- 172.21.15.33:0/2852290187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8102f40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009990 tx=0x7f5e9c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 -- 172.21.15.33:0/2852290187 shutdown_connections 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 --2- 172.21.15.33:0/2852290187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8103480 0x7f5ea810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 --2- 172.21.15.33:0/2852290187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 0x7f5ea8108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 --2- 172.21.15.33:0/2852290187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8102f40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 -- 172.21.15.33:0/2852290187 >> 172.21.15.33:0/2852290187 conn(0x7f5ea8079bb0 msgr2=0x7f5ea8079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 -- 172.21.15.33:0/2852290187 shutdown_connections 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.246+0000 7f5eada12700 1 -- 172.21.15.33:0/2852290187 wait complete. 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 Processor -- start 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 -- start start 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 0x7f5ea819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112c00 con 0x7f5ea8103480 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112a80 con 0x7f5ea8102ac0 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5eada12700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea8112d80 con 0x7f5ea8108a70 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5ea67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.247+0000 7f5ea67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43640/0 (socket says 172.21.15.33:43640) 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 -- 172.21.15.33:0/1020163417 learned_addr learned my addr 172.21.15.33:0/1020163417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea77fe700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 0x7f5ea819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea6ffd700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 -- 172.21.15.33:0/1020163417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 msgr2=0x7f5ea819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 0x7f5ea819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 -- 172.21.15.33:0/1020163417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8102ac0 msgr2=0x7f5ea8199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 -- 172.21.15.33:0/1020163417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea819e060 con 0x7f5ea8103480 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5ea67fc700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0057f0 tx=0x7f5e9c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c027070 con 0x7f5ea8103480 2024-04-23T18:23:45.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c0041c0 con 0x7f5ea8103480 2024-04-23T18:23:45.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea819e2f0 con 0x7f5ea8103480 2024-04-23T18:23:45.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c00f460 con 0x7f5ea8103480 2024-04-23T18:23:45.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.248+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a5bd0 con 0x7f5ea8103480 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.249+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea8069730 con 0x7f5ea8103480 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.250+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e9c02a070 con 0x7f5ea8103480 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.251+0000 7f5eaca10700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 0x7f5e900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.251+0000 7f5ea6ffd700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 0x7f5e900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.251+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5e9c0d0f60 con 0x7f5ea8103480 2024-04-23T18:23:45.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.251+0000 7f5ea6ffd700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 0x7f5e900a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005fd0 tx=0x7f5e9800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:45.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.255+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e9c05eff0 con 0x7f5ea8103480 2024-04-23T18:23:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.463+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ea802df80 con 0x7f5ea8103480 2024-04-23T18:23:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.464+0000 7f5eaca10700 1 -- 172.21.15.33:0/1020163417 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f5e9c013070 con 0x7f5ea8103480 2024-04-23T18:23:45.457 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 msgr2=0x7f5e900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 0x7f5e900a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e98005fd0 tx=0x7f5e9800af20 comp rx=0 tx=0).stop 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 msgr2=0x7f5ea8199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0057f0 tx=0x7f5e9c003730 comp rx=0 tx=0).stop 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 shutdown_connections 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5e900a3c40 0x7f5e900a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ea8108a70 0x7f5ea819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ea8102ac0 0x7f5ea8199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 --2- 172.21.15.33:0/1020163417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ea8103480 0x7f5ea8199590 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 >> 172.21.15.33:0/1020163417 conn(0x7f5ea8079bb0 msgr2=0x7f5ea81016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 shutdown_connections 2024-04-23T18:23:45.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:45.466+0000 7f5eada12700 1 -- 172.21.15.33:0/1020163417 wait complete. 2024-04-23T18:23:46.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:45 smithi033 ceph-mon[29274]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:45 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1020163417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:45 smithi122 ceph-mon[32546]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:45 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1020163417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.b on smithi106 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.e on smithi106 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.h on smithi106 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.k on smithi106 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.c on smithi122 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.f on smithi122 2024-04-23T18:23:46.201 INFO:tasks.cephadm:Adding mds.i on smithi122 2024-04-23T18:23:46.202 INFO:tasks.cephadm:Adding mds.l on smithi122 2024-04-23T18:23:46.202 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l' 2024-04-23T18:23:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:45 smithi106 ceph-mon[32716]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:45 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1020163417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:46.452 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.c/config 2024-04-23T18:23:48.002 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:47 smithi122 ceph-mon[32546]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:48.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 -- 172.21.15.122:0/2224137464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04781019a0 msgr2=0x7f0478101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:48.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 --2- 172.21.15.122:0/2224137464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04781019a0 0x7f0478101dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0468009990 tx=0x7f0468009ca0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 -- 172.21.15.122:0/2224137464 shutdown_connections 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 --2- 172.21.15.122:0/2224137464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0478103540 0x7f0478109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 --2- 172.21.15.122:0/2224137464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04781019a0 0x7f0478101dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 --2- 172.21.15.122:0/2224137464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 0x7f0478103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 -- 172.21.15.122:0/2224137464 >> 172.21.15.122:0/2224137464 conn(0x7f04780fcf40 msgr2=0x7f04780ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 -- 172.21.15.122:0/2224137464 shutdown_connections 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.115+0000 7f047dada700 1 -- 172.21.15.122:0/2224137464 wait complete. 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 Processor -- start 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 -- start start 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04781019a0 0x7f047806b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 0x7f047819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f047810d460 con 0x7f04781019a0 2024-04-23T18:23:48.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f047810d2e0 con 0x7f0478102ba0 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.116+0000 7f047dada700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f047810d5e0 con 0x7f0478103540 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50846/0 (socket says 172.21.15.122:50846) 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f04777fe700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04781019a0 0x7f047806b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 -- 172.21.15.122:0/3297914036 learned_addr learned my addr 172.21.15.122:0/3297914036 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0476ffd700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 0x7f047819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 -- 172.21.15.122:0/3297914036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 msgr2=0x7f047819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 0x7f047819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 -- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04781019a0 msgr2=0x7f047806b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04781019a0 0x7f047806b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 -- 172.21.15.122:0/3297914036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04781a0890 con 0x7f0478103540 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0477fff700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0470007d70 tx=0x7f047000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470016020 con 0x7f0478103540 2024-04-23T18:23:48.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047000ac90 con 0x7f0478103540 2024-04-23T18:23:48.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04781a0b80 con 0x7f0478103540 2024-04-23T18:23:48.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700115e0 con 0x7f0478103540 2024-04-23T18:23:48.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.117+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04781a10c0 con 0x7f0478103540 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.118+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0464005360 con 0x7f0478103540 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.120+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0470019020 con 0x7f0478103540 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.121+0000 7f0474ff9700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 0x7f04600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.121+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f04700d0210 con 0x7f0478103540 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.121+0000 7f04777fe700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 0x7f04600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:48.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.122+0000 7f04777fe700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 0x7f04600a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f046800b2e0 tx=0x7f0468005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:48.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.124+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0470020020 con 0x7f0478103540 2024-04-23T18:23:48.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:47 smithi033 ceph-mon[29274]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:48.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.338+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}) -- 0x7f0464000cf0 con 0x7f04600a3cc0 2024-04-23T18:23:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.346+0000 7f0474ff9700 1 -- 172.21.15.122:0/3297914036 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0464000cf0 con 0x7f04600a3cc0 2024-04-23T18:23:48.341 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mds.all update... 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 msgr2=0x7f04600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 0x7f04600a6180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f046800b2e0 tx=0x7f0468005fd0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 msgr2=0x7f047819ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0470007d70 tx=0x7f047000a340 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 shutdown_connections 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04600a3cc0 0x7f04600a6180 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0478103540 0x7f047819ffc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04781019a0 0x7f047806b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 --2- 172.21.15.122:0/3297914036 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0478102ba0 0x7f047819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 >> 172.21.15.122:0/3297914036 conn(0x7f04780fcf40 msgr2=0x7f0478105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 shutdown_connections 2024-04-23T18:23:48.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-23T18:23:48.349+0000 7f047dada700 1 -- 172.21.15.122:0/3297914036 wait complete. 2024-04-23T18:23:48.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:47 smithi106 ceph-mon[32716]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:49.009 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.a.service 2024-04-23T18:23:49.012 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service 2024-04-23T18:23:49.055 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.g.service 2024-04-23T18:23:49.098 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service 2024-04-23T18:23:49.141 DEBUG:teuthology.orchestra.run.smithi106:mds.b> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.b.service 2024-04-23T18:23:49.144 DEBUG:teuthology.orchestra.run.smithi106:mds.e> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.e.service 2024-04-23T18:23:49.187 DEBUG:teuthology.orchestra.run.smithi106:mds.h> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.h.service 2024-04-23T18:23:49.230 DEBUG:teuthology.orchestra.run.smithi106:mds.k> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.k.service 2024-04-23T18:23:49.274 DEBUG:teuthology.orchestra.run.smithi122:mds.c> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.c.service 2024-04-23T18:23:49.277 DEBUG:teuthology.orchestra.run.smithi122:mds.f> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service 2024-04-23T18:23:49.321 DEBUG:teuthology.orchestra.run.smithi122:mds.i> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.i.service 2024-04-23T18:23:49.363 DEBUG:teuthology.orchestra.run.smithi122:mds.l> sudo journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.l.service 2024-04-23T18:23:49.407 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-23T18:23:49.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:23:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 -- 172.21.15.33:0/1149415611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21041029a0 msgr2=0x7f21041092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 --2- 172.21.15.33:0/1149415611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21041029a0 0x7f21041092c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009ca0 tx=0x7f20fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 -- 172.21.15.33:0/1149415611 shutdown_connections 2024-04-23T18:23:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 --2- 172.21.15.33:0/1149415611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21041029a0 0x7f21041092c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 --2- 172.21.15.33:0/1149415611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f2104102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 --2- 172.21.15.33:0/1149415611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2104100e90 0x7f21041012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 -- 172.21.15.33:0/1149415611 >> 172.21.15.33:0/1149415611 conn(0x7f21040fc400 msgr2=0x7f21040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 -- 172.21.15.33:0/1149415611 shutdown_connections 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.534+0000 7f210b15d640 1 -- 172.21.15.33:0/1149415611 wait complete. 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 Processor -- start 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 -- start start 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f21041986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 0x7f210419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f210410c9d0 con 0x7f2104100e90 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f210410c850 con 0x7f21041029a0 2024-04-23T18:23:49.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f210b15d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f210410cb50 con 0x7f2104101fe0 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58922/0 (socket says 172.21.15.33:58922) 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 -- 172.21.15.33:0/2389266336 learned_addr learned my addr 172.21.15.33:0/2389266336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2103fff640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f21041986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f21096fa640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 0x7f210419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 -- 172.21.15.33:0/2389266336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 msgr2=0x7f21041986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f21041986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 -- 172.21.15.33:0/2389266336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 msgr2=0x7f210419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 0x7f210419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.535+0000 7f2108ef9640 1 -- 172.21.15.33:0/2389266336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210419fd80 con 0x7f2104100e90 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f2103fff640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f21041986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:49.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f21096fa640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 0x7f210419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f2108ef9640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8017030 con 0x7f2104100e90 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041a0070 con 0x7f2104100e90 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20f8004020 con 0x7f2104100e90 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80047a0 con 0x7f2104100e90 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.536+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a05c0 con 0x7f2104100e90 2024-04-23T18:23:49.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.537+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21041a0880 con 0x7f2104100e90 2024-04-23T18:23:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.538+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20f8009a30 con 0x7f2104100e90 2024-04-23T18:23:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.539+0000 7f2101ffb640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 0x7f20f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.539+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f20f80cfb00 con 0x7f2104100e90 2024-04-23T18:23:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.539+0000 7f2103fff640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 0x7f20f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.539+0000 7f2103fff640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 0x7f20f40a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2104199670 tx=0x7f20f0005cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.543+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20f809c980 con 0x7f2104100e90 2024-04-23T18:23:49.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:49.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.579 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:49 smithi122 ceph-mon[32546]: Deploying daemon mds.l on smithi122 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:49.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:49.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:49 smithi106 ceph-mon[32716]: Deploying daemon mds.l on smithi122 2024-04-23T18:23:49.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.732+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21041a0af0 con 0x7f2104100e90 2024-04-23T18:23:49.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.733+0000 7f2101ffb640 1 -- 172.21.15.33:0/2389266336 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18481 (secure 0 0 0) 0x7f20f809c280 con 0x7f2104100e90 2024-04-23T18:23:49.726 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:49.726 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:23:37.902951+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 msgr2=0x7f20f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 0x7f20f40a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2104199670 tx=0x7f20f0005cb0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 msgr2=0x7f2104198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).stop 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 shutdown_connections 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20f40a3bb0 0x7f20f40a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21041029a0 0x7f210419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2104101fe0 0x7f21041986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 --2- 172.21.15.33:0/2389266336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2104100e90 0x7f2104198160 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 >> 172.21.15.33:0/2389266336 conn(0x7f21040fc400 msgr2=0x7f21040fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 shutdown_connections 2024-04-23T18:23:49.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.735+0000 7f210b15d640 1 -- 172.21.15.33:0/2389266336 wait complete. 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:23:49.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:49.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:49 smithi033 ceph-mon[29274]: Deploying daemon mds.l on smithi122 2024-04-23T18:23:49.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:23:49.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:23:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 -- 172.21.15.33:0/805207413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38102a00 msgr2=0x7f0e38109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/805207413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38102a00 0x7f0e38109320 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009ca0 tx=0x7f0e3002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 -- 172.21.15.33:0/805207413 shutdown_connections 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/805207413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38102a00 0x7f0e38109320 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/805207413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102040 0x7f0e381024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/805207413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38100e60 0x7f0e38101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 -- 172.21.15.33:0/805207413 >> 172.21.15.33:0/805207413 conn(0x7f0e380fc440 msgr2=0x7f0e380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 -- 172.21.15.33:0/805207413 shutdown_connections 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.854+0000 7f0e3ce60640 1 -- 172.21.15.33:0/805207413 wait complete. 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 Processor -- start 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 -- start start 2024-04-23T18:23:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 0x7f0e38198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102a00 0x7f0e3819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810db90 con 0x7f0e38100e60 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810da10 con 0x7f0e38102040 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3ce60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3810dd10 con 0x7f0e38102a00 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58934/0 (socket says 172.21.15.33:58934) 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3659c640 1 -- 172.21.15.33:0/2528964719 learned_addr learned my addr 172.21.15.33:0/2528964719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e35d9b640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 0x7f0e38198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e36d9d640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102a00 0x7f0e3819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3659c640 1 -- 172.21.15.33:0/2528964719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102a00 msgr2=0x7f0e3819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.855+0000 7f0e3659c640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102a00 0x7f0e3819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3659c640 1 -- 172.21.15.33:0/2528964719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 msgr2=0x7f0e38198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3659c640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 0x7f0e38198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:49.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3659c640 1 -- 172.21.15.33:0/2528964719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e3819fd00 con 0x7f0e38100e60 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e35d9b640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 0x7f0e38198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3659c640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d770 tx=0x7f0e2800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017070 con 0x7f0e38100e60 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e28009c30 con 0x7f0e38100e60 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3819fff0 con 0x7f0e38100e60 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003cc0 con 0x7f0e38100e60 2024-04-23T18:23:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.856+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e381a0450 con 0x7f0e38100e60 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.857+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e380691c0 con 0x7f0e38100e60 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.858+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0e28003e60 con 0x7f0e38100e60 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.859+0000 7f0e277fe640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 0x7f0e200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.859+0000 7f0e35d9b640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 0x7f0e200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.859+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0e280cfea0 con 0x7f0e38100e60 2024-04-23T18:23:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.859+0000 7f0e35d9b640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 0x7f0e200a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e381995f0 tx=0x7f0e2c00a5c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:49.862+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e28009de0 con 0x7f0e38100e60 2024-04-23T18:23:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.049+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e3802d5c0 con 0x7f0e38100e60 2024-04-23T18:23:50.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.050+0000 7f0e277fe640 1 -- 172.21.15.33:0/2528964719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f0e2809cc80 con 0x7f0e38100e60 2024-04-23T18:23:50.043 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:23:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 msgr2=0x7f0e200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 0x7f0e200a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e381995f0 tx=0x7f0e2c00a5c0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 msgr2=0x7f0e381980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800d770 tx=0x7f0e2800dc30 comp rx=0 tx=0).stop 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 shutdown_connections 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0e200a3bb0 0x7f0e200a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e38102a00 0x7f0e3819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e38102040 0x7f0e38198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 --2- 172.21.15.33:0/2528964719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e38100e60 0x7f0e381980e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 >> 172.21.15.33:0/2528964719 conn(0x7f0e380fc440 msgr2=0x7f0e38105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 shutdown_connections 2024-04-23T18:23:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.052+0000 7f0e3ce60640 1 -- 172.21.15.33:0/2528964719 wait complete. 2024-04-23T18:23:50.056 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1981310506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 msgr2=0x7efe98101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1981310506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98101280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efe88009900 tx=0x7efe8802a410 comp rx=0 tx=0).stop 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1981310506 shutdown_connections 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1981310506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102a00 0x7efe98109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1981310506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102040 0x7efe981024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1981310506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98101280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1981310506 >> 172.21.15.33:0/1981310506 conn(0x7efe980fc440 msgr2=0x7efe980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.165+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1981310506 shutdown_connections 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1981310506 wait complete. 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 Processor -- start 2024-04-23T18:23:50.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 -- start start 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 0x7efe98198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102a00 0x7efe9819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe9810db60 con 0x7efe98100e60 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe9810d9e0 con 0x7efe98102040 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9c9cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe9810dce0 con 0x7efe98102a00 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.166+0000 7efe9659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58948/0 (socket says 172.21.15.33:58948) 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 -- 172.21.15.33:0/1451827378 learned_addr learned my addr 172.21.15.33:0/1451827378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe96d9d640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102a00 0x7efe9819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe95d9b640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 0x7efe98198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 -- 172.21.15.33:0/1451827378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102a00 msgr2=0x7efe9819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102a00 0x7efe9819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 -- 172.21.15.33:0/1451827378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 msgr2=0x7efe98198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 0x7efe98198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 -- 172.21.15.33:0/1451827378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9819fe30 con 0x7efe98100e60 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe95d9b640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 0x7efe98198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9659c640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe88031500 tx=0x7efe88003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8803f070 con 0x7efe98100e60 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a00c0 con 0x7efe98100e60 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe88031c50 con 0x7efe98100e60 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88035030 con 0x7efe98100e60 2024-04-23T18:23:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.167+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a05b0 con 0x7efe98100e60 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.168+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe98108250 con 0x7efe98100e60 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.169+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efe88042070 con 0x7efe98100e60 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.170+0000 7efe877fe640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 0x7efe800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.170+0000 7efe95d9b640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 0x7efe800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.170+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efe880f07b0 con 0x7efe98100e60 2024-04-23T18:23:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.170+0000 7efe95d9b640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 0x7efe800a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe98199720 tx=0x7efe8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:50.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.174+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe880bd590 con 0x7efe98100e60 2024-04-23T18:23:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.402+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe9802d5c0 con 0x7efe98100e60 2024-04-23T18:23:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.403+0000 7efe877fe640 1 -- 172.21.15.33:0/1451827378 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7efe880bce90 con 0x7efe98100e60 2024-04-23T18:23:50.397 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:50.397 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-23T18:23:50.397 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 msgr2=0x7efe800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 0x7efe800a6070 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efe98199720 tx=0x7efe8c005f90 comp rx=0 tx=0).stop 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 msgr2=0x7efe98198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe88031500 tx=0x7efe88003c10 comp rx=0 tx=0).stop 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 shutdown_connections 2024-04-23T18:23:50.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe800a3bb0 0x7efe800a6070 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe98102a00 0x7efe9819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe98102040 0x7efe98198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 --2- 172.21.15.33:0/1451827378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe98100e60 0x7efe98198210 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 >> 172.21.15.33:0/1451827378 conn(0x7efe980fc440 msgr2=0x7efe98105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 shutdown_connections 2024-04-23T18:23:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:50.405+0000 7efe9c9cd640 1 -- 172.21.15.33:0/1451827378 wait complete. 2024-04-23T18:23:50.527 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:50 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2389266336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:50.528 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:50 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2528964719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:50.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:50 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2389266336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:50.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:50 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2528964719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:50.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:50 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2389266336' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:23:50.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:50 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2528964719' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:23:50.919 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:50 smithi122 systemd[1]: Starting Ceph mds.l for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:51.199 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:50 smithi122 podman[62632]: 2024-04-23 18:23:50.829263381 +0000 UTC m=+0.018299496 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:51.199 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:50 smithi122 podman[62632]: 2024-04-23 18:23:50.944061792 +0000 UTC m=+0.133097927 container create 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-23T18:23:51.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:51 smithi106 ceph-mon[32716]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:51.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:51 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1451827378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:51 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6832/2194426420,v1:172.21.15.122:6833/2194426420] up:boot 2024-04-23T18:23:51.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:51 smithi106 ceph-mon[32716]: fsmap 1 up:standby 2024-04-23T18:23:51.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:51 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:51.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mon[32546]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1451827378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6832/2194426420,v1:172.21.15.122:6833/2194426420] up:boot 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mon[32546]: fsmap 1 up:standby 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 podman[62632]: 2024-04-23 18:23:51.247779065 +0000 UTC m=+0.436815158 container init 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 podman[62632]: 2024-04-23 18:23:51.250247017 +0000 UTC m=+0.439283112 container start 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-mds[62651]: -- 172.21.15.122:0/314024252 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d91713a40 con 0x555d916e7000 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: starting mds.l at 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 bash[62632]: 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a 2024-04-23T18:23:51.699 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 18:23:51 smithi122 systemd[1]: Started Ceph mds.l for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:51.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:51 smithi033 ceph-mon[29274]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:51.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:51 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1451827378' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:51.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:51 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6832/2194426420,v1:172.21.15.122:6833/2194426420] up:boot 2024-04-23T18:23:51.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:51 smithi033 ceph-mon[29274]: fsmap 1 up:standby 2024-04-23T18:23:51.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:51 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:23:52.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:52.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 -- 172.21.15.33:0/3081316198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0105360 msgr2=0x7fe4a0107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/3081316198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0105360 0x7fe4a0107750 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe494009900 tx=0x7fe49402a410 comp rx=0 tx=0).stop 2024-04-23T18:23:52.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 -- 172.21.15.33:0/3081316198 shutdown_connections 2024-04-23T18:23:52.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/3081316198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0107c90 0x7fe4a010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/3081316198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0105360 0x7fe4a0107750 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/3081316198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0102a30 0x7fe4a0104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.533+0000 7fe4a5e05640 1 -- 172.21.15.33:0/3081316198 >> 172.21.15.33:0/3081316198 conn(0x7fe4a00fc440 msgr2=0x7fe4a00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- 172.21.15.33:0/3081316198 shutdown_connections 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- 172.21.15.33:0/3081316198 wait complete. 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 Processor -- start 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- start start 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 0x7fe4a0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0107c90 0x7fe4a019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a010daf0 con 0x7fe4a0102a30 2024-04-23T18:23:52.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a010d970 con 0x7fe4a0107c90 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.534+0000 7fe4a5e05640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a010dc70 con 0x7fe4a0105360 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58966/0 (socket says 172.21.15.33:58966) 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 -- 172.21.15.33:0/2139087606 learned_addr learned my addr 172.21.15.33:0/2139087606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49effd640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 0x7fe4a0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49ffff640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0107c90 0x7fe4a019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 -- 172.21.15.33:0/2139087606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 msgr2=0x7fe4a0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 0x7fe4a0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 -- 172.21.15.33:0/2139087606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0107c90 msgr2=0x7fe4a019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0107c90 0x7fe4a019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 -- 172.21.15.33:0/2139087606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a019fe40 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49effd640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 0x7fe4a0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49f7fe640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d950 tx=0x7fe49000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490017070 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe490009c30 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a01a0130 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000f620 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.535+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a01a0650 con 0x7fe4a0102a30 2024-04-23T18:23:52.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.536+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a00fdf30 con 0x7fe4a0102a30 2024-04-23T18:23:52.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.538+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe490010460 con 0x7fe4a0102a30 2024-04-23T18:23:52.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.538+0000 7fe49cff9640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 0x7fe4880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:52.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.538+0000 7fe49effd640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 0x7fe4880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:52.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.538+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe4900cfce0 con 0x7fe4a0102a30 2024-04-23T18:23:52.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.539+0000 7fe49effd640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 0x7fe4880a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe4940314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:52.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.543+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe49009cac0 con 0x7fe4a0102a30 2024-04-23T18:23:52.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:52.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:52 smithi122 ceph-mon[32546]: Deploying daemon mds.k on smithi106 2024-04-23T18:23:52.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:52.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:52.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:52.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:52 smithi033 ceph-mon[29274]: Deploying daemon mds.k on smithi106 2024-04-23T18:23:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.770+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4a002d5c0 con 0x7fe4a0102a30 2024-04-23T18:23:52.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.770+0000 7fe49cff9640 1 -- 172.21.15.33:0/2139087606 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fe490009de0 con 0x7fe4a0102a30 2024-04-23T18:23:52.764 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:52.764 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-23T18:23:52.764 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 msgr2=0x7fe4880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 0x7fe4880a6070 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe494000c00 tx=0x7fe4940314a0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 msgr2=0x7fe4a0198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe49000d950 tx=0x7fe49000de10 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 shutdown_connections 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe4880a3bb0 0x7fe4880a6070 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4a0107c90 0x7fe4a019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a0105360 0x7fe4a0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 --2- 172.21.15.33:0/2139087606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4a0102a30 0x7fe4a0198220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 >> 172.21.15.33:0/2139087606 conn(0x7fe4a00fc440 msgr2=0x7fe4a0103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:52.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 shutdown_connections 2024-04-23T18:23:52.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:52.773+0000 7fe4a5e05640 1 -- 172.21.15.33:0/2139087606 wait complete. 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:52.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:52 smithi106 ceph-mon[32716]: Deploying daemon mds.k on smithi106 2024-04-23T18:23:53.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:53 smithi122 ceph-mon[32546]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:53.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2139087606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:53.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:53 smithi033 ceph-mon[29274]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:53.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2139087606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:53.868 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:53 smithi106 ceph-mon[32716]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:53.868 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2139087606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:53.868 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:53 smithi106 systemd[1]: Starting Ceph mds.k for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:54.120 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:53 smithi106 podman[62021]: 2024-04-23 18:23:53.778224571 +0000 UTC m=+0.017554081 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:54.121 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:53 smithi106 podman[62021]: 2024-04-23 18:23:53.894137361 +0000 UTC m=+0.133466940 container create 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 podman[62021]: 2024-04-23 18:23:54.172205068 +0000 UTC m=+0.411534579 container init 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 podman[62021]: 2024-04-23 18:23:54.174889773 +0000 UTC m=+0.414219293 container start 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 ceph-mds[62040]: -- 172.21.15.106:0/2423352679 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558708dda40 con 0x5558708b1000 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: starting mds.k at 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 bash[62021]: 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 2024-04-23T18:23:54.621 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 18:23:54 smithi106 systemd[1]: Started Ceph mds.k for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:54.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:54.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 -- 172.21.15.33:0/2864630538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798075f10 msgr2=0x7f8798074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 --2- 172.21.15.33:0/2864630538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798075f10 0x7f8798074360 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8788009900 tx=0x7f878802a480 comp rx=0 tx=0).stop 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 -- 172.21.15.33:0/2864630538 shutdown_connections 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 --2- 172.21.15.33:0/2864630538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f879810c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 --2- 172.21.15.33:0/2864630538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798074930 0x7f8798074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 --2- 172.21.15.33:0/2864630538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798075f10 0x7f8798074360 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 -- 172.21.15.33:0/2864630538 >> 172.21.15.33:0/2864630538 conn(0x7f87980fd530 msgr2=0x7f87980ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 -- 172.21.15.33:0/2864630538 shutdown_connections 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.892+0000 7f879e87c640 1 -- 172.21.15.33:0/2864630538 wait complete. 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 Processor -- start 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 -- start start 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798075f10 0x7f879819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f87981a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f879810fce0 con 0x7f8798074930 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f879810fb60 con 0x7f8798109e70 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f879e87c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f879810fe60 con 0x7f8798075f10 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f8797fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f8797fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58982/0 (socket says 172.21.15.33:58982) 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f8797fff640 1 -- 172.21.15.33:0/694845896 learned_addr learned my addr 172.21.15.33:0/694845896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.893+0000 7f87977fe640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798075f10 0x7f879819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f879ce19640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f87981a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 -- 172.21.15.33:0/694845896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798075f10 msgr2=0x7f879819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798075f10 0x7f879819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 -- 172.21.15.33:0/694845896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 msgr2=0x7f87981a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f87981a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 -- 172.21.15.33:0/694845896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87981a1fa0 con 0x7f8798074930 2024-04-23T18:23:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f879ce19640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f87981a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f8797fff640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f8788004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878803f030 con 0x7f8798074930 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8788003cb0 con 0x7f8798074930 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788035070 con 0x7f8798074930 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87981a2230 con 0x7f8798074930 2024-04-23T18:23:54.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.894+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87981a26d0 con 0x7f8798074930 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.895+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8798108c30 con 0x7f8798074930 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.896+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8788042030 con 0x7f8798074930 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.897+0000 7f87957fa640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 0x7f87800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.897+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f87880f0a00 con 0x7f8798074930 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.897+0000 7f87977fe640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 0x7f87800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:54.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.898+0000 7f87977fe640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 0x7f87800a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f878c005e10 tx=0x7f878c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:54.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:54.902+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8788003e60 con 0x7f8798074930 2024-04-23T18:23:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.141+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f879802d800 con 0x7f8798074930 2024-04-23T18:23:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.141+0000 7f87957fa640 1 -- 172.21.15.33:0/694845896 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f87880bd7e0 con 0x7f8798074930 2024-04-23T18:23:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:55.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.143+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 msgr2=0x7f87800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.143+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 0x7f87800a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f878c005e10 tx=0x7f878c005da0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 msgr2=0x7f879819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8788005940 tx=0x7f8788004430 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 shutdown_connections 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f87800a3bb0 0x7f87800a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8798109e70 0x7f87981a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8798075f10 0x7f879819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 --2- 172.21.15.33:0/694845896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8798074930 0x7f879819a380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 >> 172.21.15.33:0/694845896 conn(0x7f87980fd530 msgr2=0x7f8798105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 shutdown_connections 2024-04-23T18:23:55.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:55.144+0000 7f879e87c640 1 -- 172.21.15.33:0/694845896 wait complete. 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: Deploying daemon mds.j on smithi033 2024-04-23T18:23:55.402 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6832/637332963,v1:172.21.15.106:6833/637332963] up:boot 2024-04-23T18:23:55.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: fsmap 2 up:standby 2024-04-23T18:23:55.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:55.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:55 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/694845896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:55.608 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: Deploying daemon mds.j on smithi033 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6832/637332963,v1:172.21.15.106:6833/637332963] up:boot 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: fsmap 2 up:standby 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:55.609 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:55 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/694845896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: Deploying daemon mds.j on smithi033 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6832/637332963,v1:172.21.15.106:6833/637332963] up:boot 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: fsmap 2 up:standby 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:23:55.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:55 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/694845896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:56.731 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:56 smithi033 systemd[1]: Starting Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:57.055 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:56 smithi033 podman[69512]: 2024-04-23 18:23:56.693542149 +0000 UTC m=+0.018236269 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:57.055 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:56 smithi033 podman[69512]: 2024-04-23 18:23:56.800106642 +0000 UTC m=+0.124800747 container create 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T18:23:57.055 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 podman[69512]: 2024-04-23 18:23:57.036350362 +0000 UTC m=+0.361044470 container init 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:23:57.055 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 podman[69512]: 2024-04-23 18:23:57.039054242 +0000 UTC m=+0.363748350 container start 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T18:23:57.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- 172.21.15.33:0/298869122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 msgr2=0x7f2c5c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/298869122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c103610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- 172.21.15.33:0/298869122 shutdown_connections 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/298869122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 0x7f2c5c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/298869122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c103610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/298869122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 0x7f2c5c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- 172.21.15.33:0/298869122 >> 172.21.15.33:0/298869122 conn(0x7f2c5c0fd530 msgr2=0x7f2c5c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- 172.21.15.33:0/298869122 shutdown_connections 2024-04-23T18:23:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- 172.21.15.33:0/298869122 wait complete. 2024-04-23T18:23:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 Processor -- start 2024-04-23T18:23:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.272+0000 7f2c63b9b640 1 -- start start 2024-04-23T18:23:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 0x7f2c5c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 0x7f2c5c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10db30 con 0x7f2c5c103190 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10d9b0 con 0x7f2c5c103b50 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c63b9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5c10dcb0 con 0x7f2c5c101fa0 2024-04-23T18:23:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47582/0 (socket says 172.21.15.33:47582) 2024-04-23T18:23:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 -- 172.21.15.33:0/1528463002 learned_addr learned my addr 172.21.15.33:0/1528463002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c62138640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 0x7f2c5c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61937640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 0x7f2c5c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 -- 172.21.15.33:0/1528463002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 msgr2=0x7f2c5c1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 0x7f2c5c1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 -- 172.21.15.33:0/1528463002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 msgr2=0x7f2c5c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 0x7f2c5c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 -- 172.21.15.33:0/1528463002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5c19fe00 con 0x7f2c5c103190 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.273+0000 7f2c61136640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005ab0 tx=0x7f2c4c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.274+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c03f070 con 0x7f2c5c103190 2024-04-23T18:23:57.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.274+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c5c1a0090 con 0x7f2c5c103190 2024-04-23T18:23:57.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.274+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c4c031c50 con 0x7f2c5c103190 2024-04-23T18:23:57.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.274+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c035030 con 0x7f2c5c103190 2024-04-23T18:23:57.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.274+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5c1a0580 con 0x7f2c5c103190 2024-04-23T18:23:57.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.275+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5c109340 con 0x7f2c5c103190 2024-04-23T18:23:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.276+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2c4c042070 con 0x7f2c5c103190 2024-04-23T18:23:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.277+0000 7f2c52ffd640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 0x7f2c480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.277+0000 7f2c61937640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 0x7f2c480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.277+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f2c4c0f0de0 con 0x7f2c5c103190 2024-04-23T18:23:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.277+0000 7f2c61937640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 0x7f2c480a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c580058c0 tx=0x7f2c5800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.280+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c4c0bd570 con 0x7f2c5c103190 2024-04-23T18:23:57.343 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mds[69531]: -- 172.21.15.33:0/3874262595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555635af5a40 con 0x555635ac8c00 2024-04-23T18:23:57.343 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: starting mds.j at 2024-04-23T18:23:57.343 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 bash[69512]: 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e 2024-04-23T18:23:57.343 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:23:57 smithi033 systemd[1]: Started Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:23:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.512+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c5c02d780 con 0x7f2c5c103190 2024-04-23T18:23:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.512+0000 7f2c52ffd640 1 -- 172.21.15.33:0/1528463002 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f2c4c0c1870 con 0x7f2c5c103190 2024-04-23T18:23:57.506 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:57.506 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:57.506 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-04-23T18:23:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 msgr2=0x7f2c480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 0x7f2c480a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2c580058c0 tx=0x7f2c5800a2d0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 msgr2=0x7f2c5c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005ab0 tx=0x7f2c4c003b90 comp rx=0 tx=0).stop 2024-04-23T18:23:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 shutdown_connections 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2c480a3bb0 0x7f2c480a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c103b50 0x7f2c5c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c5c103190 0x7f2c5c198720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 --2- 172.21.15.33:0/1528463002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c5c101fa0 0x7f2c5c1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 >> 172.21.15.33:0/1528463002 conn(0x7f2c5c0fd530 msgr2=0x7f2c5c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 shutdown_connections 2024-04-23T18:23:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:57.514+0000 7f2c63b9b640 1 -- 172.21.15.33:0/1528463002 wait complete. 2024-04-23T18:23:57.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:57.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:57 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:57.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:57.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:57 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:57.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:57.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:23:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:23:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:23:57.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:57 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:23:58.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:58 smithi033 ceph-mon[29274]: Deploying daemon mds.d on smithi033 2024-04-23T18:23:58.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:58 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6834/2258040370,v1:172.21.15.33:6835/2258040370] up:boot 2024-04-23T18:23:58.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:58 smithi033 ceph-mon[29274]: fsmap 3 up:standby 2024-04-23T18:23:58.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:58 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:58.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:58 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1528463002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:58.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:58 smithi106 ceph-mon[32716]: Deploying daemon mds.d on smithi033 2024-04-23T18:23:58.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:58 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6834/2258040370,v1:172.21.15.33:6835/2258040370] up:boot 2024-04-23T18:23:58.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:58 smithi106 ceph-mon[32716]: fsmap 3 up:standby 2024-04-23T18:23:58.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:58 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:58.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:58 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1528463002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:58.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:58 smithi122 ceph-mon[32546]: Deploying daemon mds.d on smithi033 2024-04-23T18:23:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:58 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6834/2258040370,v1:172.21.15.33:6835/2258040370] up:boot 2024-04-23T18:23:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:58 smithi122 ceph-mon[32546]: fsmap 3 up:standby 2024-04-23T18:23:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:58 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:23:58.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:58 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1528463002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:23:59.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:23:59 smithi033 ceph-mon[29274]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:59.481 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:23:59.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:23:59.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:23:59 smithi106 ceph-mon[32716]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 -- 172.21.15.33:0/1132912914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c103b50 msgr2=0x7f187c10a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1132912914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c103b50 0x7f187c10a3e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1874009900 tx=0x7f187402a410 comp rx=0 tx=0).stop 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 -- 172.21.15.33:0/1132912914 shutdown_connections 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1132912914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c103b50 0x7f187c10a3e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1132912914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103190 0x7f187c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1132912914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c101fa0 0x7f187c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 -- 172.21.15.33:0/1132912914 >> 172.21.15.33:0/1132912914 conn(0x7f187c0fd530 msgr2=0x7f187c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 -- 172.21.15.33:0/1132912914 shutdown_connections 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.633+0000 7f1883ea7640 1 -- 172.21.15.33:0/1132912914 wait complete. 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 Processor -- start 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 -- start start 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 0x7f187c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 0x7f187c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10db20 con 0x7f187c101fa0 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10d9a0 con 0x7f187c103190 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1883ea7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f187c10dca0 con 0x7f187c103b50 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47600/0 (socket says 172.21.15.33:47600) 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 -- 172.21.15.33:0/1768258355 learned_addr learned my addr 172.21.15.33:0/1768258355 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881442640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 0x7f187c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1882444640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 0x7f187c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 -- 172.21.15.33:0/1768258355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 msgr2=0x7f187c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 0x7f187c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 -- 172.21.15.33:0/1768258355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 msgr2=0x7f187c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 0x7f187c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881c43640 1 -- 172.21.15.33:0/1768258355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c19fde0 con 0x7f187c101fa0 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1881442640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 0x7f187c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.634+0000 7f1882444640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 0x7f187c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1881c43640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f187800ea80 tx=0x7f187800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187800ce40 con 0x7f187c101fa0 2024-04-23T18:23:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1a00d0 con 0x7f187c101fa0 2024-04-23T18:23:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1878011920 con 0x7f187c101fa0 2024-04-23T18:23:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187801aaa0 con 0x7f187c101fa0 2024-04-23T18:23:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.635+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c1a06a0 con 0x7f187c101fa0 2024-04-23T18:23:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.636+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187c1092c0 con 0x7f187c101fa0 2024-04-23T18:23:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.637+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f187801ac40 con 0x7f187c101fa0 2024-04-23T18:23:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.637+0000 7f1872ffd640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 0x7f18680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:23:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.637+0000 7f1881442640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 0x7f18680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:23:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.637+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f18780d05d0 con 0x7f187c101fa0 2024-04-23T18:23:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.638+0000 7f1881442640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 0x7f18680a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f187c1996d0 tx=0x7f186c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:23:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.642+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1878016030 con 0x7f187c101fa0 2024-04-23T18:23:59.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:23:59 smithi122 ceph-mon[32546]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:23:59.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.872+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f187c02d5c0 con 0x7f187c101fa0 2024-04-23T18:23:59.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.872+0000 7f1872ffd640 1 -- 172.21.15.33:0/1768258355 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f187809d450 con 0x7f187c101fa0 2024-04-23T18:23:59.865 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:23:59.866 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:23:59.866 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-04-23T18:23:59.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 msgr2=0x7f18680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 0x7f18680a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f187c1996d0 tx=0x7f186c008040 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 msgr2=0x7f187c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f187800ea80 tx=0x7f187800ef40 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 shutdown_connections 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f18680a3bb0 0x7f18680a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f187c103b50 0x7f187c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f187c103190 0x7f187c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 --2- 172.21.15.33:0/1768258355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f187c101fa0 0x7f187c1981c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 >> 172.21.15.33:0/1768258355 conn(0x7f187c0fd530 msgr2=0x7f187c106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.874+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 shutdown_connections 2024-04-23T18:23:59.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:23:59.875+0000 7f1883ea7640 1 -- 172.21.15.33:0/1768258355 wait complete. 2024-04-23T18:23:59.877 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 podman[69864]: 2024-04-23 18:23:59.493004631 +0000 UTC m=+0.019039589 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:23:59.878 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 podman[69864]: 2024-04-23 18:23:59.622766694 +0000 UTC m=+0.148801619 container create 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T18:24:00.230 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 podman[69864]: 2024-04-23 18:23:59.892136764 +0000 UTC m=+0.418171701 container init 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-23T18:24:00.231 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 podman[69864]: 2024-04-23 18:23:59.894736717 +0000 UTC m=+0.420771642 container start 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T18:24:00.231 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 ceph-mds[69924]: -- 172.21.15.33:0/3402323560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645d5f6da40 con 0x5645d5f40c00 2024-04-23T18:24:00.231 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:23:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: starting mds.d at 2024-04-23T18:24:00.231 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:24:00 smithi033 bash[69864]: 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a 2024-04-23T18:24:00.231 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:24:00 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:00.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1768258355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:00.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:00 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:00.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1768258355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:00.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:00 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:00.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1768258355' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:00.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:00.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:00 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:01.360 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:01 smithi106 ceph-mon[32716]: Deploying daemon mds.e on smithi106 2024-04-23T18:24:01.360 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:01 smithi106 ceph-mon[32716]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:01.360 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:01 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/3122213420,v1:172.21.15.33:6837/3122213420] up:boot 2024-04-23T18:24:01.361 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:01 smithi106 ceph-mon[32716]: fsmap 4 up:standby 2024-04-23T18:24:01.361 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:01 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:24:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:01 smithi122 ceph-mon[32546]: Deploying daemon mds.e on smithi106 2024-04-23T18:24:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:01 smithi122 ceph-mon[32546]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:01 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/3122213420,v1:172.21.15.33:6837/3122213420] up:boot 2024-04-23T18:24:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:01 smithi122 ceph-mon[32546]: fsmap 4 up:standby 2024-04-23T18:24:01.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:01 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:24:01.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:01 smithi033 ceph-mon[29274]: Deploying daemon mds.e on smithi106 2024-04-23T18:24:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:01 smithi033 ceph-mon[29274]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:01 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/3122213420,v1:172.21.15.33:6837/3122213420] up:boot 2024-04-23T18:24:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:01 smithi033 ceph-mon[29274]: fsmap 4 up:standby 2024-04-23T18:24:01.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:01 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:24:01.880 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 -- 172.21.15.33:0/1706886139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4105360 msgr2=0x7ffaa4107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/1706886139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4105360 0x7ffaa4107750 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009900 tx=0x7ffa9402a410 comp rx=0 tx=0).stop 2024-04-23T18:24:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 -- 172.21.15.33:0/1706886139 shutdown_connections 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/1706886139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4107c90 0x7ffaa410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/1706886139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4105360 0x7ffaa4107750 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/1706886139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 -- 172.21.15.33:0/1706886139 >> 172.21.15.33:0/1706886139 conn(0x7ffaa40fc440 msgr2=0x7ffaa40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 -- 172.21.15.33:0/1706886139 shutdown_connections 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 -- 172.21.15.33:0/1706886139 wait complete. 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.005+0000 7ffaac7d8640 1 Processor -- start 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 -- start start 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 0x7ffaa41986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410dab0 con 0x7ffaa4107c90 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410d930 con 0x7ffaa4105360 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaac7d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa410dc30 con 0x7ffaa4102a30 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47642/0 (socket says 172.21.15.33:47642) 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 -- 172.21.15.33:0/4277173820 learned_addr learned my addr 172.21.15.33:0/4277173820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaa574640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 -- 172.21.15.33:0/4277173820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 msgr2=0x7ffaa4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaa9d73640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 0x7ffaa41986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 -- 172.21.15.33:0/4277173820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 msgr2=0x7ffaa41986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 0x7ffaa41986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 -- 172.21.15.33:0/4277173820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa419fdb0 con 0x7ffaa4107c90 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaa574640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaa9d73640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 0x7ffaa41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffaaad75640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00eb10 tx=0x7ffa9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.006+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c004b20 con 0x7ffaa4107c90 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.007+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c004cc0 con 0x7ffaa4107c90 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.007+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41a00a0 con 0x7ffaa4107c90 2024-04-23T18:24:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.007+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c010670 con 0x7ffaa4107c90 2024-04-23T18:24:02.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.007+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a05f0 con 0x7ffaa4107c90 2024-04-23T18:24:02.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.007+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa40fdf30 con 0x7ffaa4107c90 2024-04-23T18:24:02.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.009+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa9c005000 con 0x7ffaa4107c90 2024-04-23T18:24:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.009+0000 7ffa9b7fe640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 0x7ffa900a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.009+0000 7ffaaa574640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 0x7ffa900a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.009+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ffa9c0d07c0 con 0x7ffaa4107c90 2024-04-23T18:24:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.010+0000 7ffaaa574640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 0x7ffa900a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000baa0 tx=0x7ffaa0005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:02.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.013+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa9c053ff0 con 0x7ffaa4107c90 2024-04-23T18:24:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.241+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaa402d5c0 con 0x7ffaa4107c90 2024-04-23T18:24:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.242+0000 7ffa9b7fe640 1 -- 172.21.15.33:0/4277173820 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7ffa9c09d640 con 0x7ffaa4107c90 2024-04-23T18:24:02.235 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:02.235 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-23T18:24:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 msgr2=0x7ffa900a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 0x7ffa900a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000baa0 tx=0x7ffaa0005d90 comp rx=0 tx=0).stop 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 msgr2=0x7ffaa419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00eb10 tx=0x7ffa9c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 shutdown_connections 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ffa900a3c30 0x7ffa900a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaa4107c90 0x7ffaa419f740 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaa4105360 0x7ffaa41986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 --2- 172.21.15.33:0/4277173820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4102a30 0x7ffaa4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 >> 172.21.15.33:0/4277173820 conn(0x7ffaa40fc440 msgr2=0x7ffaa41032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 shutdown_connections 2024-04-23T18:24:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:02.244+0000 7ffaac7d8640 1 -- 172.21.15.33:0/4277173820 wait complete. 2024-04-23T18:24:02.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:02 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4277173820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:02.621 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 systemd[1]: Starting Ceph mds.e for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:02.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:02 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4277173820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:02.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:02 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4277173820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:02.955 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 podman[62341]: 2024-04-23 18:24:02.58069767 +0000 UTC m=+0.018205824 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:02.955 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 podman[62341]: 2024-04-23 18:24:02.710098585 +0000 UTC m=+0.147606723 container create be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:24:03.370 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 podman[62341]: 2024-04-23 18:24:02.963006317 +0000 UTC m=+0.400514454 container init be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:24:03.370 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 podman[62341]: 2024-04-23 18:24:02.966026097 +0000 UTC m=+0.403534235 container start be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:24:03.370 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 ceph-mds[62360]: -- 172.21.15.106:0/1346780644 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56534f0d1a40 con 0x56534f0a4800 2024-04-23T18:24:03.370 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: starting mds.e at 2024-04-23T18:24:03.370 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:03 smithi106 bash[62341]: be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca 2024-04-23T18:24:03.371 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 18:24:03 smithi106 systemd[1]: Started Ceph mds.e for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:03.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:03.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:03 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:03 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:03.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:03.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:03.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:03 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:04.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:04.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 -- 172.21.15.33:0/2176759393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 msgr2=0x7fa598102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2176759393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598102460 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009900 tx=0x7fa58c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:04.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 -- 172.21.15.33:0/2176759393 shutdown_connections 2024-04-23T18:24:04.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2176759393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 0x7fa5981092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2176759393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598102460 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2176759393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 0x7fa5981012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 -- 172.21.15.33:0/2176759393 >> 172.21.15.33:0/2176759393 conn(0x7fa5980fc400 msgr2=0x7fa5980fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 -- 172.21.15.33:0/2176759393 shutdown_connections 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 -- 172.21.15.33:0/2176759393 wait complete. 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.371+0000 7fa59dd37640 1 Processor -- start 2024-04-23T18:24:04.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 -- start start 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 0x7fa598198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 0x7fa59819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810c9f0 con 0x7fa598101fe0 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810c870 con 0x7fa5981029a0 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa59dd37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa59810cb70 con 0x7fa598100e90 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47656/0 (socket says 172.21.15.33:47656) 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 -- 172.21.15.33:0/2576105720 learned_addr learned my addr 172.21.15.33:0/2576105720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa597fff640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 0x7fa59819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa5977fe640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 0x7fa598198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 -- 172.21.15.33:0/2576105720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 msgr2=0x7fa598198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 0x7fa598198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 -- 172.21.15.33:0/2576105720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 msgr2=0x7fa59819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 0x7fa59819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 -- 172.21.15.33:0/2576105720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59819fe30 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.372+0000 7fa596ffd640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa58c006010 tx=0x7fa58c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.373+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c03f030 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.373+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5981a00c0 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.373+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa58c0318f0 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.373+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c035070 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.373+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5981a05b0 con 0x7fa598101fe0 2024-04-23T18:24:04.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.374+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5981a0810 con 0x7fa598101fe0 2024-04-23T18:24:04.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.375+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa58c042030 con 0x7fa598101fe0 2024-04-23T18:24:04.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.376+0000 7fa594ff9640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 0x7fa5800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:04.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.376+0000 7fa5977fe640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 0x7fa5800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.376+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa58c0f0800 con 0x7fa598101fe0 2024-04-23T18:24:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.376+0000 7fa5977fe640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 0x7fa5800a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa588009bd0 tx=0x7fa588005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.379+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa58c0bd030 con 0x7fa598101fe0 2024-04-23T18:24:04.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.609+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5981a0ba0 con 0x7fa598101fe0 2024-04-23T18:24:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.610+0000 7fa594ff9640 1 -- 172.21.15.33:0/2576105720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7fa58c0c1330 con 0x7fa598101fe0 2024-04-23T18:24:04.603 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:04.603 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:24:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-04-23T18:24:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 msgr2=0x7fa5800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 0x7fa5800a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa588009bd0 tx=0x7fa588005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 msgr2=0x7fa598198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:04.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa58c006010 tx=0x7fa58c0042f0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 shutdown_connections 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa5800a3bb0 0x7fa5800a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa5981029a0 0x7fa59819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa598101fe0 0x7fa598198750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 --2- 172.21.15.33:0/2576105720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa598100e90 0x7fa598198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 >> 172.21.15.33:0/2576105720 conn(0x7fa5980fc400 msgr2=0x7fa5980fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 shutdown_connections 2024-04-23T18:24:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:04.612+0000 7fa59dd37640 1 -- 172.21.15.33:0/2576105720 wait complete. 2024-04-23T18:24:04.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:04 smithi033 ceph-mon[29274]: Deploying daemon mds.f on smithi122 2024-04-23T18:24:04.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:04 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:boot 2024-04-23T18:24:04.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:04 smithi033 ceph-mon[29274]: fsmap 5 up:standby 2024-04-23T18:24:04.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:04 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:24:04.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:04 smithi122 ceph-mon[32546]: Deploying daemon mds.f on smithi122 2024-04-23T18:24:04.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:04 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:boot 2024-04-23T18:24:04.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:04 smithi122 ceph-mon[32546]: fsmap 5 up:standby 2024-04-23T18:24:04.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:04 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:24:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:04 smithi106 ceph-mon[32716]: Deploying daemon mds.f on smithi122 2024-04-23T18:24:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:04 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:boot 2024-04-23T18:24:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:04 smithi106 ceph-mon[32716]: fsmap 5 up:standby 2024-04-23T18:24:04.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:04 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:24:05.387 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:05.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:05 smithi122 ceph-mon[32546]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:05.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:05 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2576105720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 podman[62936]: 2024-04-23 18:24:05.351259379 +0000 UTC m=+0.018159999 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:05.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 podman[62936]: 2024-04-23 18:24:05.460457421 +0000 UTC m=+0.127358010 container create bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-23T18:24:05.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:05 smithi033 ceph-mon[29274]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:05.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:05 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2576105720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:05.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:05 smithi106 ceph-mon[32716]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:05.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:05 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2576105720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:06.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 podman[62936]: 2024-04-23 18:24:05.713436976 +0000 UTC m=+0.380337564 container init bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:24:06.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 podman[62936]: 2024-04-23 18:24:05.715935361 +0000 UTC m=+0.382835952 container start bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:24:06.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 ceph-mds[62955]: -- 172.21.15.122:0/1492761114 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d44c13a40 con 0x564d44be6c00 2024-04-23T18:24:06.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: starting mds.f at 2024-04-23T18:24:06.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 bash[62936]: bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 2024-04-23T18:24:06.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:24:05 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:06.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 -- 172.21.15.33:0/995309616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c100e90 msgr2=0x7f317c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 --2- 172.21.15.33:0/995309616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c100e90 0x7f317c1012b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f316c009860 tx=0x7f316c02a340 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 -- 172.21.15.33:0/995309616 shutdown_connections 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 --2- 172.21.15.33:0/995309616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c1029a0 0x7f317c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 --2- 172.21.15.33:0/995309616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 0x7f317c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 --2- 172.21.15.33:0/995309616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c100e90 0x7f317c1012b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 -- 172.21.15.33:0/995309616 >> 172.21.15.33:0/995309616 conn(0x7f317c0fc400 msgr2=0x7f317c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 -- 172.21.15.33:0/995309616 shutdown_connections 2024-04-23T18:24:06.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.729+0000 7f3180b20640 1 -- 172.21.15.33:0/995309616 wait complete. 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 Processor -- start 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 -- start start 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c100e90 0x7f317c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 0x7f317c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f317c10c9e0 con 0x7f317c1029a0 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f317c10c860 con 0x7f317c100e90 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3180b20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f317c10cb60 con 0x7f317c101fe0 2024-04-23T18:24:06.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53560/0 (socket says 172.21.15.33:53560) 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 -- 172.21.15.33:0/1062099074 learned_addr learned my addr 172.21.15.33:0/1062099074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f3179d9b640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 0x7f317c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317a59c640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c100e90 0x7f317c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 -- 172.21.15.33:0/1062099074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 msgr2=0x7f317c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 0x7f317c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 -- 172.21.15.33:0/1062099074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c100e90 msgr2=0x7f317c197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c100e90 0x7f317c197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.730+0000 7f317ad9d640 1 -- 172.21.15.33:0/1062099074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317c19ed20 con 0x7f317c1029a0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f317ad9d640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f317400efc0 tx=0x7f317400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174003970 con 0x7f317c1029a0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3174004020 con 0x7f317c1029a0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317c19efb0 con 0x7f317c1029a0 2024-04-23T18:24:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317400b940 con 0x7f317c1029a0 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.731+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317c19f480 con 0x7f317c1029a0 2024-04-23T18:24:06.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.732+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317c108270 con 0x7f317c1029a0 2024-04-23T18:24:06.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.733+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f317401d020 con 0x7f317c1029a0 2024-04-23T18:24:06.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.734+0000 7f316b7fe640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 0x7f31640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:06.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.734+0000 7f317a59c640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 0x7f31640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:06.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.734+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f31740cfe20 con 0x7f317c1029a0 2024-04-23T18:24:06.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.734+0000 7f317a59c640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 0x7f31640a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:06.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.738+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f317409cc80 con 0x7f317c1029a0 2024-04-23T18:24:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.965+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f317c062b50 con 0x7f317c1029a0 2024-04-23T18:24:06.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.966+0000 7f316b7fe640 1 -- 172.21.15.33:0/1062099074 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5527 (secure 0 0 0) 0x7f317409c580 con 0x7f317c1029a0 2024-04-23T18:24:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:06.960 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:24:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-23T18:24:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 msgr2=0x7f31640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 0x7f31640a6070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f316c000c00 tx=0x7f316c030470 comp rx=0 tx=0).stop 2024-04-23T18:24:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 msgr2=0x7f317c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f317400efc0 tx=0x7f317400c800 comp rx=0 tx=0).stop 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 shutdown_connections 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f31640a3bb0 0x7f31640a6070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f317c1029a0 0x7f317c19e6b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f317c101fe0 0x7f317c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 --2- 172.21.15.33:0/1062099074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c100e90 0x7f317c197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 >> 172.21.15.33:0/1062099074 conn(0x7f317c0fc400 msgr2=0x7f317c105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 shutdown_connections 2024-04-23T18:24:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:06.968+0000 7f3180b20640 1 -- 172.21.15.33:0/1062099074 wait complete. 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:06.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: Deploying daemon mds.c on smithi122 2024-04-23T18:24:06.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/3209242154,v1:172.21.15.122:6835/3209242154] up:boot 2024-04-23T18:24:06.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: fsmap 6 up:standby 2024-04-23T18:24:06.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:06 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:24:07.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: Deploying daemon mds.c on smithi122 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/3209242154,v1:172.21.15.122:6835/3209242154] up:boot 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: fsmap 6 up:standby 2024-04-23T18:24:07.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:06 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: Deploying daemon mds.c on smithi122 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/3209242154,v1:172.21.15.122:6835/3209242154] up:boot 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: fsmap 6 up:standby 2024-04-23T18:24:07.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:06 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:24:08.124 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:07 smithi122 ceph-mon[32546]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:08.124 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:07 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1062099074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:07 smithi033 ceph-mon[29274]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:08.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:07 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1062099074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:08.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:07 smithi106 ceph-mon[32716]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:08.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:07 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1062099074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:08.698 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 18:24:08 smithi122 systemd[1]: Started Ceph mds.c for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:08.974 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 -- 172.21.15.33:0/1213273214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 msgr2=0x7f40501077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 --2- 172.21.15.33:0/1213273214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f40501077b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404402a410 comp rx=0 tx=0).stop 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 -- 172.21.15.33:0/1213273214 shutdown_connections 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 --2- 172.21.15.33:0/1213273214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f405010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 --2- 172.21.15.33:0/1213273214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f40501077b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 --2- 172.21.15.33:0/1213273214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 0x7f4050104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 -- 172.21.15.33:0/1213273214 >> 172.21.15.33:0/1213273214 conn(0x7f40500fc440 msgr2=0x7f40500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 -- 172.21.15.33:0/1213273214 shutdown_connections 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.096+0000 7f40561bf640 1 -- 172.21.15.33:0/1213273214 wait complete. 2024-04-23T18:24:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 Processor -- start 2024-04-23T18:24:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 -- start start 2024-04-23T18:24:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 0x7f4050102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f4050100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f405010daf0 con 0x7f40501053c0 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f405010d970 con 0x7f4050107cf0 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f40561bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f405010dc70 con 0x7f4050102a90 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f404effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f404effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53580/0 (socket says 172.21.15.33:53580) 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f404effd640 1 -- 172.21.15.33:0/1043325388 learned_addr learned my addr 172.21.15.33:0/1043325388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f404ffff640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f4050100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.097+0000 7f404f7fe640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 0x7f4050102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 -- 172.21.15.33:0/1043325388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 msgr2=0x7f4050102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 0x7f4050102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 -- 172.21.15.33:0/1043325388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 msgr2=0x7f4050100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f4050100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 -- 172.21.15.33:0/1043325388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4050101810 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404ffff640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f4050100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404effd640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4044006010 tx=0x7f40440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404403f030 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4044031910 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40501a2310 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044035070 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.098+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40501a2860 con 0x7f40501053c0 2024-04-23T18:24:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.099+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40500fdf30 con 0x7f40501053c0 2024-04-23T18:24:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.100+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4044042030 con 0x7f40501053c0 2024-04-23T18:24:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.101+0000 7f404cff9640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 0x7f40380a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.101+0000 7f404f7fe640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 0x7f40380a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.101+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f40440f08e0 con 0x7f40501053c0 2024-04-23T18:24:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.102+0000 7f404f7fe640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 0x7f40380a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4040009bd0 tx=0x7f4040005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:09.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.105+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40440bd070 con 0x7f40501053c0 2024-04-23T18:24:09.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.332+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f405002d810 con 0x7f40501053c0 2024-04-23T18:24:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.333+0000 7f404cff9640 1 -- 172.21.15.33:0/1043325388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f40440c1370 con 0x7f40501053c0 2024-04-23T18:24:09.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:09.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:24:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-04-23T18:24:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 msgr2=0x7f40380a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 0x7f40380a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4040009bd0 tx=0x7f4040005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 msgr2=0x7f4050100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4044006010 tx=0x7f40440042f0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 shutdown_connections 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40380a3bb0 0x7f40380a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4050107cf0 0x7f4050100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40501053c0 0x7f4050100a40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 --2- 172.21.15.33:0/1043325388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4050102a90 0x7f4050102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 >> 172.21.15.33:0/1043325388 conn(0x7f40500fc440 msgr2=0x7f4050104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 shutdown_connections 2024-04-23T18:24:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:09.335+0000 7f40561bf640 1 -- 172.21.15.33:0/1043325388 wait complete. 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: Deploying daemon mds.h on smithi106 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:boot 2024-04-23T18:24:09.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: fsmap 7 up:standby 2024-04-23T18:24:09.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:09.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:09 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1043325388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:10.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: Deploying daemon mds.h on smithi106 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:boot 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: fsmap 7 up:standby 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:10.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:09 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1043325388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:10.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: Deploying daemon mds.h on smithi106 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:boot 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: fsmap 7 up:standby 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:24:10.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:09 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1043325388' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:11.121 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:10 smithi106 systemd[1]: Starting Ceph mds.h for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:11.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 -- 172.21.15.33:0/3591597087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7141029a0 msgr2=0x7fe7141092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 --2- 172.21.15.33:0/3591597087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7141029a0 0x7fe7141092c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009ca0 tx=0x7fe70c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 -- 172.21.15.33:0/3591597087 shutdown_connections 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 --2- 172.21.15.33:0/3591597087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7141029a0 0x7fe7141092c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 --2- 172.21.15.33:0/3591597087 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe714101fe0 0x7fe714102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 --2- 172.21.15.33:0/3591597087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714100e90 0x7fe7141012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 -- 172.21.15.33:0/3591597087 >> 172.21.15.33:0/3591597087 conn(0x7fe7140fc400 msgr2=0x7fe7140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 -- 172.21.15.33:0/3591597087 shutdown_connections 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.461+0000 7fe718afa640 1 -- 172.21.15.33:0/3591597087 wait complete. 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 Processor -- start 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 -- start start 2024-04-23T18:24:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 0x7fe714109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 0x7fe71410a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410daf0 con 0x7fe714100e90 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410d970 con 0x7fe7141029a0 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe718afa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe71410dc70 con 0x7fe714101fe0 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53604/0 (socket says 172.21.15.33:53604) 2024-04-23T18:24:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 -- 172.21.15.33:0/3829603193 learned_addr learned my addr 172.21.15.33:0/3829603193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe712d9d640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 0x7fe71410a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe711d9b640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 0x7fe714109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 -- 172.21.15.33:0/3829603193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 msgr2=0x7fe714109e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 0x7fe714109e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 -- 172.21.15.33:0/3829603193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 msgr2=0x7fe71410a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 0x7fe71410a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe71259c640 1 -- 172.21.15.33:0/3829603193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71410ac40 con 0x7fe714100e90 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe711d9b640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 0x7fe714109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.462+0000 7fe712d9d640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 0x7fe71410a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe71259c640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe70400d950 tx=0x7fe70400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704017030 con 0x7fe714100e90 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7141a2270 con 0x7fe714100e90 2024-04-23T18:24:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe704004020 con 0x7fe714100e90 2024-04-23T18:24:11.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040047a0 con 0x7fe714100e90 2024-04-23T18:24:11.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.463+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7141a27c0 con 0x7fe714100e90 2024-04-23T18:24:11.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.464+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7141081a0 con 0x7fe714100e90 2024-04-23T18:24:11.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.465+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe704009a30 con 0x7fe714100e90 2024-04-23T18:24:11.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.466+0000 7fe7037fe640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 0x7fe6fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:11.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.466+0000 7fe711d9b640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 0x7fe6fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:11.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.466+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7040cfbc0 con 0x7fe714100e90 2024-04-23T18:24:11.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.466+0000 7fe711d9b640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 0x7fe6fc0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe71410b5a0 tx=0x7fe708005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:11.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.471+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe70409c9a0 con 0x7fe714100e90 2024-04-23T18:24:11.478 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 podman[62651]: 2024-04-23 18:24:11.111908584 +0000 UTC m=+0.017877540 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:11.478 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 podman[62651]: 2024-04-23 18:24:11.241060353 +0000 UTC m=+0.147029295 container create a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-23T18:24:11.478 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 podman[62651]: 2024-04-23 18:24:11.485568042 +0000 UTC m=+0.391536985 container init a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T18:24:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.699+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe71402d5c0 con 0x7fe714100e90 2024-04-23T18:24:11.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.700+0000 7fe7037fe640 1 -- 172.21.15.33:0/3829603193 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fe70409c2a0 con 0x7fe714100e90 2024-04-23T18:24:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:11.693 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-23T18:24:11.694 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.701+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 msgr2=0x7fe6fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.701+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 0x7fe6fc0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe71410b5a0 tx=0x7fe708005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 msgr2=0x7fe71410b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe70400d950 tx=0x7fe70400de10 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 shutdown_connections 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe6fc0a3bb0 0x7fe6fc0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7141029a0 0x7fe71410a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe714101fe0 0x7fe714109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 --2- 172.21.15.33:0/3829603193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe714100e90 0x7fe71410b830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 >> 172.21.15.33:0/3829603193 conn(0x7fe7140fc400 msgr2=0x7fe714105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 shutdown_connections 2024-04-23T18:24:11.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:11.702+0000 7fe718afa640 1 -- 172.21.15.33:0/3829603193 wait complete. 2024-04-23T18:24:11.728 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 podman[62651]: 2024-04-23 18:24:11.488148426 +0000 UTC m=+0.394117382 container start a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T18:24:11.728 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mds[62670]: -- 172.21.15.106:0/4070705379 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a5423fa40 con 0x560a54212c00 2024-04-23T18:24:11.728 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: starting mds.h at 2024-04-23T18:24:11.728 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 bash[62651]: a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 2024-04-23T18:24:11.728 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 18:24:11 smithi106 systemd[1]: Started Ceph mds.h for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:11.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:11.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:11.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:11 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3829603193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:12.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:12.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:12.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:11 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3829603193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:12.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:12.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:12.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:11 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3829603193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:12.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:12 smithi122 ceph-mon[32546]: Deploying daemon mds.i on smithi122 2024-04-23T18:24:12.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:12 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:boot 2024-04-23T18:24:12.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:12 smithi122 ceph-mon[32546]: fsmap 8 up:standby 2024-04-23T18:24:12.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:12 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:24:13.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:12 smithi106 ceph-mon[32716]: Deploying daemon mds.i on smithi122 2024-04-23T18:24:13.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:12 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:boot 2024-04-23T18:24:13.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:12 smithi106 ceph-mon[32716]: fsmap 8 up:standby 2024-04-23T18:24:13.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:12 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:24:13.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:12 smithi033 ceph-mon[29274]: Deploying daemon mds.i on smithi122 2024-04-23T18:24:13.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:12 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:boot 2024-04-23T18:24:13.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:12 smithi033 ceph-mon[29274]: fsmap 8 up:standby 2024-04-23T18:24:13.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:12 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:24:13.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:13.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- 172.21.15.33:0/1484030493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 msgr2=0x7fc490074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:13.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 --2- 172.21.15.33:0/1484030493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc490074360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009900 tx=0x7fc48c02a480 comp rx=0 tx=0).stop 2024-04-23T18:24:13.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- 172.21.15.33:0/1484030493 shutdown_connections 2024-04-23T18:24:13.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 --2- 172.21.15.33:0/1484030493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc49010c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 --2- 172.21.15.33:0/1484030493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 0x7fc490074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 --2- 172.21.15.33:0/1484030493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc490074360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- 172.21.15.33:0/1484030493 >> 172.21.15.33:0/1484030493 conn(0x7fc4900fd550 msgr2=0x7fc4900ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- 172.21.15.33:0/1484030493 shutdown_connections 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- 172.21.15.33:0/1484030493 wait complete. 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 Processor -- start 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.830+0000 7fc49749a640 1 -- start start 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 0x7fc49019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc4901a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:13.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc49010fd10 con 0x7fc490075f10 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc49010fb90 con 0x7fc490109e70 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc49010fe90 con 0x7fc490074930 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53628/0 (socket says 172.21.15.33:53628) 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 -- 172.21.15.33:0/2742094107 learned_addr learned my addr 172.21.15.33:0/2742094107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc495236640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 0x7fc49019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc495a37640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc4901a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 -- 172.21.15.33:0/2742094107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 msgr2=0x7fc49019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 0x7fc49019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 -- 172.21.15.33:0/2742094107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 msgr2=0x7fc4901a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc4901a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 -- 172.21.15.33:0/2742094107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4901a1f80 con 0x7fc490075f10 2024-04-23T18:24:13.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc495a37640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc4901a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc494a35640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d770 tx=0x7fc48000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc490075f10 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4901a2270 con 0x7fc490075f10 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4800054d0 con 0x7fc490075f10 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.831+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480004c30 con 0x7fc490075f10 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.832+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4901a2770 con 0x7fc490075f10 2024-04-23T18:24:13.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.832+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc490108c30 con 0x7fc490075f10 2024-04-23T18:24:13.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.834+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc480004dd0 con 0x7fc490075f10 2024-04-23T18:24:13.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.834+0000 7fc4867fc640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:13.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.834+0000 7fc495236640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:13.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.834+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc4800cfe60 con 0x7fc490075f10 2024-04-23T18:24:13.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.835+0000 7fc495236640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:13.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:13.838+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc48009cce0 con 0x7fc490075f10 2024-04-23T18:24:13.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:13 smithi122 ceph-mon[32546]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:13.949 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:13 smithi122 systemd[1]: Starting Ceph mds.i for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:14.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.066+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc49002d5c0 con 0x7fc490075f10 2024-04-23T18:24:14.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.066+0000 7fc4867fc640 1 -- 172.21.15.33:0/2742094107 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7fc48009c5e0 con 0x7fc490075f10 2024-04-23T18:24:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:14.059 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:24:14.060 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-23T18:24:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 msgr2=0x7fc47c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc48c000c00 tx=0x7fc48c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 msgr2=0x7fc49019a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d770 tx=0x7fc48000dc30 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 shutdown_connections 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc490109e70 0x7fc4901a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc490075f10 0x7fc49019a8a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 --2- 172.21.15.33:0/2742094107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc490074930 0x7fc49019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 >> 172.21.15.33:0/2742094107 conn(0x7fc4900fd550 msgr2=0x7fc490105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 shutdown_connections 2024-04-23T18:24:14.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:14.069+0000 7fc49749a640 1 -- 172.21.15.33:0/2742094107 wait complete. 2024-04-23T18:24:14.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:13 smithi033 ceph-mon[29274]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:14.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:13 smithi106 ceph-mon[32716]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:14.248 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:13 smithi122 podman[63553]: 2024-04-23 18:24:13.868518966 +0000 UTC m=+0.017969662 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:14.248 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:13 smithi122 podman[63553]: 2024-04-23 18:24:13.976870496 +0000 UTC m=+0.126321178 container create 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T18:24:14.248 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 podman[63553]: 2024-04-23 18:24:14.255624587 +0000 UTC m=+0.405075270 container init 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:24:14.698 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 podman[63553]: 2024-04-23 18:24:14.258724181 +0000 UTC m=+0.408174868 container start 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T18:24:14.699 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mds[63572]: -- 172.21.15.122:0/261862299 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5578a0a73a40 con 0x5578a0a46800 2024-04-23T18:24:14.699 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: starting mds.i at 2024-04-23T18:24:14.699 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 bash[63553]: 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f 2024-04-23T18:24:14.699 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 18:24:14 smithi122 systemd[1]: Started Ceph mds.i for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2742094107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:15.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:14 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2742094107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:15.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:15.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:14 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2742094107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:15.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:15.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:15.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:14 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:16.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:16.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:15 smithi033 ceph-mon[29274]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:16.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:15 smithi033 ceph-mon[29274]: Deploying daemon mds.g on smithi033 2024-04-23T18:24:16.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:boot 2024-04-23T18:24:16.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:15 smithi033 ceph-mon[29274]: fsmap 9 up:standby 2024-04-23T18:24:16.101 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:15 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:24:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:15 smithi106 ceph-mon[32716]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:15 smithi106 ceph-mon[32716]: Deploying daemon mds.g on smithi033 2024-04-23T18:24:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:boot 2024-04-23T18:24:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:15 smithi106 ceph-mon[32716]: fsmap 9 up:standby 2024-04-23T18:24:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:15 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 -- 172.21.15.33:0/1418741173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 msgr2=0x7fc60410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 --2- 172.21.15.33:0/1418741173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60410a3e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009ca0 tx=0x7fc5fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 -- 172.21.15.33:0/1418741173 shutdown_connections 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 --2- 172.21.15.33:0/1418741173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60410a3e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 --2- 172.21.15.33:0/1418741173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 0x7fc604103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 --2- 172.21.15.33:0/1418741173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 0x7fc6041023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 -- 172.21.15.33:0/1418741173 >> 172.21.15.33:0/1418741173 conn(0x7fc6040fd530 msgr2=0x7fc6040ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 -- 172.21.15.33:0/1418741173 shutdown_connections 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.198+0000 7fc60ab07640 1 -- 172.21.15.33:0/1418741173 wait complete. 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 Processor -- start 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 -- start start 2024-04-23T18:24:16.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 0x7fc604198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 0x7fc6041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410dae0 con 0x7fc604103b50 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410d960 con 0x7fc604101fa0 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc60ab07640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc60410dc60 con 0x7fc604103190 2024-04-23T18:24:16.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53644/0 (socket says 172.21.15.33:53644) 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 -- 172.21.15.33:0/3489260921 learned_addr learned my addr 172.21.15.33:0/3489260921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6088a3640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 0x7fc604198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc603fff640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 0x7fc6041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 -- 172.21.15.33:0/3489260921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 msgr2=0x7fc6041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 0x7fc6041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 -- 172.21.15.33:0/3489260921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 msgr2=0x7fc604198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 0x7fc604198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.199+0000 7fc6090a4640 1 -- 172.21.15.33:0/3489260921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60419fdb0 con 0x7fc604103b50 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc6090a4640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc031a20 tx=0x7fc5fc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc004240 con 0x7fc604103b50 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5fc003e10 con 0x7fc604103b50 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6041a0040 con 0x7fc604103b50 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0305c0 con 0x7fc604103b50 2024-04-23T18:24:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.200+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041a05e0 con 0x7fc604103b50 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.201+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc604109310 con 0x7fc604103b50 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.202+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc5fc02f660 con 0x7fc604103b50 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.203+0000 7fc601ffb640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 0x7fc5f40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.203+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc5fc0f1920 con 0x7fc604103b50 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.203+0000 7fc6088a3640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 0x7fc5f40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:16.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.204+0000 7fc6088a3640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 0x7fc5f40a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc604102ff0 tx=0x7fc5f800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.207+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5fc0be7a0 con 0x7fc604103b50 2024-04-23T18:24:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:15 smithi122 ceph-mon[32546]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:15 smithi122 ceph-mon[32546]: Deploying daemon mds.g on smithi033 2024-04-23T18:24:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:boot 2024-04-23T18:24:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:15 smithi122 ceph-mon[32546]: fsmap 9 up:standby 2024-04-23T18:24:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:15 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:24:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.449+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc60402d5c0 con 0x7fc604103b50 2024-04-23T18:24:16.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.450+0000 7fc601ffb640 1 -- 172.21.15.33:0/3489260921 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fc5fc0be0a0 con 0x7fc604103b50 2024-04-23T18:24:16.443 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:16.443 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:24:16.443 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 10 2024-04-23T18:24:16.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 msgr2=0x7fc5f40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 0x7fc5f40a60f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc604102ff0 tx=0x7fc5f800a2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 msgr2=0x7fc60419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:16.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc031a20 tx=0x7fc5fc003c20 comp rx=0 tx=0).stop 2024-04-23T18:24:16.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 shutdown_connections 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5f40a3c30 0x7fc5f40a60f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc604103b50 0x7fc60419f740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc604103190 0x7fc6041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 --2- 172.21.15.33:0/3489260921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc604101fa0 0x7fc604198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 >> 172.21.15.33:0/3489260921 conn(0x7fc6040fd530 msgr2=0x7fc6041064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 shutdown_connections 2024-04-23T18:24:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:16.453+0000 7fc60ab07640 1 -- 172.21.15.33:0/3489260921 wait complete. 2024-04-23T18:24:17.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:16 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3489260921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:17.049 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:16 smithi033 systemd[1]: Starting Ceph mds.g for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:16 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3489260921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:17.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:16 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3489260921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:17.307 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 podman[70512]: 2024-04-23 18:24:16.958763755 +0000 UTC m=+0.018318139 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:17.307 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 podman[70512]: 2024-04-23 18:24:17.087019873 +0000 UTC m=+0.146574236 container create 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:24:17.307 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 podman[70512]: 2024-04-23 18:24:17.314805928 +0000 UTC m=+0.374360295 container init 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:24:17.730 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 podman[70512]: 2024-04-23 18:24:17.317580884 +0000 UTC m=+0.377135249 container start 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T18:24:17.731 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mds[70531]: -- 172.21.15.33:0/1200399311 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c34d41a40 con 0x556c34d14800 2024-04-23T18:24:17.731 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: starting mds.g at 2024-04-23T18:24:17.731 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 bash[70512]: 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 2024-04-23T18:24:17.731 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 18:24:17 smithi033 systemd[1]: Started Ceph mds.g for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:18.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:18.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:17 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:18.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:18.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:18.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:18.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:17 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:18.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:17 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:18.461 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 -- 172.21.15.33:0/4225930838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c102a90 msgr2=0x7f027c104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 --2- 172.21.15.33:0/4225930838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c102a90 0x7f027c104e80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0264009900 tx=0x7f026402a410 comp rx=0 tx=0).stop 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 -- 172.21.15.33:0/4225930838 shutdown_connections 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 --2- 172.21.15.33:0/4225930838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c107cf0 0x7f027c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 --2- 172.21.15.33:0/4225930838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 0x7f027c1077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 --2- 172.21.15.33:0/4225930838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c102a90 0x7f027c104e80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 -- 172.21.15.33:0/4225930838 >> 172.21.15.33:0/4225930838 conn(0x7f027c0fc400 msgr2=0x7f027c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 -- 172.21.15.33:0/4225930838 shutdown_connections 2024-04-23T18:24:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.575+0000 7f02811bf640 1 -- 172.21.15.33:0/4225930838 wait complete. 2024-04-23T18:24:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 Processor -- start 2024-04-23T18:24:18.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 -- start start 2024-04-23T18:24:18.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c102a90 0x7f027c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 0x7f027c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10dac0 con 0x7f027c107cf0 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10d940 con 0x7f027c102a90 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f02811bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10dc40 con 0x7f027c1053c0 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f027b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f027b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35780/0 (socket says 172.21.15.33:35780) 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f027b59e640 1 -- 172.21.15.33:0/711314467 learned_addr learned my addr 172.21.15.33:0/711314467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f027a59c640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 0x7f027c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.576+0000 7f027ad9d640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c102a90 0x7f027c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 -- 172.21.15.33:0/711314467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 msgr2=0x7f027c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 0x7f027c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 -- 172.21.15.33:0/711314467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c102a90 msgr2=0x7f027c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c102a90 0x7f027c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 -- 172.21.15.33:0/711314467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c19fe10 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f027b59e640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f027400eb10 tx=0x7f027400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274004b20 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274004cc0 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1a0100 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274010670 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.577+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a0600 con 0x7f027c107cf0 2024-04-23T18:24:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.578+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f027c0fdef0 con 0x7f027c107cf0 2024-04-23T18:24:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.580+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0274005000 con 0x7f027c107cf0 2024-04-23T18:24:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.580+0000 7f0273fff640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 0x7f02680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.580+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f02740d06f0 con 0x7f027c107cf0 2024-04-23T18:24:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.580+0000 7f027ad9d640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 0x7f02680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.581+0000 7f027ad9d640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 0x7f02680a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0264000c00 tx=0x7f02640314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.584+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0274053ff0 con 0x7f027c107cf0 2024-04-23T18:24:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.817+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f027c02d5c0 con 0x7f027c107cf0 2024-04-23T18:24:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.818+0000 7f0273fff640 1 -- 172.21.15.33:0/711314467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f027409d570 con 0x7f027c107cf0 2024-04-23T18:24:18.811 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:18.811 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:24:18.811 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 msgr2=0x7f02680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 0x7f02680a6070 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0264000c00 tx=0x7f02640314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 msgr2=0x7f027c19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f027400eb10 tx=0x7f027400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 shutdown_connections 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f02680a3bb0 0x7f02680a6070 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f027c107cf0 0x7f027c19f7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f027c1053c0 0x7f027c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 --2- 172.21.15.33:0/711314467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f027c102a90 0x7f027c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 >> 172.21.15.33:0/711314467 conn(0x7f027c0fc400 msgr2=0x7f027c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 shutdown_connections 2024-04-23T18:24:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:18.820+0000 7f02811bf640 1 -- 172.21.15.33:0/711314467 wait complete. 2024-04-23T18:24:18.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:18 smithi122 ceph-mon[32546]: Deploying daemon mds.a on smithi033 2024-04-23T18:24:18.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:18 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6838/1373616976,v1:172.21.15.33:6839/1373616976] up:boot 2024-04-23T18:24:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:18 smithi122 ceph-mon[32546]: fsmap 10 up:standby 2024-04-23T18:24:18.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:18 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:24:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:18 smithi106 ceph-mon[32716]: Deploying daemon mds.a on smithi033 2024-04-23T18:24:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:18 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6838/1373616976,v1:172.21.15.33:6839/1373616976] up:boot 2024-04-23T18:24:19.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:18 smithi106 ceph-mon[32716]: fsmap 10 up:standby 2024-04-23T18:24:19.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:18 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:24:19.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:18 smithi033 ceph-mon[29274]: Deploying daemon mds.a on smithi033 2024-04-23T18:24:19.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:18 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6838/1373616976,v1:172.21.15.33:6839/1373616976] up:boot 2024-04-23T18:24:19.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:18 smithi033 ceph-mon[29274]: fsmap 10 up:standby 2024-04-23T18:24:19.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:18 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:24:19.766 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:19 smithi033 systemd[1]: Starting Ceph mds.a for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:24:20.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:19 smithi033 ceph-mon[29274]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:20.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/711314467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:20.029 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:19 smithi033 podman[70876]: 2024-04-23 18:24:19.813930066 +0000 UTC m=+0.018733663 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:24:20.029 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:19 smithi033 podman[70876]: 2024-04-23 18:24:19.943192552 +0000 UTC m=+0.147996182 container create 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T18:24:20.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:19 smithi106 ceph-mon[32716]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:20.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/711314467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:20.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:19 smithi122 ceph-mon[32546]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:20.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/711314467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:20.480 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 podman[70876]: 2024-04-23 18:24:20.145834871 +0000 UTC m=+0.350638457 container init 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T18:24:20.480 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 podman[70876]: 2024-04-23 18:24:20.148947055 +0000 UTC m=+0.353750650 container start 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T18:24:20.481 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 ceph-mds[70895]: -- 172.21.15.33:0/4071361382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563524d1da40 con 0x563524cf0800 2024-04-23T18:24:20.481 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: starting mds.a at 2024-04-23T18:24:20.481 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 bash[70876]: 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 2024-04-23T18:24:20.481 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 18:24:20 smithi033 systemd[1]: Started Ceph mds.a for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:24:20.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- 172.21.15.33:0/172571548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 msgr2=0x7f0d4410a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/172571548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4410a490 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009cd0 tx=0x7f0d3c02a8e0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- 172.21.15.33:0/172571548 shutdown_connections 2024-04-23T18:24:20.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/172571548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4410a490 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/172571548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 0x7f0d441036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/172571548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- 172.21.15.33:0/172571548 >> 172.21.15.33:0/172571548 conn(0x7f0d440fbbc0 msgr2=0x7f0d440fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- 172.21.15.33:0/172571548 shutdown_connections 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- 172.21.15.33:0/172571548 wait complete. 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 Processor -- start 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- start start 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 0x7f0d441987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410db70 con 0x7f0d44103c00 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410d9f0 con 0x7f0d44103240 2024-04-23T18:24:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.943+0000 7f0d4ad20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4410dcf0 con 0x7f0d44100270 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35810/0 (socket says 172.21.15.33:35810) 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 -- 172.21.15.33:0/1849884681 learned_addr learned my addr 172.21.15.33:0/1849884681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d3bfff640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 0x7f0d441987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d48abc640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 -- 172.21.15.33:0/1849884681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 msgr2=0x7f0d44198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 -- 172.21.15.33:0/1849884681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 msgr2=0x7f0d441987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 0x7f0d441987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 -- 172.21.15.33:0/1849884681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419fe80 con 0x7f0d44103c00 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d48abc640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d492bd640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0318b0 tx=0x7f0d3c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c004250 con 0x7f0d44103c00 2024-04-23T18:24:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d441a0110 con 0x7f0d44103c00 2024-04-23T18:24:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c0305a0 con 0x7f0d44103c00 2024-04-23T18:24:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0466c0 con 0x7f0d44103c00 2024-04-23T18:24:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.944+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a06b0 con 0x7f0d44103c00 2024-04-23T18:24:20.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.945+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d44109440 con 0x7f0d44103c00 2024-04-23T18:24:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.946+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0d3c030750 con 0x7f0d44103c00 2024-04-23T18:24:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.947+0000 7f0d39ffb640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 0x7f0d2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.947+0000 7f0d48abc640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 0x7f0d2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.947+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f0d3c02e030 con 0x7f0d44103c00 2024-04-23T18:24:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.948+0000 7f0d48abc640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 0x7f0d2c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006fd0 tx=0x7f0d40008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:20.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:20.952+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d3c04b060 con 0x7f0d44103c00 2024-04-23T18:24:21.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.175+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d44062b50 con 0x7f0d44103c00 2024-04-23T18:24:21.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.176+0000 7f0d39ffb640 1 -- 172.21.15.33:0/1849884681 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f0d3c0be810 con 0x7f0d44103c00 2024-04-23T18:24:21.169 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:21.169 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-23T18:24:21.169 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 msgr2=0x7f0d2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 0x7f0d2c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d40006fd0 tx=0x7f0d40008040 comp rx=0 tx=0).stop 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 msgr2=0x7f0d4419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0318b0 tx=0x7f0d3c003c10 comp rx=0 tx=0).stop 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 shutdown_connections 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0d2c0a3bb0 0x7f0d2c0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d44103c00 0x7f0d4419f810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d44103240 0x7f0d441987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 --2- 172.21.15.33:0/1849884681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d44100270 0x7f0d44198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 >> 172.21.15.33:0/1849884681 conn(0x7f0d440fbbc0 msgr2=0x7f0d44106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 shutdown_connections 2024-04-23T18:24:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:21.178+0000 7f0d4ad20640 1 -- 172.21.15.33:0/1849884681 wait complete. 2024-04-23T18:24:21.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: Deploying daemon mds.b on smithi106 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6840/363309287,v1:172.21.15.33:6841/363309287] up:boot 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: fsmap 11 up:standby 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:24:21.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:21 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1849884681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:21.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: Deploying daemon mds.b on smithi106 2024-04-23T18:24:21.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6840/363309287,v1:172.21.15.33:6841/363309287] up:boot 2024-04-23T18:24:21.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: fsmap 11 up:standby 2024-04-23T18:24:21.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:24:21.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:21 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1849884681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:21.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: Deploying daemon mds.b on smithi106 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6840/363309287,v1:172.21.15.33:6841/363309287] up:boot 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: fsmap 11 up:standby 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:24:21.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:21 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1849884681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:23.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:23.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- 172.21.15.33:0/1009927581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd081029a0 msgr2=0x7efd081092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 --2- 172.21.15.33:0/1009927581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd081029a0 0x7efd081092c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7efd00009ca0 tx=0x7efd0002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- 172.21.15.33:0/1009927581 shutdown_connections 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 --2- 172.21.15.33:0/1009927581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd081029a0 0x7efd081092c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 --2- 172.21.15.33:0/1009927581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 0x7efd08102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 --2- 172.21.15.33:0/1009927581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd08100e90 0x7efd081012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- 172.21.15.33:0/1009927581 >> 172.21.15.33:0/1009927581 conn(0x7efd080fc400 msgr2=0x7efd080fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- 172.21.15.33:0/1009927581 shutdown_connections 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- 172.21.15.33:0/1009927581 wait complete. 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 Processor -- start 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.309+0000 7efd0fe69640 1 -- start start 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 0x7efd0819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd081029a0 0x7efd0819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810c9d0 con 0x7efd08100e90 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810c850 con 0x7efd08101fe0 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd0810cb50 con 0x7efd081029a0 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35826/0 (socket says 172.21.15.33:35826) 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 -- 172.21.15.33:0/2121826349 learned_addr learned my addr 172.21.15.33:0/2121826349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0d404640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 0x7efd0819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 -- 172.21.15.33:0/2121826349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd081029a0 msgr2=0x7efd0819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd081029a0 0x7efd0819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 -- 172.21.15.33:0/2121826349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 msgr2=0x7efd0819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 0x7efd0819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 -- 172.21.15.33:0/2121826349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0819fd80 con 0x7efd08100e90 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0dc05640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c890 tx=0x7efd0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004220 con 0x7efd08100e90 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd040043c0 con 0x7efd08100e90 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd081a0070 con 0x7efd08100e90 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd04004a00 con 0x7efd08100e90 2024-04-23T18:24:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.310+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd081a0610 con 0x7efd08100e90 2024-04-23T18:24:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.312+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd08108270 con 0x7efd08100e90 2024-04-23T18:24:23.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.313+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd040079b0 con 0x7efd08100e90 2024-04-23T18:24:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.314+0000 7efcfeffd640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 0x7efcf40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.314+0000 7efd0d404640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 0x7efcf40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.314+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efd040cfcc0 con 0x7efd08100e90 2024-04-23T18:24:23.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.314+0000 7efd0d404640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 0x7efcf40a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd080699e0 tx=0x7efcf800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.319+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd040d6080 con 0x7efd08100e90 2024-04-23T18:24:23.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:23 smithi106 ceph-mon[32716]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:23.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:23 smithi033 ceph-mon[29274]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:23.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.550+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd0802d5c0 con 0x7efd08100e90 2024-04-23T18:24:23.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.551+0000 7efcfeffd640 1 -- 172.21.15.33:0/2121826349 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7efd0802d5c0 con 0x7efd08100e90 2024-04-23T18:24:23.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:23.544 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24446,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24466,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24482,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24515,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24551,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-23T18:24:23.544 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-04-23T18:24:23.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 msgr2=0x7efcf40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 0x7efcf40a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd080699e0 tx=0x7efcf800a2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 msgr2=0x7efd08069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd0400c890 tx=0x7efd0400cd50 comp rx=0 tx=0).stop 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 shutdown_connections 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efcf40a3c30 0x7efcf40a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd081029a0 0x7efd0819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd08101fe0 0x7efd0819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 --2- 172.21.15.33:0/2121826349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd08100e90 0x7efd08069c70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 >> 172.21.15.33:0/2121826349 conn(0x7efd080fc400 msgr2=0x7efd08105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 shutdown_connections 2024-04-23T18:24:23.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.553+0000 7efd0fe69640 1 -- 172.21.15.33:0/2121826349 wait complete. 2024-04-23T18:24:23.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:24:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 -- 172.21.15.33:0/1668306091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 msgr2=0x7f3504104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 --2- 172.21.15.33:0/1668306091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504104e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009900 tx=0x7f34f802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 -- 172.21.15.33:0/1668306091 shutdown_connections 2024-04-23T18:24:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 --2- 172.21.15.33:0/1668306091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504107c70 0x7f350410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 --2- 172.21.15.33:0/1668306091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504105340 0x7f3504107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 --2- 172.21.15.33:0/1668306091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504104e00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 -- 172.21.15.33:0/1668306091 >> 172.21.15.33:0/1668306091 conn(0x7f35040fc400 msgr2=0x7f35040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 -- 172.21.15.33:0/1668306091 shutdown_connections 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.669+0000 7f350bbfe640 1 -- 172.21.15.33:0/1668306091 wait complete. 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 Processor -- start 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 -- start start 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504105340 0x7f3504198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504107c70 0x7f350419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f350410dad0 con 0x7f3504102a10 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f350410d950 con 0x7f3504105340 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350bbfe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f350410dc50 con 0x7f3504107c70 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35842/0 (socket says 172.21.15.33:35842) 2024-04-23T18:24:23.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 -- 172.21.15.33:0/2305786109 learned_addr learned my addr 172.21.15.33:0/2305786109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f3509199640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504105340 0x7f3504198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350a19b640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504107c70 0x7f350419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 -- 172.21.15.33:0/2305786109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504107c70 msgr2=0x7f350419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504107c70 0x7f350419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 -- 172.21.15.33:0/2305786109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504105340 msgr2=0x7f3504198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504105340 0x7f3504198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 -- 172.21.15.33:0/2305786109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350419fe20 con 0x7f3504102a10 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.670+0000 7f350999a640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005940 tx=0x7f34f8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.671+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f803f030 con 0x7f3504102a10 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.671+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35041a00b0 con 0x7f3504102a10 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.671+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34f8003cb0 con 0x7f3504102a10 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.671+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8035070 con 0x7f3504102a10 2024-04-23T18:24:23.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.671+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35041a05a0 con 0x7f3504102a10 2024-04-23T18:24:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.672+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040fdef0 con 0x7f3504102a10 2024-04-23T18:24:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.673+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f34f8042030 con 0x7f3504102a10 2024-04-23T18:24:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.673+0000 7f34f6ffd640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 0x7f34f00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.674+0000 7f3509199640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 0x7f34f00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:23.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.674+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f34f80f0800 con 0x7f3504102a10 2024-04-23T18:24:23.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.674+0000 7f3509199640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 0x7f34f00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3504199710 tx=0x7f350000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:23.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.678+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34f8003e60 con 0x7f3504102a10 2024-04-23T18:24:23.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:23 smithi122 ceph-mon[32546]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:23.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.886+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3504062b50 con 0x7f3504102a10 2024-04-23T18:24:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.887+0000 7f34f6ffd640 1 -- 172.21.15.33:0/2305786109 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18481 (secure 0 0 0) 0x7f34f80bd030 con 0x7f3504102a10 2024-04-23T18:24:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:23.881 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:23:37.902951+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 msgr2=0x7f34f00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 0x7f34f00a6070 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3504199710 tx=0x7f350000b390 comp rx=0 tx=0).stop 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 msgr2=0x7f3504198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f34f8005940 tx=0x7f34f8004430 comp rx=0 tx=0).stop 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 shutdown_connections 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f34f00a3bb0 0x7f34f00a6070 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3504107c70 0x7f350419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3504105340 0x7f3504198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 --2- 172.21.15.33:0/2305786109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3504102a10 0x7f3504198200 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 >> 172.21.15.33:0/2305786109 conn(0x7f35040fc400 msgr2=0x7f3504103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 shutdown_connections 2024-04-23T18:24:23.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:23.890+0000 7f350bbfe640 1 -- 172.21.15.33:0/2305786109 wait complete. 2024-04-23T18:24:23.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:24:23.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:24:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.010+0000 7efff1091640 1 -- 172.21.15.33:0/2987878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec075f30 msgr2=0x7effec074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.010+0000 7efff1091640 1 --2- 172.21.15.33:0/2987878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec075f30 0x7effec074380 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7effdc009900 tx=0x7effdc02a480 comp rx=0 tx=0).stop 2024-04-23T18:24:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 -- 172.21.15.33:0/2987878038 shutdown_connections 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 --2- 172.21.15.33:0/2987878038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec109e30 0x7effec10c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 --2- 172.21.15.33:0/2987878038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec074950 0x7effec074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 --2- 172.21.15.33:0/2987878038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec075f30 0x7effec074380 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 -- 172.21.15.33:0/2987878038 >> 172.21.15.33:0/2987878038 conn(0x7effec0fd530 msgr2=0x7effec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 -- 172.21.15.33:0/2987878038 shutdown_connections 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.011+0000 7efff1091640 1 -- 172.21.15.33:0/2987878038 wait complete. 2024-04-23T18:24:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 Processor -- start 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 -- start start 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec075f30 0x7effec19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec109e30 0x7effec1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7effec10fce0 con 0x7effec074950 2024-04-23T18:24:24.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effec10fb60 con 0x7effec109e30 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7efff1091640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effec10fe60 con 0x7effec075f30 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35846/0 (socket says 172.21.15.33:35846) 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 -- 172.21.15.33:0/1906391145 learned_addr learned my addr 172.21.15.33:0/1906391145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effea59c640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec075f30 0x7effec19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effeb59e640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec109e30 0x7effec1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 -- 172.21.15.33:0/1906391145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec075f30 msgr2=0x7effec19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec075f30 0x7effec19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.012+0000 7effead9d640 1 -- 172.21.15.33:0/1906391145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec109e30 msgr2=0x7effec1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effead9d640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec109e30 0x7effec1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effead9d640 1 -- 172.21.15.33:0/1906391145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effec1a2040 con 0x7effec074950 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effead9d640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7effdc005940 tx=0x7effdc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc03f030 con 0x7effec074950 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effdc003aa0 con 0x7effec074950 2024-04-23T18:24:24.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effdc035070 con 0x7effec074950 2024-04-23T18:24:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7efff1091640 1 -- 172.21.15.33:0/1906391145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effec1a22d0 con 0x7effec074950 2024-04-23T18:24:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.013+0000 7efff1091640 1 -- 172.21.15.33:0/1906391145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effec1a2770 con 0x7effec074950 2024-04-23T18:24:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.014+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effcc005290 con 0x7effec074950 2024-04-23T18:24:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.016+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7effdc042030 con 0x7effec074950 2024-04-23T18:24:24.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.017+0000 7effd3fff640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 0x7effd40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.017+0000 7effea59c640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 0x7effd40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.017+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7effdc0f0b40 con 0x7effec074950 2024-04-23T18:24:24.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.017+0000 7effea59c640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 0x7effd40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effec19b930 tx=0x7effe0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.023+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effdc0bd2d0 con 0x7effec074950 2024-04-23T18:24:24.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.233+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7effcc005850 con 0x7effec074950 2024-04-23T18:24:24.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.234+0000 7effd3fff640 1 -- 172.21.15.33:0/1906391145 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7effdc0c15d0 con 0x7effec074950 2024-04-23T18:24:24.227 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 msgr2=0x7effd40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 0x7effd40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effec19b930 tx=0x7effe0005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 msgr2=0x7effec19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7effdc005940 tx=0x7effdc031a50 comp rx=0 tx=0).stop 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 shutdown_connections 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7effd40a3c30 0x7effd40a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effec109e30 0x7effec1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effec075f30 0x7effec19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 --2- 172.21.15.33:0/1906391145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effec074950 0x7effec19a420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 >> 172.21.15.33:0/1906391145 conn(0x7effec0fd530 msgr2=0x7effec105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 shutdown_connections 2024-04-23T18:24:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.236+0000 7effd1ffb640 1 -- 172.21.15.33:0/1906391145 wait complete. 2024-04-23T18:24:24.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-23T18:24:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 -- 172.21.15.33:0/3844826167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 msgr2=0x7fcdf41023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3844826167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf41023d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009ca0 tx=0x7fcde402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 -- 172.21.15.33:0/3844826167 shutdown_connections 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3844826167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf4103b60 0x7fcdf410a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3844826167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf41031a0 0x7fcdf4103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3844826167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf41023d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 -- 172.21.15.33:0/3844826167 >> 172.21.15.33:0/3844826167 conn(0x7fcdf40fd500 msgr2=0x7fcdf40ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 -- 172.21.15.33:0/3844826167 shutdown_connections 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 -- 172.21.15.33:0/3844826167 wait complete. 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.351+0000 7fcdf8936640 1 Processor -- start 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 -- start start 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 0x7fcdf4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf4103b60 0x7fcdf419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf410daf0 con 0x7fcdf4101fb0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf410d970 con 0x7fcdf41031a0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf8936640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf410dc70 con 0x7fcdf4103b60 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35856/0 (socket says 172.21.15.33:35856) 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 -- 172.21.15.33:0/3029438811 learned_addr learned my addr 172.21.15.33:0/3029438811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf1d9b640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 0x7fcdf4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf2d9d640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf4103b60 0x7fcdf419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 -- 172.21.15.33:0/3029438811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf4103b60 msgr2=0x7fcdf419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf4103b60 0x7fcdf419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 -- 172.21.15.33:0/3029438811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 msgr2=0x7fcdf4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 0x7fcdf4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 -- 172.21.15.33:0/3029438811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf419fe20 con 0x7fcdf4101fb0 2024-04-23T18:24:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf1d9b640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 0x7fcdf4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdf259c640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcde40098d0 tx=0x7fcde4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4003cb0 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcde4003e50 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.352+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde402fc60 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.353+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf41a00b0 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.353+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf41a0600 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.354+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf4109390 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.355+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcde4004040 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.355+0000 7fcdeb7fe640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 0x7fcde00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.356+0000 7fcdf1d9b640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 0x7fcde00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.356+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fcde40f1620 con 0x7fcdf4101fb0 2024-04-23T18:24:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.356+0000 7fcdf1d9b640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 0x7fcde00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4199710 tx=0x7fcddc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.359+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcde40be530 con 0x7fcdf4101fb0 2024-04-23T18:24:24.537 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:boot 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: fsmap 12 up:standby 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2121826349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2305786109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:24 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1906391145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:24:24.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.553+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcdf4062b50 con 0x7fcdf4101fb0 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.554+0000 7fcdeb7fe640 1 -- 172.21.15.33:0/3029438811 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fcde40bde30 con 0x7fcdf4101fb0 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:24:24.547 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 msgr2=0x7fcde00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 0x7fcde00a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4199710 tx=0x7fcddc00b410 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 msgr2=0x7fcdf4198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcde40098d0 tx=0x7fcde4031840 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 shutdown_connections 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fcde00a3bb0 0x7fcde00a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdf4103b60 0x7fcdf419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcdf41031a0 0x7fcdf4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 --2- 172.21.15.33:0/3029438811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf4101fb0 0x7fcdf4198200 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 >> 172.21.15.33:0/3029438811 conn(0x7fcdf40fd500 msgr2=0x7fcdf4106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:24.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 shutdown_connections 2024-04-23T18:24:24.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.556+0000 7fcdf8936640 1 -- 172.21.15.33:0/3029438811 wait complete. 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:boot 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: fsmap 12 up:standby 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2121826349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2305786109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:24 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1906391145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:24:24.560 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-23T18:24:24.561 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-23T18:24:24.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:boot 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: fsmap 12 up:standby 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2121826349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2305786109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:24.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:24 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1906391145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:24:24.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- 172.21.15.33:0/778009931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe314100e90 msgr2=0x7fe3141012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/778009931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe314100e90 0x7fe3141012b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe310009900 tx=0x7fe31002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:24.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- 172.21.15.33:0/778009931 shutdown_connections 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/778009931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3141029a0 0x7fe3141092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/778009931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314101fe0 0x7fe314102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/778009931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe314100e90 0x7fe3141012b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- 172.21.15.33:0/778009931 >> 172.21.15.33:0/778009931 conn(0x7fe3140fc400 msgr2=0x7fe3140fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- 172.21.15.33:0/778009931 shutdown_connections 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- 172.21.15.33:0/778009931 wait complete. 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 Processor -- start 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 -- start start 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.675+0000 7fe31c7bf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314100e90 0x7fe314069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe314101fe0 0x7fe31419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe31410c9e0 con 0x7fe3141029a0 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe31410c860 con 0x7fe314100e90 2024-04-23T18:24:24.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe31410cb60 con 0x7fe314101fe0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35880/0 (socket says 172.21.15.33:35880) 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31a55b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314100e90 0x7fe314069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 -- 172.21.15.33:0/2256057498 learned_addr learned my addr 172.21.15.33:0/2256057498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe319d5a640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe314101fe0 0x7fe31419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 -- 172.21.15.33:0/2256057498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe314101fe0 msgr2=0x7fe31419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe314101fe0 0x7fe31419cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 -- 172.21.15.33:0/2256057498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314100e90 msgr2=0x7fe314069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314100e90 0x7fe314069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 -- 172.21.15.33:0/2256057498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31419fea0 con 0x7fe3141029a0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31ad5c640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00cad0 tx=0x7fe30c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00ae20 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe30c005020 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30c00f6b0 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3141a0130 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.676+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3141a0630 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.677+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3141082c0 con 0x7fe3141029a0 2024-04-23T18:24:24.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.678+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe30c0051d0 con 0x7fe3141029a0 2024-04-23T18:24:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.679+0000 7fe30b7fe640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 0x7fe3000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.679+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe30c0cff30 con 0x7fe3141029a0 2024-04-23T18:24:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.679+0000 7fe31a55b640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 0x7fe3000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.680+0000 7fe31a55b640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 0x7fe3000a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe3100314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.684+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe30c014030 con 0x7fe3141029a0 2024-04-23T18:24:24.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:24.901+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe31402d810 con 0x7fe3141029a0 2024-04-23T18:24:25.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.268+0000 7fe30b7fe640 1 -- 172.21.15.33:0/2256057498 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fe30c09cdb0 con 0x7fe3141029a0 2024-04-23T18:24:25.261 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-04-23T18:24:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 msgr2=0x7fe3000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 0x7fe3000a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c00 tx=0x7fe3100314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 msgr2=0x7fe31419f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00cad0 tx=0x7fe30c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 shutdown_connections 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe3000a3bb0 0x7fe3000a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3141029a0 0x7fe31419f590 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe314101fe0 0x7fe31419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 --2- 172.21.15.33:0/2256057498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe314100e90 0x7fe314069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 >> 172.21.15.33:0/2256057498 conn(0x7fe3140fc400 msgr2=0x7fe314105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.270+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 shutdown_connections 2024-04-23T18:24:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.271+0000 7fe31c7bf640 1 -- 172.21.15.33:0/2256057498 wait complete. 2024-04-23T18:24:25.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-23T18:24:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 -- 172.21.15.33:0/2031582971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d81011e0 msgr2=0x7f90d8101640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 --2- 172.21.15.33:0/2031582971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d81011e0 0x7f90d8101640 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f90c0009900 tx=0x7f90c002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 -- 172.21.15.33:0/2031582971 shutdown_connections 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 --2- 172.21.15.33:0/2031582971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8101b80 0x7f90d8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 --2- 172.21.15.33:0/2031582971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d81011e0 0x7f90d8101640 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 --2- 172.21.15.33:0/2031582971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d8100880 0x7f90d8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 -- 172.21.15.33:0/2031582971 >> 172.21.15.33:0/2031582971 conn(0x7f90d80fc440 msgr2=0x7f90d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 -- 172.21.15.33:0/2031582971 shutdown_connections 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.396+0000 7f90dee35640 1 -- 172.21.15.33:0/2031582971 wait complete. 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 Processor -- start 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 -- start start 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8100880 0x7f90d819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 0x7f90d819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810fd00 con 0x7f90d8101b80 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810fb80 con 0x7f90d81011e0 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dee35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90d810fe80 con 0x7f90d8100880 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35888/0 (socket says 172.21.15.33:35888) 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 -- 172.21.15.33:0/218145429 learned_addr learned my addr 172.21.15.33:0/218145429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dcbd1640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8100880 0x7f90d819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90d7fff640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 0x7f90d819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 -- 172.21.15.33:0/218145429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8100880 msgr2=0x7f90d819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8100880 0x7f90d819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 -- 172.21.15.33:0/218145429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 msgr2=0x7f90d819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:25.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 0x7f90d819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90dd3d2640 1 -- 172.21.15.33:0/218145429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d81a1ff0 con 0x7f90d8101b80 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.397+0000 7f90d7fff640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 0x7f90d819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90dd3d2640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d950 tx=0x7f90c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8017030 con 0x7f90d8101b80 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d81a22e0 con 0x7f90d8101b80 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90c8004020 con 0x7f90d8101b80 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8003b50 con 0x7f90d8101b80 2024-04-23T18:24:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.398+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81a27e0 con 0x7f90d8101b80 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.399+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d810b5d0 con 0x7f90d8101b80 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.400+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f90c801a070 con 0x7f90d8101b80 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.401+0000 7f90d5ffb640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.401+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f90c80cfbe0 con 0x7f90d8101b80 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.401+0000 7f90dcbd1640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.401+0000 7f90dcbd1640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 0x7f90c40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005c40 tx=0x7f90d000a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.405+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90c8021070 con 0x7f90d8101b80 2024-04-23T18:24:25.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:25.592+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f90d802d5c0 con 0x7f90d8101b80 2024-04-23T18:24:25.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:25 smithi106 ceph-mon[32716]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:25.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:25 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3029438811' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:24:25.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:25 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:24:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:25.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:25.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:25 smithi122 ceph-mon[32546]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:25.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:25 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3029438811' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:24:25.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:25 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:24:25.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:25.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:25 smithi033 ceph-mon[29274]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:25 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3029438811' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-23T18:24:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:25 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-23T18:24:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:25.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.270+0000 7f90d5ffb640 1 -- 172.21.15.33:0/218145429 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f90c809c8f0 con 0x7f90d8101b80 2024-04-23T18:24:26.263 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-04-23T18:24:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 msgr2=0x7f90c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 0x7f90c40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f90d0005c40 tx=0x7f90d000a590 comp rx=0 tx=0).stop 2024-04-23T18:24:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 msgr2=0x7f90d81a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d950 tx=0x7f90c800de10 comp rx=0 tx=0).stop 2024-04-23T18:24:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 shutdown_connections 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f90c40a3bb0 0x7f90c40a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90d8101b80 0x7f90d81a1980 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90d81011e0 0x7f90d819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 --2- 172.21.15.33:0/218145429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90d8100880 0x7f90d819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 >> 172.21.15.33:0/218145429 conn(0x7f90d80fc440 msgr2=0x7f90d80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 shutdown_connections 2024-04-23T18:24:26.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.273+0000 7f90dee35640 1 -- 172.21.15.33:0/218145429 wait complete. 2024-04-23T18:24:26.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-23T18:24:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 -- 172.21.15.33:0/336486135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9260101fe0 msgr2=0x7f9260102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 --2- 172.21.15.33:0/336486135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9260101fe0 0x7f9260102460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f925c009900 tx=0x7f925c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 -- 172.21.15.33:0/336486135 shutdown_connections 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 --2- 172.21.15.33:0/336486135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92601029a0 0x7f92601092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 --2- 172.21.15.33:0/336486135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9260101fe0 0x7f9260102460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 --2- 172.21.15.33:0/336486135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260100e90 0x7f92601012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 -- 172.21.15.33:0/336486135 >> 172.21.15.33:0/336486135 conn(0x7f92600fc400 msgr2=0x7f92600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 -- 172.21.15.33:0/336486135 shutdown_connections 2024-04-23T18:24:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.403+0000 7f9267f20640 1 -- 172.21.15.33:0/336486135 wait complete. 2024-04-23T18:24:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 Processor -- start 2024-04-23T18:24:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 -- start start 2024-04-23T18:24:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 0x7f9260069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 0x7f926019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f926010ca30 con 0x7f92601029a0 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f926010c8b0 con 0x7f9260101fe0 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9267f20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f926010cbb0 con 0x7f9260100e90 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f92664bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f92664bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35914/0 (socket says 172.21.15.33:35914) 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f92664bd640 1 -- 172.21.15.33:0/83066797 learned_addr learned my addr 172.21.15.33:0/83066797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f9265cbc640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 0x7f9260069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.404+0000 7f92654bb640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 0x7f926019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 -- 172.21.15.33:0/83066797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 msgr2=0x7f9260069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 0x7f9260069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 -- 172.21.15.33:0/83066797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 msgr2=0x7f926019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 0x7f926019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 -- 172.21.15.33:0/83066797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92601a20a0 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9265cbc640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 0x7f9260069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92654bb640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 0x7f926019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f92664bd640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f925800eb10 tx=0x7f925800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258004b20 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9258004cc0 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601a2330 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258010670 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.405+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a2850 con 0x7f92601029a0 2024-04-23T18:24:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.406+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92601081c0 con 0x7f92601029a0 2024-04-23T18:24:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.407+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9258004fe0 con 0x7f92601029a0 2024-04-23T18:24:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.408+0000 7f9256ffd640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 0x7f924c0a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.408+0000 7f9265cbc640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 0x7f924c0a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:26.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.408+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f92580d09f0 con 0x7f92601029a0 2024-04-23T18:24:26.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.409+0000 7f9265cbc640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 0x7f924c0a60a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9250009fc0 tx=0x7f9250005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:26.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.412+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9258053ff0 con 0x7f92601029a0 2024-04-23T18:24:26.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:26.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:26 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:26.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:26.660+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9260062bc0 con 0x7f92601029a0 2024-04-23T18:24:26.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:26 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:26.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2256057498' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-23T18:24:26.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: osdmap e82: 12 total, 12 up, 12 in 2024-04-23T18:24:26.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:26.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:26 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:24:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.280+0000 7f9256ffd640 1 -- 172.21.15.33:0/83066797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f925809d4e0 con 0x7f92601029a0 2024-04-23T18:24:27.274 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 msgr2=0x7f924c0a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 0x7f924c0a60a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9250009fc0 tx=0x7f9250005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 msgr2=0x7f92601a18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f925800eb10 tx=0x7f925800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 shutdown_connections 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f924c0a3be0 0x7f924c0a60a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92601029a0 0x7f92601a18b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9260101fe0 0x7f926019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 --2- 172.21.15.33:0/83066797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9260100e90 0x7f9260069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.283+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 >> 172.21.15.33:0/83066797 conn(0x7f92600fc400 msgr2=0x7f9260105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.284+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 shutdown_connections 2024-04-23T18:24:27.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.284+0000 7f9267f20640 1 -- 172.21.15.33:0/83066797 wait complete. 2024-04-23T18:24:27.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:24:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 -- 172.21.15.33:0/218906658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd041021e0 msgr2=0x7fbd04108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 --2- 172.21.15.33:0/218906658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd041021e0 0x7fbd04108a70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009ca0 tx=0x7fbcfc02ecd0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 -- 172.21.15.33:0/218906658 shutdown_connections 2024-04-23T18:24:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 --2- 172.21.15.33:0/218906658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd041021e0 0x7fbd04108a70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 --2- 172.21.15.33:0/218906658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd04101820 0x7fbd04101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 --2- 172.21.15.33:0/218906658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04100630 0x7fbd04100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 -- 172.21.15.33:0/218906658 >> 172.21.15.33:0/218906658 conn(0x7fbd040fbbc0 msgr2=0x7fbd040fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 -- 172.21.15.33:0/218906658 shutdown_connections 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.416+0000 7fbd09e49640 1 -- 172.21.15.33:0/218906658 wait complete. 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 Processor -- start 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 -- start start 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 0x7fbd04196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd041021e0 0x7fbd0419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410b8a0 con 0x7fbd04100630 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410b720 con 0x7fbd04101820 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.417+0000 7fbd09e49640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0410ba20 con 0x7fbd041021e0 2024-04-23T18:24:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd02ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 0x7fbd04196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44446/0 (socket says 172.21.15.33:44446) 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 -- 172.21.15.33:0/2251164268 learned_addr learned my addr 172.21.15.33:0/2251164268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd03fff640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd041021e0 0x7fbd0419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 -- 172.21.15.33:0/2251164268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd041021e0 msgr2=0x7fbd0419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd041021e0 0x7fbd0419d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 -- 172.21.15.33:0/2251164268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 msgr2=0x7fbd04196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 0x7fbd04196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 -- 172.21.15.33:0/2251164268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0419db70 con 0x7fbd04100630 2024-04-23T18:24:27.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd02ffd640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 0x7fbd04196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd037fe640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400e9f0 tx=0x7fbcf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf400cce0 con 0x7fbd04100630 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf40035f0 con 0x7fbd04100630 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0419de60 con 0x7fbd04100630 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.418+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4016670 con 0x7fbd04100630 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.419+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd0419e2c0 con 0x7fbd04100630 2024-04-23T18:24:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.419+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd04107a10 con 0x7fbd04100630 2024-04-23T18:24:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.421+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbcf401d020 con 0x7fbd04100630 2024-04-23T18:24:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.422+0000 7fbd00ff9640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.422+0000 7fbd02ffd640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.422+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7fbcf40d03b0 con 0x7fbd04100630 2024-04-23T18:24:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.422+0000 7fbd02ffd640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197460 tx=0x7fbcf800f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.426+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcf409cec0 con 0x7fbd04100630 2024-04-23T18:24:27.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:27.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:24:27.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:24:27.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:24:27.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:24:27.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:24:27.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:27 smithi122 ceph-mon[32546]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:24:27.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:27.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:24:27.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:24:27.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:24:27.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:24:27.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:24:27.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:27 smithi033 ceph-mon[29274]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:24:27.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: pgmap v283: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:24:27.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/218145429' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-23T18:24:27.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: osdmap e83: 12 total, 12 up, 12 in 2024-04-23T18:24:27.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-23T18:24:27.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-23T18:24:27.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-23T18:24:27.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:27 smithi106 ceph-mon[32716]: osdmap e84: 12 total, 12 up, 12 in 2024-04-23T18:24:27.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.629+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd0402d790 con 0x7fbd04100630 2024-04-23T18:24:27.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.631+0000 7fbd00ff9640 1 -- 172.21.15.33:0/2251164268 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22257 (secure 0 0 0) 0x7fbcf409c7c0 con 0x7fbd04100630 2024-04-23T18:24:27.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:27.624 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:24:26.661520+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 msgr2=0x7fbcec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbd04197460 tx=0x7fbcf800f040 comp rx=0 tx=0).stop 2024-04-23T18:24:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 msgr2=0x7fbd04195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcf400e9f0 tx=0x7fbcf400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 shutdown_connections 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcec0a3bb0 0x7fbcec0a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd041021e0 0x7fbd0419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd04101820 0x7fbd04196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 --2- 172.21.15.33:0/2251164268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd04100630 0x7fbd04195f50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 >> 172.21.15.33:0/2251164268 conn(0x7fbd040fbbc0 msgr2=0x7fbd04104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 shutdown_connections 2024-04-23T18:24:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.633+0000 7fbd09e49640 1 -- 172.21.15.33:0/2251164268 wait complete. 2024-04-23T18:24:27.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:24:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 -- 172.21.15.33:0/1748129006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 msgr2=0x7f78a010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/1748129006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a010a0c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7898009ca0 tx=0x7f789802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 -- 172.21.15.33:0/1748129006 shutdown_connections 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/1748129006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a010a0c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/1748129006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 0x7f78a0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/1748129006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 0x7f78a0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 -- 172.21.15.33:0/1748129006 >> 172.21.15.33:0/1748129006 conn(0x7f78a00fc400 msgr2=0x7f78a00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 -- 172.21.15.33:0/1748129006 shutdown_connections 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.754+0000 7f78a5b4f640 1 -- 172.21.15.33:0/1748129006 wait complete. 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 Processor -- start 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 -- start start 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 0x7f78a0198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 0x7f78a01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010dae0 con 0x7f78a0107c70 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010d960 con 0x7f78a0105340 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f78a5b4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78a010dc60 con 0x7f78a0102a10 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f789ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f789ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44468/0 (socket says 172.21.15.33:44468) 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f789ffff640 1 -- 172.21.15.33:0/608771334 learned_addr learned my addr 172.21.15.33:0/608771334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f789f7fe640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 0x7f78a0198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.755+0000 7f789effd640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 0x7f78a01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 -- 172.21.15.33:0/608771334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 msgr2=0x7f78a0198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 0x7f78a0198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 -- 172.21.15.33:0/608771334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 msgr2=0x7f78a01986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 0x7f78a01986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 -- 172.21.15.33:0/608771334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a019fdb0 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789ffff640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78980097b0 tx=0x7f7898031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898003bf0 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7898003d90 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a01a0040 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789802fc70 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.756+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a01a0590 con 0x7f78a0107c70 2024-04-23T18:24:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.757+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a00fdef0 con 0x7f78a0107c70 2024-04-23T18:24:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.758+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7898004040 con 0x7f78a0107c70 2024-04-23T18:24:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.759+0000 7f789cff9640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 0x7f788c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.759+0000 7f789f7fe640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 0x7f788c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.759+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f78980f18c0 con 0x7f78a0107c70 2024-04-23T18:24:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.760+0000 7f789f7fe640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 0x7f788c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7888005fd0 tx=0x7f788800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.763+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78980be3f0 con 0x7f78a0107c70 2024-04-23T18:24:27.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.977+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78a0062b50 con 0x7f78a0107c70 2024-04-23T18:24:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.979+0000 7f789cff9640 1 -- 172.21.15.33:0/608771334 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7f78980bdcf0 con 0x7f78a0107c70 2024-04-23T18:24:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:27.973 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:24:27.868758+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 msgr2=0x7f788c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 0x7f788c0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7888005fd0 tx=0x7f788800b390 comp rx=0 tx=0).stop 2024-04-23T18:24:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 msgr2=0x7f78a019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f78980097b0 tx=0x7f7898031840 comp rx=0 tx=0).stop 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 shutdown_connections 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f788c0a3bb0 0x7f788c0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78a0107c70 0x7f78a019f740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0105340 0x7f78a01986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 --2- 172.21.15.33:0/608771334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78a0102a10 0x7f78a0198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 >> 172.21.15.33:0/608771334 conn(0x7f78a00fc400 msgr2=0x7f78a01032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 shutdown_connections 2024-04-23T18:24:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:27.981+0000 7f78a5b4f640 1 -- 172.21.15.33:0/608771334 wait complete. 2024-04-23T18:24:27.987 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-23T18:24:28.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 -- 172.21.15.33:0/2706408659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20105360 msgr2=0x7f9e20107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/2706408659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20105360 0x7f9e20107750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009900 tx=0x7f9e1002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 -- 172.21.15.33:0/2706408659 shutdown_connections 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/2706408659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20107c90 0x7f9e2010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/2706408659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20105360 0x7f9e20107750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/2706408659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 0x7f9e20104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 -- 172.21.15.33:0/2706408659 >> 172.21.15.33:0/2706408659 conn(0x7f9e200fc420 msgr2=0x7f9e200fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 -- 172.21.15.33:0/2706408659 shutdown_connections 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.099+0000 7f9e27fe2640 1 -- 172.21.15.33:0/2706408659 wait complete. 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 Processor -- start 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 -- start start 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 0x7f9e201023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 0x7f9e201009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010daf0 con 0x7f9e20107c90 2024-04-23T18:24:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010d970 con 0x7f9e20102a30 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e27fe2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2010dc70 con 0x7f9e20105360 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44482/0 (socket says 172.21.15.33:44482) 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 -- 172.21.15.33:0/995281945 learned_addr learned my addr 172.21.15.33:0/995281945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2557d640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 0x7f9e201009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e25d7e640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 0x7f9e201023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 -- 172.21.15.33:0/995281945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 msgr2=0x7f9e201009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 0x7f9e201009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 -- 172.21.15.33:0/995281945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 msgr2=0x7f9e201023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 0x7f9e201023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2657f640 1 -- 172.21.15.33:0/995281945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e201017b0 con 0x7f9e20107c90 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.100+0000 7f9e2557d640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 0x7f9e201009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e2657f640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb10 tx=0x7f9e1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18004b20 con 0x7f9e20107c90 2024-04-23T18:24:28.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e18004cc0 con 0x7f9e20107c90 2024-04-23T18:24:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e201a22f0 con 0x7f9e20107c90 2024-04-23T18:24:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18010670 con 0x7f9e20107c90 2024-04-23T18:24:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.101+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e201a2810 con 0x7f9e20107c90 2024-04-23T18:24:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.102+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e200691c0 con 0x7f9e20107c90 2024-04-23T18:24:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.103+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e18005020 con 0x7f9e20107c90 2024-04-23T18:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.103+0000 7f9e16ffd640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 0x7f9e0c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.104+0000 7f9e25d7e640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 0x7f9e0c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.104+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f9e180d09d0 con 0x7f9e20107c90 2024-04-23T18:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.104+0000 7f9e25d7e640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 0x7f9e0c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005e10 tx=0x7f9e1c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:28.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.108+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e1809d500 con 0x7f9e20107c90 2024-04-23T18:24:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:28.344+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9e20062b50 con 0x7f9e20107c90 2024-04-23T18:24:28.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2251164268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:24:28.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/608771334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2251164268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:24:28.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/608771334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/83066797' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-23T18:24:28.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-23T18:24:28.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2251164268' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: osdmap e85: 12 total, 12 up, 12 in 2024-04-23T18:24:28.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/608771334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:29.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.300+0000 7f9e16ffd640 1 -- 172.21.15.33:0/995281945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f9e1809ce00 con 0x7f9e20107c90 2024-04-23T18:24:29.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 msgr2=0x7f9e0c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 0x7f9e0c0a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005e10 tx=0x7f9e1c005da0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 msgr2=0x7f9e20100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800eb10 tx=0x7f9e1800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 shutdown_connections 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9e0c0a3bb0 0x7f9e0c0a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e20107c90 0x7f9e20100f20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e20105360 0x7f9e201009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 --2- 172.21.15.33:0/995281945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e20102a30 0x7f9e201023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 >> 172.21.15.33:0/995281945 conn(0x7f9e200fc420 msgr2=0x7f9e201034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.303+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 shutdown_connections 2024-04-23T18:24:29.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.304+0000 7f9e27fe2640 1 -- 172.21.15.33:0/995281945 wait complete. 2024-04-23T18:24:29.310 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-23T18:24:29.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: pgmap v287: 97 pgs: 53 active+clean, 28 creating+peering, 16 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:active 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:24:29.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:24:29.334 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:29 smithi033 ceph-mon[29274]: Cluster is now healthy 2024-04-23T18:24:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 -- 172.21.15.33:0/2445148282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105400 msgr2=0x7f71941077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 --2- 172.21.15.33:0/2445148282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105400 0x7f71941077f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7188009900 tx=0x7f718802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 -- 172.21.15.33:0/2445148282 shutdown_connections 2024-04-23T18:24:29.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 --2- 172.21.15.33:0/2445148282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194107d30 0x7f719410a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 --2- 172.21.15.33:0/2445148282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194105400 0x7f71941077f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 --2- 172.21.15.33:0/2445148282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194102ad0 0x7f7194104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 -- 172.21.15.33:0/2445148282 >> 172.21.15.33:0/2445148282 conn(0x7f71940fc440 msgr2=0x7f71940fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 -- 172.21.15.33:0/2445148282 shutdown_connections 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.428+0000 7f719ab76640 1 -- 172.21.15.33:0/2445148282 wait complete. 2024-04-23T18:24:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 Processor -- start 2024-04-23T18:24:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 -- start start 2024-04-23T18:24:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 0x7f7194198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194105400 0x7f71941987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f719410db10 con 0x7f7194107d30 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f719410d990 con 0x7f7194102ad0 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f719ab76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f719410dc90 con 0x7f7194105400 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f7199113640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f7199113640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44512/0 (socket says 172.21.15.33:44512) 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f7199113640 1 -- 172.21.15.33:0/1635279248 learned_addr learned my addr 172.21.15.33:0/1635279248 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.429+0000 7f7193fff640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194105400 0x7f71941987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7198912640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 0x7f7194198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 -- 172.21.15.33:0/1635279248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194105400 msgr2=0x7f71941987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194105400 0x7f71941987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 -- 172.21.15.33:0/1635279248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 msgr2=0x7f7194198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 0x7f7194198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 -- 172.21.15.33:0/1635279248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719419fe90 con 0x7f7194107d30 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7198912640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 0x7f7194198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7199113640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f718c00cad0 tx=0x7f718c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c00ae20 con 0x7f7194107d30 2024-04-23T18:24:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718c005020 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71941a0180 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c00f6b0 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.430+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a0650 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.431+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71940fdf30 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.432+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f718c0051d0 con 0x7f7194107d30 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.433+0000 7f7191ffb640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 0x7f717c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.433+0000 7f7198912640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 0x7f717c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.433+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f718c0d0410 con 0x7f7194107d30 2024-04-23T18:24:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.433+0000 7f7198912640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 0x7f717c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7184009b40 tx=0x7f7184005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.437+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f718c014030 con 0x7f7194107d30 2024-04-23T18:24:29.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: pgmap v287: 97 pgs: 53 active+clean, 28 creating+peering, 16 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:24:29.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:active 2024-04-23T18:24:29.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:24:29.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:24:29.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:29 smithi106 ceph-mon[32716]: Cluster is now healthy 2024-04-23T18:24:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:29.666+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f7194062b50 con 0x7f7194107d30 2024-04-23T18:24:29.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: pgmap v287: 97 pgs: 53 active+clean, 28 creating+peering, 16 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-23T18:24:29.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] up:active 2024-04-23T18:24:29.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:29.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-23T18:24:29.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-23T18:24:29.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:29 smithi122 ceph-mon[32546]: Cluster is now healthy 2024-04-23T18:24:30.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.305+0000 7f7191ffb640 1 -- 172.21.15.33:0/1635279248 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f718c09cea0 con 0x7f7194107d30 2024-04-23T18:24:30.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 msgr2=0x7f717c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 0x7f717c0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7184009b40 tx=0x7f7184005e50 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 msgr2=0x7f719419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f718c00cad0 tx=0x7f718c00cf90 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 shutdown_connections 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f717c0a3bb0 0x7f717c0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7194107d30 0x7f719419f820 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7194105400 0x7f71941987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 --2- 172.21.15.33:0/1635279248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7194102ad0 0x7f7194198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 >> 172.21.15.33:0/1635279248 conn(0x7f71940fc440 msgr2=0x7f7194104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 shutdown_connections 2024-04-23T18:24:30.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.309+0000 7f719ab76640 1 -- 172.21.15.33:0/1635279248 wait complete. 2024-04-23T18:24:30.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-23T18:24:30.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 -- 172.21.15.33:0/3748271079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828107c90 msgr2=0x7ff82810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 --2- 172.21.15.33:0/3748271079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828107c90 0x7ff82810a0e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff820009ca0 tx=0x7ff82002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 -- 172.21.15.33:0/3748271079 shutdown_connections 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 --2- 172.21.15.33:0/3748271079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828107c90 0x7ff82810a0e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 --2- 172.21.15.33:0/3748271079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828105360 0x7ff828107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 --2- 172.21.15.33:0/3748271079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828102a30 0x7ff828104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 -- 172.21.15.33:0/3748271079 >> 172.21.15.33:0/3748271079 conn(0x7ff8280fc440 msgr2=0x7ff8280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 -- 172.21.15.33:0/3748271079 shutdown_connections 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 -- 172.21.15.33:0/3748271079 wait complete. 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.480+0000 7ff82e072640 1 Processor -- start 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 -- start start 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828105360 0x7ff8281986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828107c90 0x7ff82819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:30.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810daf0 con 0x7ff828102a30 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810d970 con 0x7ff828107c90 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff82e072640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff82810dc70 con 0x7ff828105360 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44540/0 (socket says 172.21.15.33:44540) 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 -- 172.21.15.33:0/2266247828 learned_addr learned my addr 172.21.15.33:0/2266247828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff826ffd640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828105360 0x7ff8281986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff827fff640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828107c90 0x7ff82819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 -- 172.21.15.33:0/2266247828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828105360 msgr2=0x7ff8281986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828105360 0x7ff8281986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 -- 172.21.15.33:0/2266247828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828107c90 msgr2=0x7ff82819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828107c90 0x7ff82819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 -- 172.21.15.33:0/2266247828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82819fdc0 con 0x7ff828102a30 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.481+0000 7ff8277fe640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d950 tx=0x7ff81800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.482+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818017030 con 0x7ff828102a30 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.482+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff818004020 con 0x7ff828102a30 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.482+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8281a00b0 con 0x7ff828102a30 2024-04-23T18:24:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.482+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8180047a0 con 0x7ff828102a30 2024-04-23T18:24:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.482+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8281a0600 con 0x7ff828102a30 2024-04-23T18:24:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.483+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8280fdf30 con 0x7ff828102a30 2024-04-23T18:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.484+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff818009a30 con 0x7ff828102a30 2024-04-23T18:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.484+0000 7ff824ff9640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 0x7ff8100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.484+0000 7ff826ffd640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 0x7ff8100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.484+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7ff8180cfe60 con 0x7ff828102a30 2024-04-23T18:24:30.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.485+0000 7ff826ffd640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 0x7ff8100a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8281996b0 tx=0x7ff81c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:30.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.489+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff81809c990 con 0x7ff828102a30 2024-04-23T18:24:30.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:30 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:24:30.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:30 smithi106 ceph-mon[32716]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:30.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:30 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:24:30.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:30 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:24:30.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:30 smithi122 ceph-mon[32546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:30.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:30 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:24:30.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:30.719+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff82802d830 con 0x7ff828102a30 2024-04-23T18:24:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:30 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/995281945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-23T18:24:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:30 smithi033 ceph-mon[29274]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:30 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-23T18:24:31.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.313+0000 7ff824ff9640 1 -- 172.21.15.33:0/2266247828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7ff81809c290 con 0x7ff828102a30 2024-04-23T18:24:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 msgr2=0x7ff8100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 0x7ff8100a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff8281996b0 tx=0x7ff81c005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 msgr2=0x7ff8281981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff81800d950 tx=0x7ff81800de10 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 shutdown_connections 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff8100a3bb0 0x7ff8100a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff828107c90 0x7ff82819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff828105360 0x7ff8281986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 --2- 172.21.15.33:0/2266247828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff828102a30 0x7ff8281981a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 >> 172.21.15.33:0/2266247828 conn(0x7ff8280fc440 msgr2=0x7ff8281032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 shutdown_connections 2024-04-23T18:24:31.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.316+0000 7ff82e072640 1 -- 172.21.15.33:0/2266247828 wait complete. 2024-04-23T18:24:31.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 -- 172.21.15.33:0/2579618197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28681053e0 msgr2=0x7f28681077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 --2- 172.21.15.33:0/2579618197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28681053e0 0x7f28681077d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f285c009900 tx=0x7f285c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 -- 172.21.15.33:0/2579618197 shutdown_connections 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 --2- 172.21.15.33:0/2579618197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868107d10 0x7f286810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 --2- 172.21.15.33:0/2579618197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28681053e0 0x7f28681077d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 --2- 172.21.15.33:0/2579618197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2868102ab0 0x7f2868104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 -- 172.21.15.33:0/2579618197 >> 172.21.15.33:0/2579618197 conn(0x7f28680fc400 msgr2=0x7f28680fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.444+0000 7f286e915640 1 -- 172.21.15.33:0/2579618197 shutdown_connections 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 -- 172.21.15.33:0/2579618197 wait complete. 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 Processor -- start 2024-04-23T18:24:31.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 -- start start 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868102ab0 0x7f2868198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 0x7f2868198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f286810daf0 con 0x7f2868107d10 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f286810d970 con 0x7f28681053e0 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.445+0000 7f286e915640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f286810dc70 con 0x7f2868102ab0 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44568/0 (socket says 172.21.15.33:44568) 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 -- 172.21.15.33:0/2948806557 learned_addr learned my addr 172.21.15.33:0/2948806557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f28677fe640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 0x7f2868198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f2867fff640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868102ab0 0x7f2868198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 -- 172.21.15.33:0/2948806557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868102ab0 msgr2=0x7f2868198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868102ab0 0x7f2868198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 -- 172.21.15.33:0/2948806557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 msgr2=0x7f2868198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 0x7f2868198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 -- 172.21.15.33:0/2948806557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286819fe00 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f28677fe640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 0x7f2868198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286ceb2640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f286000cce0 tx=0x7f286000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286000f450 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f286000faa0 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860018b70 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28681a00f0 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.446+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28681a0610 con 0x7f2868107d10 2024-04-23T18:24:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.447+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28680fdef0 con 0x7f2868107d10 2024-04-23T18:24:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.448+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2860018d10 con 0x7f2868107d10 2024-04-23T18:24:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.449+0000 7f28657fa640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 0x7f28500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.449+0000 7f2867fff640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 0x7f28500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.449+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f28600d0600 con 0x7f2868107d10 2024-04-23T18:24:31.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.450+0000 7f2867fff640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 0x7f28500a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2858009bd0 tx=0x7f2858005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:31.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.454+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2860014030 con 0x7f2868107d10 2024-04-23T18:24:31.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: pgmap v288: 97 pgs: 58 active+clean, 28 creating+peering, 11 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-23T18:24:31.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:24:31.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:31 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:31.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:31.681+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f286802d5c0 con 0x7f2868107d10 2024-04-23T18:24:31.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: pgmap v288: 97 pgs: 58 active+clean, 28 creating+peering, 11 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-23T18:24:31.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:24:31.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:31 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: pgmap v288: 97 pgs: 58 active+clean, 28 creating+peering, 11 unknown; 577 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 2 op/s 2024-04-23T18:24:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-23T18:24:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1635279248' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-23T18:24:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-23T18:24:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-23T18:24:31.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-23T18:24:31.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-23T18:24:31.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:31 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-23T18:24:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.325+0000 7f28657fa640 1 -- 172.21.15.33:0/2948806557 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f286009d130 con 0x7f2868107d10 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 msgr2=0x7f28500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 0x7f28500a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2858009bd0 tx=0x7f2858005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 msgr2=0x7f286819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f286000cce0 tx=0x7f286000a620 comp rx=0 tx=0).stop 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 shutdown_connections 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f28500a3c30 0x7f28500a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2868107d10 0x7f286819f790 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28681053e0 0x7f2868198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 --2- 172.21.15.33:0/2948806557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2868102ab0 0x7f2868198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 >> 172.21.15.33:0/2948806557 conn(0x7f28680fc400 msgr2=0x7f28681035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 shutdown_connections 2024-04-23T18:24:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.328+0000 7f286e915640 1 -- 172.21.15.33:0/2948806557 wait complete. 2024-04-23T18:24:32.340 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-23T18:24:32.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-23T18:24:32.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.458+0000 7fc87a231640 1 -- 172.21.15.33:0/1220278854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 msgr2=0x7fc8741012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.458+0000 7fc87a231640 1 --2- 172.21.15.33:0/1220278854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc8741012b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc864009ca0 tx=0x7fc86402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 -- 172.21.15.33:0/1220278854 shutdown_connections 2024-04-23T18:24:32.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 --2- 172.21.15.33:0/1220278854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc8741092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 --2- 172.21.15.33:0/1220278854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 0x7fc874102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 --2- 172.21.15.33:0/1220278854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc8741012b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 -- 172.21.15.33:0/1220278854 >> 172.21.15.33:0/1220278854 conn(0x7fc8740fc400 msgr2=0x7fc8740fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 -- 172.21.15.33:0/1220278854 shutdown_connections 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 -- 172.21.15.33:0/1220278854 wait complete. 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 Processor -- start 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 -- start start 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.459+0000 7fc87a231640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 0x7fc87419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc87419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc87410c9c0 con 0x7fc874100e90 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc87410c840 con 0x7fc8741029a0 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc87410cb40 con 0x7fc874101fe0 2024-04-23T18:24:32.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44588/0 (socket says 172.21.15.33:44588) 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 -- 172.21.15.33:0/286142042 learned_addr learned my addr 172.21.15.33:0/286142042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc873fff640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc87419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc872ffd640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 0x7fc87419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 -- 172.21.15.33:0/286142042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 msgr2=0x7fc87419cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 0x7fc87419cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 -- 172.21.15.33:0/286142042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 msgr2=0x7fc87419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc87419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 -- 172.21.15.33:0/286142042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87419fe30 con 0x7fc874100e90 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc873fff640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc87419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc8737fe640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc864006010 tx=0x7fc864003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86402f380 con 0x7fc874100e90 2024-04-23T18:24:32.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8640317c0 con 0x7fc874100e90 2024-04-23T18:24:32.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8741a00c0 con 0x7fc874100e90 2024-04-23T18:24:32.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86402fc60 con 0x7fc874100e90 2024-04-23T18:24:32.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.460+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8741a0610 con 0x7fc874100e90 2024-04-23T18:24:32.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.461+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc874108310 con 0x7fc874100e90 2024-04-23T18:24:32.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.463+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc864004040 con 0x7fc874100e90 2024-04-23T18:24:32.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.463+0000 7fc870ff9640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 0x7fc8600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.463+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fc8640f1940 con 0x7fc874100e90 2024-04-23T18:24:32.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.463+0000 7fc872ffd640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 0x7fc8600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.464+0000 7fc872ffd640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 0x7fc8600a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc8740699e0 tx=0x7fc85c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:32.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.467+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8640be470 con 0x7fc874100e90 2024-04-23T18:24:32.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:32.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:active 2024-04-23T18:24:32.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:32.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:24:32.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:24:32.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:24:32.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:32 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:32.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.682+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc874062390 con 0x7fc8600a3d50 2024-04-23T18:24:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:active 2024-04-23T18:24:32.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:32.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:24:32.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:24:32.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:24:32.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:32.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2266247828' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-23T18:24:32.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] up:active 2024-04-23T18:24:32.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-23T18:24:32.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-23T18:24:32.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-23T18:24:32.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-23T18:24:32.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:32 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-23T18:24:32.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.708+0000 7fc870ff9640 1 -- 172.21.15.33:0/286142042 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc874062390 con 0x7fc8600a3d50 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.710+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 msgr2=0x7fc8600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.710+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 0x7fc8600a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc8740699e0 tx=0x7fc85c00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 msgr2=0x7fc874069c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc864006010 tx=0x7fc864003ba0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 shutdown_connections 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc8600a3d50 0x7fc8600a6210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8741029a0 0x7fc87419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc874101fe0 0x7fc87419cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 --2- 172.21.15.33:0/286142042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc874100e90 0x7fc874069c70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 >> 172.21.15.33:0/286142042 conn(0x7fc8740fc400 msgr2=0x7fc8741054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:32.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 shutdown_connections 2024-04-23T18:24:32.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.711+0000 7fc87a231640 1 -- 172.21.15.33:0/286142042 wait complete. 2024-04-23T18:24:32.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- 172.21.15.33:0/209075703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70100e90 msgr2=0x7efe701012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 --2- 172.21.15.33:0/209075703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70100e90 0x7efe701012b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009900 tx=0x7efe6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- 172.21.15.33:0/209075703 shutdown_connections 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 --2- 172.21.15.33:0/209075703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 0x7efe701092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 --2- 172.21.15.33:0/209075703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70101fe0 0x7efe70102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 --2- 172.21.15.33:0/209075703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70100e90 0x7efe701012b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- 172.21.15.33:0/209075703 >> 172.21.15.33:0/209075703 conn(0x7efe700fc400 msgr2=0x7efe700fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- 172.21.15.33:0/209075703 shutdown_connections 2024-04-23T18:24:32.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- 172.21.15.33:0/209075703 wait complete. 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 Processor -- start 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 -- start start 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.829+0000 7efe77b29640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70100e90 0x7efe70198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 0x7efe7019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010c9c0 con 0x7efe70101fe0 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010c840 con 0x7efe70100e90 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe7010cb40 con 0x7efe701029a0 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44622/0 (socket says 172.21.15.33:44622) 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 -- 172.21.15.33:0/799754821 learned_addr learned my addr 172.21.15.33:0/799754821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:32.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe758c5640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70100e90 0x7efe70198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe760c6640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 0x7efe7019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 -- 172.21.15.33:0/799754821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 msgr2=0x7efe7019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 0x7efe7019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 -- 172.21.15.33:0/799754821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70100e90 msgr2=0x7efe70198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70100e90 0x7efe70198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 -- 172.21.15.33:0/799754821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7019fe30 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe750c4640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e9c0 tx=0x7efe6000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000cd80 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe600035f0 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60010690 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe701a0120 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.830+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe701a06c0 con 0x7efe70101fe0 2024-04-23T18:24:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.831+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe701a0980 con 0x7efe70101fe0 2024-04-23T18:24:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.832+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efe600037a0 con 0x7efe70101fe0 2024-04-23T18:24:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.833+0000 7efe66ffd640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 0x7efe5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.833+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7efe600d0750 con 0x7efe70101fe0 2024-04-23T18:24:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.833+0000 7efe758c5640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 0x7efe5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.834+0000 7efe758c5640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 0x7efe5c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:32.837+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe6009d280 con 0x7efe70101fe0 2024-04-23T18:24:32.980 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:24:32.686+0000 7f1ebbeb4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:32.980 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:24:32.686+0000 7f1ebbeb4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:32.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:24:32.686+0000 7f1ebbeb4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:32.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:24:32.686+0000 7f1ebbeb4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:32.981 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:24:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:24:32.686+0000 7f1ebbeb4700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:24:33.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.049+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efe700697f0 con 0x7efe5c0a3c30 2024-04-23T18:24:33.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.077+0000 7efe66ffd640 1 -- 172.21.15.33:0/799754821 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe700697f0 con 0x7efe5c0a3c30 2024-04-23T18:24:33.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 msgr2=0x7efe5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 0x7efe5c0a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 msgr2=0x7efe70198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efe6000e9c0 tx=0x7efe6000ee80 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 shutdown_connections 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7efe5c0a3c30 0x7efe5c0a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe701029a0 0x7efe7019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe70101fe0 0x7efe70198750 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 --2- 172.21.15.33:0/799754821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe70100e90 0x7efe70198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 >> 172.21.15.33:0/799754821 conn(0x7efe700fc400 msgr2=0x7efe700690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 shutdown_connections 2024-04-23T18:24:33.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.078+0000 7efe77b29640 1 -- 172.21.15.33:0/799754821 wait complete. 2024-04-23T18:24:33.083 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-23T18:24:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 -- 172.21.15.33:0/2175942293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c103190 msgr2=0x7f3d9c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 --2- 172.21.15.33:0/2175942293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c103190 0x7f3d9c103610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009900 tx=0x7f3d8c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:33.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 -- 172.21.15.33:0/2175942293 shutdown_connections 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 --2- 172.21.15.33:0/2175942293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103b50 0x7f3d9c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 --2- 172.21.15.33:0/2175942293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c103190 0x7f3d9c103610 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 --2- 172.21.15.33:0/2175942293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 -- 172.21.15.33:0/2175942293 >> 172.21.15.33:0/2175942293 conn(0x7f3d9c0fd530 msgr2=0x7f3d9c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 -- 172.21.15.33:0/2175942293 shutdown_connections 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.196+0000 7f3da3659640 1 -- 172.21.15.33:0/2175942293 wait complete. 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 Processor -- start 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 -- start start 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 0x7f3d9c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c103b50 0x7f3d9c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10dae0 con 0x7f3d9c101fa0 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10d960 con 0x7f3d9c103190 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da3659640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c10dc60 con 0x7f3d9c103b50 2024-04-23T18:24:33.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44654/0 (socket says 172.21.15.33:44654) 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 -- 172.21.15.33:0/1721401178 learned_addr learned my addr 172.21.15.33:0/1721401178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da1bf6640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c103b50 0x7f3d9c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da0bf4640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 0x7f3d9c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 -- 172.21.15.33:0/1721401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c103b50 msgr2=0x7f3d9c19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c103b50 0x7f3d9c19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 -- 172.21.15.33:0/1721401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 msgr2=0x7f3d9c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 0x7f3d9c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 -- 172.21.15.33:0/1721401178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c19fdf0 con 0x7f3d9c101fa0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da0bf4640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 0x7f3d9c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.197+0000 7f3da13f5640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d770 tx=0x7f3d9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.198+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98017070 con 0x7f3d9c101fa0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.198+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c1a00e0 con 0x7f3d9c101fa0 2024-04-23T18:24:33.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.198+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d98009c30 con 0x7f3d9c101fa0 2024-04-23T18:24:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.198+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003cc0 con 0x7f3d9c101fa0 2024-04-23T18:24:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.198+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c1a0630 con 0x7f3d9c101fa0 2024-04-23T18:24:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.199+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c109430 con 0x7f3d9c101fa0 2024-04-23T18:24:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.200+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d98003e60 con 0x7f3d9c101fa0 2024-04-23T18:24:33.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.200+0000 7f3d927fc640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 0x7f3d880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.201+0000 7f3da0bf4640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 0x7f3d880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.201+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f3d980d0130 con 0x7f3d9c101fa0 2024-04-23T18:24:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.201+0000 7f3da0bf4640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 0x7f3d880a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c1996e0 tx=0x7f3d8c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.204+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d98009de0 con 0x7f3d9c101fa0 2024-04-23T18:24:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.431+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f3d9c062420 con 0x7f3d880a3bb0 2024-04-23T18:24:33.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.436+0000 7f3d927fc640 1 -- 172.21.15.33:0/1721401178 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f3d9c062420 con 0x7f3d880a3bb0 2024-04-23T18:24:33.430 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/11c33d89-0057-49d5-b660-d78a66220369 2024-04-23T18:24:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 msgr2=0x7f3d880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 0x7f3d880a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c1996e0 tx=0x7f3d8c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 msgr2=0x7f3d9c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800d770 tx=0x7f3d9800dc30 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 shutdown_connections 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3d880a3bb0 0x7f3d880a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d9c103b50 0x7f3d9c19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d9c103190 0x7f3d9c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 --2- 172.21.15.33:0/1721401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d9c101fa0 0x7f3d9c1981d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 >> 172.21.15.33:0/1721401178 conn(0x7f3d9c0fd530 msgr2=0x7f3d9c106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 shutdown_connections 2024-04-23T18:24:33.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.438+0000 7f3da3659640 1 -- 172.21.15.33:0/1721401178 wait complete. 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 11 op/s 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:active 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-23T18:24:33.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:33 smithi033 ceph-mon[29274]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:33.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-23T18:24:33.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 -- 172.21.15.33:0/1654278496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102040 msgr2=0x7f44341024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 --2- 172.21.15.33:0/1654278496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102040 0x7f44341024c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4428009900 tx=0x7f442802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 -- 172.21.15.33:0/1654278496 shutdown_connections 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 --2- 172.21.15.33:0/1654278496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102a00 0x7f4434109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 --2- 172.21.15.33:0/1654278496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102040 0x7f44341024c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 --2- 172.21.15.33:0/1654278496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f4434101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 -- 172.21.15.33:0/1654278496 >> 172.21.15.33:0/1654278496 conn(0x7f44340fc420 msgr2=0x7f44340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 -- 172.21.15.33:0/1654278496 shutdown_connections 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.556+0000 7f443a6fa640 1 -- 172.21.15.33:0/1654278496 wait complete. 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 Processor -- start 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 -- start start 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f443406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102040 0x7f443419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f443410ca10 con 0x7f4434102a00 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f443410c890 con 0x7f4434102040 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f443a6fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f443410cb90 con 0x7f4434100e60 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f4438c97640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f4433fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f443406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f4438c97640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44664/0 (socket says 172.21.15.33:44664) 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f44337fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102040 0x7f443419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f4438c97640 1 -- 172.21.15.33:0/2989126462 learned_addr learned my addr 172.21.15.33:0/2989126462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.557+0000 7f4433fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f443406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:34716/0 (socket says 172.21.15.33:34716) 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 -- 172.21.15.33:0/2989126462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 msgr2=0x7f443406b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f443406b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 -- 172.21.15.33:0/2989126462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102040 msgr2=0x7f443419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102040 0x7f443419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 -- 172.21.15.33:0/2989126462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419fe30 con 0x7f4434102a00 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f4438c97640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c017070 con 0x7f4434102a00 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44341a0120 con 0x7f4434102a00 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c003e20 con 0x7f4434102a00 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c010460 con 0x7f4434102a00 2024-04-23T18:24:33.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.558+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a0640 con 0x7f4434102a00 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.559+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434108270 con 0x7f4434102a00 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.561+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f442c003970 con 0x7f4434102a00 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.561+0000 7f44317fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 0x7f441c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.561+0000 7f4433fff640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 0x7f441c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.561+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f442c0cfea0 con 0x7f4434102a00 2024-04-23T18:24:33.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.562+0000 7f4433fff640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 0x7f441c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4424005fd0 tx=0x7f442400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.565+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f442c09c400 con 0x7f4434102a00 2024-04-23T18:24:33.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 11 op/s 2024-04-23T18:24:33.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:33.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:active 2024-04-23T18:24:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T18:24:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-23T18:24:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-23T18:24:33.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:33 smithi106 ceph-mon[32716]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:33.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: pgmap v289: 97 pgs: 97 active+clean; 580 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 11 op/s 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2948806557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] up:active 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-04-23T18:24:33.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:33 smithi122 ceph-mon[32546]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.778+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4434062390 con 0x7f441c0a3c30 2024-04-23T18:24:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.806+0000 7f44317fa640 1 -- 172.21.15.33:0/2989126462 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4434062390 con 0x7f441c0a3c30 2024-04-23T18:24:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 msgr2=0x7f441c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 0x7f441c0a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4424005fd0 tx=0x7f442400b390 comp rx=0 tx=0).stop 2024-04-23T18:24:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 msgr2=0x7f443419f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c00b260 comp rx=0 tx=0).stop 2024-04-23T18:24:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 shutdown_connections 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f441c0a3c30 0x7f441c0a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4434102a00 0x7f443419f570 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4434102040 0x7f443419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 --2- 172.21.15.33:0/2989126462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100e60 0x7f443406b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 >> 172.21.15.33:0/2989126462 conn(0x7f44340fc420 msgr2=0x7f4434105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 shutdown_connections 2024-04-23T18:24:33.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.808+0000 7f443a6fa640 1 -- 172.21.15.33:0/2989126462 wait complete. 2024-04-23T18:24:33.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.925+0000 7f1b94144640 1 -- 172.21.15.33:0/2376107570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c103b60 msgr2=0x7f1b8c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.925+0000 7f1b94144640 1 --2- 172.21.15.33:0/2376107570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c103b60 0x7f1b8c10a3f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009ca0 tx=0x7f1b8402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- 172.21.15.33:0/2376107570 shutdown_connections 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- 172.21.15.33:0/2376107570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c103b60 0x7f1b8c10a3f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- 172.21.15.33:0/2376107570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c1031a0 0x7f1b8c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- 172.21.15.33:0/2376107570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- 172.21.15.33:0/2376107570 >> 172.21.15.33:0/2376107570 conn(0x7f1b8c0fd520 msgr2=0x7f1b8c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- 172.21.15.33:0/2376107570 shutdown_connections 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- 172.21.15.33:0/2376107570 wait complete. 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 Processor -- start 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- start start 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c1031a0 0x7f1b8c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 0x7f1b8c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10daf0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10d970 con 0x7f1b8c1031a0 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b94144640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8c10dc70 con 0x7f1b8c103b60 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.926+0000 7f1b91ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44684/0 (socket says 172.21.15.33:44684) 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 -- 172.21.15.33:0/905020388 learned_addr learned my addr 172.21.15.33:0/905020388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b916df640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c1031a0 0x7f1b8c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b926e1640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 0x7f1b8c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 -- 172.21.15.33:0/905020388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 msgr2=0x7f1b8c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 0x7f1b8c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 -- 172.21.15.33:0/905020388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c1031a0 msgr2=0x7f1b8c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c1031a0 0x7f1b8c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 -- 172.21.15.33:0/905020388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b8c075110 con 0x7f1b8c101fb0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b926e1640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 0x7f1b8c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b91ee0640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e9c0 tx=0x7f1b8800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800ccc0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b880035f0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b8c1a21e0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800ce60 con 0x7f1b8c101fb0 2024-04-23T18:24:33.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.927+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b8c1a2780 con 0x7f1b8c101fb0 2024-04-23T18:24:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.928+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8c109270 con 0x7f1b8c101fb0 2024-04-23T18:24:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.929+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b880037a0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.930+0000 7f1b82ffd640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 0x7f1b780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.930+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1b880d07b0 con 0x7f1b8c101fb0 2024-04-23T18:24:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.930+0000 7f1b916df640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 0x7f1b780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.930+0000 7f1b916df640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 0x7f1b780a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c075a70 tx=0x7f1b7c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:33.935+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b8809d240 con 0x7f1b8c101fb0 2024-04-23T18:24:34.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.151+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1b8c062420 con 0x7f1b780a3c30 2024-04-23T18:24:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.156+0000 7f1b82ffd640 1 -- 172.21.15.33:0/905020388 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1b8c062420 con 0x7f1b780a3c30 2024-04-23T18:24:34.150 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/a5f2bf8e-f395-4df6-b53a-df0570b88564 2024-04-23T18:24:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 msgr2=0x7f1b780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 0x7f1b780a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c075a70 tx=0x7f1b7c00a2d0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 msgr2=0x7f1b8c075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e9c0 tx=0x7f1b8800edc0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 shutdown_connections 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1b780a3c30 0x7f1b780a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1b8c103b60 0x7f1b8c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b8c1031a0 0x7f1b8c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 --2- 172.21.15.33:0/905020388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b8c101fb0 0x7f1b8c075d00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 >> 172.21.15.33:0/905020388 conn(0x7f1b8c0fd520 msgr2=0x7f1b8c106440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 shutdown_connections 2024-04-23T18:24:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.158+0000 7f1b94144640 1 -- 172.21.15.33:0/905020388 wait complete. 2024-04-23T18:24:34.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 -- 172.21.15.33:0/4277956419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 msgr2=0x7f58d01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 --2- 172.21.15.33:0/4277956419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d01077d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009900 tx=0x7f58c002a410 comp rx=0 tx=0).stop 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 -- 172.21.15.33:0/4277956419 shutdown_connections 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 --2- 172.21.15.33:0/4277956419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0107d10 0x7f58d010a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 --2- 172.21.15.33:0/4277956419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d01077d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 --2- 172.21.15.33:0/4277956419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0102ab0 0x7f58d0104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.270+0000 7f58d87d8640 1 -- 172.21.15.33:0/4277956419 >> 172.21.15.33:0/4277956419 conn(0x7f58d00fc400 msgr2=0x7f58d00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- 172.21.15.33:0/4277956419 shutdown_connections 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- 172.21.15.33:0/4277956419 wait complete. 2024-04-23T18:24:34.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 Processor -- start 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- start start 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0102ab0 0x7f58d0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0107d10 0x7f58d019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010daf0 con 0x7f58d01053e0 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010d970 con 0x7f58d0102ab0 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d87d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58d010dc70 con 0x7f58d0107d10 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d5d73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d5d73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44696/0 (socket says 172.21.15.33:44696) 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d5d73640 1 -- 172.21.15.33:0/2630926046 learned_addr learned my addr 172.21.15.33:0/2630926046 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.271+0000 7f58d6574640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0102ab0 0x7f58d0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d6d75640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0107d10 0x7f58d019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 -- 172.21.15.33:0/2630926046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0107d10 msgr2=0x7f58d019f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0107d10 0x7f58d019f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 -- 172.21.15.33:0/2630926046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0102ab0 msgr2=0x7f58d0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0102ab0 0x7f58d0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 -- 172.21.15.33:0/2630926046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d019fe50 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d5d73640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006010 tx=0x7f58c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c003f030 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01a00e0 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58c00318f0 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0035070 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.272+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01a05d0 con 0x7f58d01053e0 2024-04-23T18:24:34.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.273+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d00fdef0 con 0x7f58d01053e0 2024-04-23T18:24:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.274+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f58c0042030 con 0x7f58d01053e0 2024-04-23T18:24:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.275+0000 7f58c77fe640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 0x7f58bc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.275+0000 7f58d6574640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 0x7f58bc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.275+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f58c00f0ca0 con 0x7f58d01053e0 2024-04-23T18:24:34.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.275+0000 7f58d6574640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 0x7f58bc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.278+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58c00bd730 con 0x7f58d01053e0 2024-04-23T18:24:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.354+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58c00b96a0 con 0x7f58d01053e0 2024-04-23T18:24:34.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:active 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: Cluster is now healthy 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:34 smithi033 ceph-mon[29274]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.513+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d002d810 con 0x7f58d01053e0 2024-04-23T18:24:34.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.514+0000 7f58c77fe640 1 -- 172.21.15.33:0/2630926046 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7f58c00b7400 con 0x7f58d01053e0 2024-04-23T18:24:34.508 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:34.508 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:24:34.508 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 msgr2=0x7f58bc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 0x7f58bc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc00b390 comp rx=0 tx=0).stop 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 msgr2=0x7f58d0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f58c0006010 tx=0x7f58c00042f0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 shutdown_connections 2024-04-23T18:24:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58bc0a3bb0 0x7f58bc0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d0107d10 0x7f58d019f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58d01053e0 0x7f58d0198770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 --2- 172.21.15.33:0/2630926046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58d0102ab0 0x7f58d0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 >> 172.21.15.33:0/2630926046 conn(0x7f58d00fc400 msgr2=0x7f58d0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 shutdown_connections 2024-04-23T18:24:34.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.516+0000 7f58d87d8640 1 -- 172.21.15.33:0/2630926046 wait complete. 2024-04-23T18:24:34.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:24:34.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:active 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: Cluster is now healthy 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:34 smithi106 ceph-mon[32716]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 -- 172.21.15.33:0/4170290786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102040 msgr2=0x7f5f781024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 --2- 172.21.15.33:0/4170290786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102040 0x7f5f781024c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009900 tx=0x7f5f6802a410 comp rx=0 tx=0).stop 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 -- 172.21.15.33:0/4170290786 shutdown_connections 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 --2- 172.21.15.33:0/4170290786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78102a00 0x7f5f78109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 --2- 172.21.15.33:0/4170290786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102040 0x7f5f781024c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 --2- 172.21.15.33:0/4170290786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78100e60 0x7f5f78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 -- 172.21.15.33:0/4170290786 >> 172.21.15.33:0/4170290786 conn(0x7f5f780fc440 msgr2=0x7f5f780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 -- 172.21.15.33:0/4170290786 shutdown_connections 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 -- 172.21.15.33:0/4170290786 wait complete. 2024-04-23T18:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.629+0000 7f5f801e1640 1 Processor -- start 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 -- start start 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78100e60 0x7f5f7806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 0x7f5f7819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810ca00 con 0x7f5f78102a00 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810c880 con 0x7f5f78102040 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f801e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7810cb80 con 0x7f5f78100e60 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44712/0 (socket says 172.21.15.33:44712) 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 -- 172.21.15.33:0/1483588697 learned_addr learned my addr 172.21.15.33:0/1483588697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7d77c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 0x7f5f7819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 -- 172.21.15.33:0/1483588697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78100e60 msgr2=0x7f5f7806b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7df7d640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78100e60 0x7f5f7806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78100e60 0x7f5f7806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 -- 172.21.15.33:0/1483588697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 msgr2=0x7f5f7819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 0x7f5f7819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 -- 172.21.15.33:0/1483588697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f7819fe80 con 0x7f5f78102a00 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7d77c640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 0x7f5f7819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f7e77e640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000cad0 tx=0x7f5f7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.630+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000ae20 con 0x7f5f78102a00 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.631+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f70005020 con 0x7f5f78102a00 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.631+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f781a0110 con 0x7f5f78102a00 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.631+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000f6b0 con 0x7f5f78102a00 2024-04-23T18:24:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.631+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f781a0680 con 0x7f5f78102a00 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.631+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f78108280 con 0x7f5f78102a00 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.632+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5f700051d0 con 0x7f5f78102a00 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.633+0000 7f5f6effd640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 0x7f5f640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.633+0000 7f5f7df7d640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 0x7f5f640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.633+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f5f700d0390 con 0x7f5f78102a00 2024-04-23T18:24:34.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.634+0000 7f5f7df7d640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 0x7f5f640a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101bb0 tx=0x7f5f74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.638+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f70014030 con 0x7f5f78102a00 2024-04-23T18:24:34.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] up:active 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: Cluster is now healthy 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:34 smithi122 ceph-mon[32546]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:34.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.826+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f7802d830 con 0x7f5f78102a00 2024-04-23T18:24:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.827+0000 7f5f6effd640 1 -- 172.21.15.33:0/1483588697 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7f5f7009cdc0 con 0x7f5f78102a00 2024-04-23T18:24:34.822 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:34.822 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:24:27.868758+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 msgr2=0x7f5f640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 0x7f5f640a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5f78101bb0 tx=0x7f5f74005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 msgr2=0x7f5f7819f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000cad0 tx=0x7f5f7000cf90 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 shutdown_connections 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5f640a3cc0 0x7f5f640a6180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f78102a00 0x7f5f7819f570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f78102040 0x7f5f7819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 --2- 172.21.15.33:0/1483588697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f78100e60 0x7f5f7806b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.829+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 >> 172.21.15.33:0/1483588697 conn(0x7f5f780fc440 msgr2=0x7f5f78105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.830+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 shutdown_connections 2024-04-23T18:24:34.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.830+0000 7f5f801e1640 1 -- 172.21.15.33:0/1483588697 wait complete. 2024-04-23T18:24:34.835 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-23T18:24:34.835 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 -- 172.21.15.33:0/3161354567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558103b70 msgr2=0x7f755810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 --2- 172.21.15.33:0/3161354567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558103b70 0x7f755810a400 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7550009ca0 tx=0x7f755002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 -- 172.21.15.33:0/3161354567 shutdown_connections 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 --2- 172.21.15.33:0/3161354567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558103b70 0x7f755810a400 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 --2- 172.21.15.33:0/3161354567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 0x7f7558103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 --2- 172.21.15.33:0/3161354567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558101fc0 0x7f75581023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 -- 172.21.15.33:0/3161354567 >> 172.21.15.33:0/3161354567 conn(0x7f75580fd550 msgr2=0x7f75580ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 -- 172.21.15.33:0/3161354567 shutdown_connections 2024-04-23T18:24:34.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.946+0000 7f755e730640 1 -- 172.21.15.33:0/3161354567 wait complete. 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 Processor -- start 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 -- start start 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 0x7f7558074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 0x7f7558074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f755810db00 con 0x7f7558101fc0 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f755810d980 con 0x7f75581031b0 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755e730640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f755810dc80 con 0x7f7558103b70 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f7557fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f7557fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44728/0 (socket says 172.21.15.33:44728) 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f7557fff640 1 -- 172.21.15.33:0/4040571028 learned_addr learned my addr 172.21.15.33:0/4040571028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f75577fe640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 0x7f7558074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.947+0000 7f755cccd640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 0x7f7558074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 -- 172.21.15.33:0/4040571028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 msgr2=0x7f7558074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 0x7f7558074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 -- 172.21.15.33:0/4040571028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 msgr2=0x7f7558074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 0x7f7558074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 -- 172.21.15.33:0/4040571028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7558075110 con 0x7f7558101fc0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f755cccd640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 0x7f7558074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f7557fff640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f754800d950 tx=0x7f754800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548017070 con 0x7f7558101fc0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7548009c30 con 0x7f7558101fc0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75581a22a0 con 0x7f7558101fc0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754800f620 con 0x7f7558101fc0 2024-04-23T18:24:34.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.948+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75581a27a0 con 0x7f7558101fc0 2024-04-23T18:24:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.949+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75581093b0 con 0x7f7558101fc0 2024-04-23T18:24:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.950+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f754801a070 con 0x7f7558101fc0 2024-04-23T18:24:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.951+0000 7f75557fa640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.951+0000 7f75577fe640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:34.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.951+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f75480cfe20 con 0x7f7558101fc0 2024-04-23T18:24:34.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.951+0000 7f75577fe640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 0x7f75400a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7558075a70 tx=0x7f754c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:34.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:34.956+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7548009de0 con 0x7f7558101fc0 2024-04-23T18:24:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.138+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7558062390 con 0x7f75400a3c40 2024-04-23T18:24:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.146+0000 7f75557fa640 1 -- 172.21.15.33:0/4040571028 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258927 (secure 0 0 0) 0x7f7558062390 con 0x7f75400a3c40 2024-04-23T18:24:35.143 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:24:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 msgr2=0x7f75400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 0x7f75400a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7558075a70 tx=0x7f754c005f90 comp rx=0 tx=0).stop 2024-04-23T18:24:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 msgr2=0x7f7558075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f754800d950 tx=0x7f754800de10 comp rx=0 tx=0).stop 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 shutdown_connections 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f75400a3c40 0x7f75400a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7558103b70 0x7f7558074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75581031b0 0x7f7558074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 --2- 172.21.15.33:0/4040571028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7558101fc0 0x7f7558075d00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 >> 172.21.15.33:0/4040571028 conn(0x7f75580fd550 msgr2=0x7f7558106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 shutdown_connections 2024-04-23T18:24:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.149+0000 7f755e730640 1 -- 172.21.15.33:0/4040571028 wait complete. 2024-04-23T18:24:35.154 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-23T18:24:34.150819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596212,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":316,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":269,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381048,"kb_used_data":11068,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560776,"statfs":{"total":1151940427776,"available":1151550234624,"internally_reserved":0,"allocated":11333632,"data_stored":7053511,"data_compressed":107010,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5844,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.878791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183163+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:24:28.183163+0000","last_peered":"2024-04-23T18:24:28.183163+0000","last_clean":"2024-04-23T18:24:28.183163+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:24:28.183163+0000","last_undegraded":"2024-04-23T18:24:28.183163+0000","last_fullsized":"2024-04-23T18:24:28.183163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182969+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:24:28.182969+0000","last_peered":"2024-04-23T18:24:28.182969+0000","last_clean":"2024-04-23T18:24:28.182969+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:24:28.182969+0000","last_undegraded":"2024-04-23T18:24:28.182969+0000","last_fullsized":"2024-04-23T18:24:28.182969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182744+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:24:28.182744+0000","last_peered":"2024-04-23T18:24:28.182744+0000","last_clean":"2024-04-23T18:24:28.182744+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:24:28.182744+0000","last_undegraded":"2024-04-23T18:24:28.182744+0000","last_fullsized":"2024-04-23T18:24:28.182744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182882+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:24:28.182882+0000","last_peered":"2024-04-23T18:24:28.182882+0000","last_clean":"2024-04-23T18:24:28.182882+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:24:28.182882+0000","last_undegraded":"2024-04-23T18:24:28.182882+0000","last_fullsized":"2024-04-23T18:24:28.182882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182447+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:24:28.182447+0000","last_peered":"2024-04-23T18:24:28.182447+0000","last_clean":"2024-04-23T18:24:28.182447+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:24:28.182447+0000","last_undegraded":"2024-04-23T18:24:28.182447+0000","last_fullsized":"2024-04-23T18:24:28.182447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182707+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:24:28.182707+0000","last_peered":"2024-04-23T18:24:28.182707+0000","last_clean":"2024-04-23T18:24:28.182707+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:24:28.182707+0000","last_undegraded":"2024-04-23T18:24:28.182707+0000","last_fullsized":"2024-04-23T18:24:28.182707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264088+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:24:28.264088+0000","last_peered":"2024-04-23T18:24:28.264088+0000","last_clean":"2024-04-23T18:24:28.264088+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:24:28.264088+0000","last_undegraded":"2024-04-23T18:24:28.264088+0000","last_fullsized":"2024-04-23T18:24:28.264088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182643+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:24:28.182643+0000","last_peered":"2024-04-23T18:24:28.182643+0000","last_clean":"2024-04-23T18:24:28.182643+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:24:28.182643+0000","last_undegraded":"2024-04-23T18:24:28.182643+0000","last_fullsized":"2024-04-23T18:24:28.182643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874419+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:24:27.874419+0000","last_peered":"2024-04-23T18:24:27.874419+0000","last_clean":"2024-04-23T18:24:27.874419+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:24:27.874419+0000","last_undegraded":"2024-04-23T18:24:27.874419+0000","last_fullsized":"2024-04-23T18:24:27.874419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183060+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:24:28.183060+0000","last_peered":"2024-04-23T18:24:28.183060+0000","last_clean":"2024-04-23T18:24:28.183060+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:24:28.183060+0000","last_undegraded":"2024-04-23T18:24:28.183060+0000","last_fullsized":"2024-04-23T18:24:28.183060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264332+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:24:28.264332+0000","last_peered":"2024-04-23T18:24:28.264332+0000","last_clean":"2024-04-23T18:24:28.264332+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:24:28.264332+0000","last_undegraded":"2024-04-23T18:24:28.264332+0000","last_fullsized":"2024-04-23T18:24:28.264332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182922+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:24:28.182922+0000","last_peered":"2024-04-23T18:24:28.182922+0000","last_clean":"2024-04-23T18:24:28.182922+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:24:28.182922+0000","last_undegraded":"2024-04-23T18:24:28.182922+0000","last_fullsized":"2024-04-23T18:24:28.182922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872464+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:24:27.872464+0000","last_peered":"2024-04-23T18:24:27.872464+0000","last_clean":"2024-04-23T18:24:27.872464+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:24:27.872464+0000","last_undegraded":"2024-04-23T18:24:27.872464+0000","last_fullsized":"2024-04-23T18:24:27.872464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182334+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:24:28.182334+0000","last_peered":"2024-04-23T18:24:28.182334+0000","last_clean":"2024-04-23T18:24:28.182334+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:24:28.182334+0000","last_undegraded":"2024-04-23T18:24:28.182334+0000","last_fullsized":"2024-04-23T18:24:28.182334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182199+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:24:28.182199+0000","last_peered":"2024-04-23T18:24:28.182199+0000","last_clean":"2024-04-23T18:24:28.182199+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:24:28.182199+0000","last_undegraded":"2024-04-23T18:24:28.182199+0000","last_fullsized":"2024-04-23T18:24:28.182199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.873962+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:24:27.873962+0000","last_peered":"2024-04-23T18:24:27.873962+0000","last_clean":"2024-04-23T18:24:27.873962+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:24:27.873962+0000","last_undegraded":"2024-04-23T18:24:27.873962+0000","last_fullsized":"2024-04-23T18:24:27.873962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183372+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:24:28.183372+0000","last_peered":"2024-04-23T18:24:28.183372+0000","last_clean":"2024-04-23T18:24:28.183372+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:24:28.183372+0000","last_undegraded":"2024-04-23T18:24:28.183372+0000","last_fullsized":"2024-04-23T18:24:28.183372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183305+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:24:28.183305+0000","last_peered":"2024-04-23T18:24:28.183305+0000","last_clean":"2024-04-23T18:24:28.183305+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:24:28.183305+0000","last_undegraded":"2024-04-23T18:24:28.183305+0000","last_fullsized":"2024-04-23T18:24:28.183305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872646+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:24:27.872646+0000","last_peered":"2024-04-23T18:24:27.872646+0000","last_clean":"2024-04-23T18:24:27.872646+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:24:27.872646+0000","last_undegraded":"2024-04-23T18:24:27.872646+0000","last_fullsized":"2024-04-23T18:24:27.872646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:33.348597+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:24:33.348597+0000","last_peered":"2024-04-23T18:24:33.348597+0000","last_clean":"2024-04-23T18:24:33.348597+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:24:33.348597+0000","last_undegraded":"2024-04-23T18:24:33.348597+0000","last_fullsized":"2024-04-23T18:24:33.348597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182801+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:24:28.182801+0000","last_peered":"2024-04-23T18:24:28.182801+0000","last_clean":"2024-04-23T18:24:28.182801+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:24:28.182801+0000","last_undegraded":"2024-04-23T18:24:28.182801+0000","last_fullsized":"2024-04-23T18:24:28.182801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183137+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:24:28.183137+0000","last_peered":"2024-04-23T18:24:28.183137+0000","last_clean":"2024-04-23T18:24:28.183137+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:24:28.183137+0000","last_undegraded":"2024-04-23T18:24:28.183137+0000","last_fullsized":"2024-04-23T18:24:28.183137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182930+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:24:28.182930+0000","last_peered":"2024-04-23T18:24:28.182930+0000","last_clean":"2024-04-23T18:24:28.182930+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:24:28.182930+0000","last_undegraded":"2024-04-23T18:24:28.182930+0000","last_fullsized":"2024-04-23T18:24:28.182930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183421+0000","last_change":"2024-04-23T18:24:26.289042+0000","last_active":"2024-04-23T18:24:28.183421+0000","last_peered":"2024-04-23T18:24:28.183421+0000","last_clean":"2024-04-23T18:24:28.183421+0000","last_became_active":"2024-04-23T18:24:26.288711+0000","last_became_peered":"2024-04-23T18:24:26.288711+0000","last_unstale":"2024-04-23T18:24:28.183421+0000","last_undegraded":"2024-04-23T18:24:28.183421+0000","last_fullsized":"2024-04-23T18:24:28.183421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:24.914645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874355+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:24:27.874355+0000","last_peered":"2024-04-23T18:24:27.874355+0000","last_clean":"2024-04-23T18:24:27.874355+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:24:27.874355+0000","last_undegraded":"2024-04-23T18:24:27.874355+0000","last_fullsized":"2024-04-23T18:24:27.874355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182880+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:24:28.182880+0000","last_peered":"2024-04-23T18:24:28.182880+0000","last_clean":"2024-04-23T18:24:28.182880+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:24:28.182880+0000","last_undegraded":"2024-04-23T18:24:28.182880+0000","last_fullsized":"2024-04-23T18:24:28.182880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183334+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:24:28.183334+0000","last_peered":"2024-04-23T18:24:28.183334+0000","last_clean":"2024-04-23T18:24:28.183334+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:24:28.183334+0000","last_undegraded":"2024-04-23T18:24:28.183334+0000","last_fullsized":"2024-04-23T18:24:28.183334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.326018+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:24:30.326018+0000","last_peered":"2024-04-23T18:24:30.326018+0000","last_clean":"2024-04-23T18:24:30.326018+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:24:30.326018+0000","last_undegraded":"2024-04-23T18:24:30.326018+0000","last_fullsized":"2024-04-23T18:24:30.326018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183267+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:24:28.183267+0000","last_peered":"2024-04-23T18:24:28.183267+0000","last_clean":"2024-04-23T18:24:28.183267+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:24:28.183267+0000","last_undegraded":"2024-04-23T18:24:28.183267+0000","last_fullsized":"2024-04-23T18:24:28.183267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182473+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:24:28.182473+0000","last_peered":"2024-04-23T18:24:28.182473+0000","last_clean":"2024-04-23T18:24:28.182473+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:24:28.182473+0000","last_undegraded":"2024-04-23T18:24:28.182473+0000","last_fullsized":"2024-04-23T18:24:28.182473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182826+0000","last_change":"2024-04-23T18:24:26.894333+0000","last_active":"2024-04-23T18:24:28.182826+0000","last_peered":"2024-04-23T18:24:28.182826+0000","last_clean":"2024-04-23T18:24:28.182826+0000","last_became_active":"2024-04-23T18:24:26.893843+0000","last_became_peered":"2024-04-23T18:24:26.893843+0000","last_unstale":"2024-04-23T18:24:28.182826+0000","last_undegraded":"2024-04-23T18:24:28.182826+0000","last_fullsized":"2024-04-23T18:24:28.182826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:44.102339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.333566+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:24:31.333566+0000","last_peered":"2024-04-23T18:24:31.333566+0000","last_clean":"2024-04-23T18:24:31.333566+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:24:31.333566+0000","last_undegraded":"2024-04-23T18:24:31.333566+0000","last_fullsized":"2024-04-23T18:24:31.333566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182490+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:24:28.182490+0000","last_peered":"2024-04-23T18:24:28.182490+0000","last_clean":"2024-04-23T18:24:28.182490+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:24:28.182490+0000","last_undegraded":"2024-04-23T18:24:28.182490+0000","last_fullsized":"2024-04-23T18:24:28.182490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182245+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:24:28.182245+0000","last_peered":"2024-04-23T18:24:28.182245+0000","last_clean":"2024-04-23T18:24:28.182245+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:24:28.182245+0000","last_undegraded":"2024-04-23T18:24:28.182245+0000","last_fullsized":"2024-04-23T18:24:28.182245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182645+0000","last_change":"2024-04-23T18:24:26.891966+0000","last_active":"2024-04-23T18:24:28.182645+0000","last_peered":"2024-04-23T18:24:28.182645+0000","last_clean":"2024-04-23T18:24:28.182645+0000","last_became_active":"2024-04-23T18:24:26.891579+0000","last_became_peered":"2024-04-23T18:24:26.891579+0000","last_unstale":"2024-04-23T18:24:28.182645+0000","last_undegraded":"2024-04-23T18:24:28.182645+0000","last_fullsized":"2024-04-23T18:24:28.182645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:10:18.633162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.324772+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:24:30.324772+0000","last_peered":"2024-04-23T18:24:30.324772+0000","last_clean":"2024-04-23T18:24:30.324772+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:24:30.324772+0000","last_undegraded":"2024-04-23T18:24:30.324772+0000","last_fullsized":"2024-04-23T18:24:30.324772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874334+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:24:27.874334+0000","last_peered":"2024-04-23T18:24:27.874334+0000","last_clean":"2024-04-23T18:24:27.874334+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:24:27.874334+0000","last_undegraded":"2024-04-23T18:24:27.874334+0000","last_fullsized":"2024-04-23T18:24:27.874334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.326117+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:24:30.326117+0000","last_peered":"2024-04-23T18:24:30.326117+0000","last_clean":"2024-04-23T18:24:30.326117+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:24:30.326117+0000","last_undegraded":"2024-04-23T18:24:30.326117+0000","last_fullsized":"2024-04-23T18:24:30.326117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182755+0000","last_change":"2024-04-23T18:24:26.892107+0000","last_active":"2024-04-23T18:24:28.182755+0000","last_peered":"2024-04-23T18:24:28.182755+0000","last_clean":"2024-04-23T18:24:28.182755+0000","last_became_active":"2024-04-23T18:24:26.891805+0000","last_became_peered":"2024-04-23T18:24:26.891805+0000","last_unstale":"2024-04-23T18:24:28.182755+0000","last_undegraded":"2024-04-23T18:24:28.182755+0000","last_fullsized":"2024-04-23T18:24:28.182755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:20:06.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183496+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:24:28.183496+0000","last_peered":"2024-04-23T18:24:28.183496+0000","last_clean":"2024-04-23T18:24:28.183496+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:24:28.183496+0000","last_undegraded":"2024-04-23T18:24:28.183496+0000","last_fullsized":"2024-04-23T18:24:28.183496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874315+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:24:27.874315+0000","last_peered":"2024-04-23T18:24:27.874315+0000","last_clean":"2024-04-23T18:24:27.874315+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:24:27.874315+0000","last_undegraded":"2024-04-23T18:24:27.874315+0000","last_fullsized":"2024-04-23T18:24:27.874315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325794+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:24:30.325794+0000","last_peered":"2024-04-23T18:24:30.325794+0000","last_clean":"2024-04-23T18:24:30.325794+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:24:30.325794+0000","last_undegraded":"2024-04-23T18:24:30.325794+0000","last_fullsized":"2024-04-23T18:24:30.325794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183129+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:24:28.183129+0000","last_peered":"2024-04-23T18:24:28.183129+0000","last_clean":"2024-04-23T18:24:28.183129+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:24:28.183129+0000","last_undegraded":"2024-04-23T18:24:28.183129+0000","last_fullsized":"2024-04-23T18:24:28.183129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.332935+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:24:31.332935+0000","last_peered":"2024-04-23T18:24:31.332935+0000","last_clean":"2024-04-23T18:24:31.332935+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:24:31.332935+0000","last_undegraded":"2024-04-23T18:24:31.332935+0000","last_fullsized":"2024-04-23T18:24:31.332935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872449+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:24:27.872449+0000","last_peered":"2024-04-23T18:24:27.872449+0000","last_clean":"2024-04-23T18:24:27.872449+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:24:27.872449+0000","last_undegraded":"2024-04-23T18:24:27.872449+0000","last_fullsized":"2024-04-23T18:24:27.872449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325656+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:24:30.325656+0000","last_peered":"2024-04-23T18:24:30.325656+0000","last_clean":"2024-04-23T18:24:30.325656+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:24:30.325656+0000","last_undegraded":"2024-04-23T18:24:30.325656+0000","last_fullsized":"2024-04-23T18:24:30.325656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264323+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:24:28.264323+0000","last_peered":"2024-04-23T18:24:28.264323+0000","last_clean":"2024-04-23T18:24:28.264323+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:24:28.264323+0000","last_undegraded":"2024-04-23T18:24:28.264323+0000","last_fullsized":"2024-04-23T18:24:28.264323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.339822+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:24:32.339822+0000","last_peered":"2024-04-23T18:24:32.339822+0000","last_clean":"2024-04-23T18:24:32.339822+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:24:32.339822+0000","last_undegraded":"2024-04-23T18:24:32.339822+0000","last_fullsized":"2024-04-23T18:24:32.339822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.263975+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:24:28.263975+0000","last_peered":"2024-04-23T18:24:28.263975+0000","last_clean":"2024-04-23T18:24:28.263975+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:24:28.263975+0000","last_undegraded":"2024-04-23T18:24:28.263975+0000","last_fullsized":"2024-04-23T18:24:28.263975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325235+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:24:30.325235+0000","last_peered":"2024-04-23T18:24:30.325235+0000","last_clean":"2024-04-23T18:24:30.325235+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:24:30.325235+0000","last_undegraded":"2024-04-23T18:24:30.325235+0000","last_fullsized":"2024-04-23T18:24:30.325235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182405+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:24:28.182405+0000","last_peered":"2024-04-23T18:24:28.182405+0000","last_clean":"2024-04-23T18:24:28.182405+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:24:28.182405+0000","last_undegraded":"2024-04-23T18:24:28.182405+0000","last_fullsized":"2024-04-23T18:24:28.182405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182463+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:24:28.182463+0000","last_peered":"2024-04-23T18:24:28.182463+0000","last_clean":"2024-04-23T18:24:28.182463+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:24:28.182463+0000","last_undegraded":"2024-04-23T18:24:28.182463+0000","last_fullsized":"2024-04-23T18:24:28.182463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183305+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:24:28.183305+0000","last_peered":"2024-04-23T18:24:28.183305+0000","last_clean":"2024-04-23T18:24:28.183305+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:24:28.183305+0000","last_undegraded":"2024-04-23T18:24:28.183305+0000","last_fullsized":"2024-04-23T18:24:28.183305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.762107+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:24:28.762107+0000","last_peered":"2024-04-23T18:24:28.762107+0000","last_clean":"2024-04-23T18:24:28.762107+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:24:28.762107+0000","last_undegraded":"2024-04-23T18:24:28.762107+0000","last_fullsized":"2024-04-23T18:24:28.762107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183054+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:24:28.183054+0000","last_peered":"2024-04-23T18:24:28.183054+0000","last_clean":"2024-04-23T18:24:28.183054+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:24:28.183054+0000","last_undegraded":"2024-04-23T18:24:28.183054+0000","last_fullsized":"2024-04-23T18:24:28.183054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264190+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:24:28.264190+0000","last_peered":"2024-04-23T18:24:28.264190+0000","last_clean":"2024-04-23T18:24:28.264190+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:24:28.264190+0000","last_undegraded":"2024-04-23T18:24:28.264190+0000","last_fullsized":"2024-04-23T18:24:28.264190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183246+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:24:28.183246+0000","last_peered":"2024-04-23T18:24:28.183246+0000","last_clean":"2024-04-23T18:24:28.183246+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:24:28.183246+0000","last_undegraded":"2024-04-23T18:24:28.183246+0000","last_fullsized":"2024-04-23T18:24:28.183246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182484+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:24:28.182484+0000","last_peered":"2024-04-23T18:24:28.182484+0000","last_clean":"2024-04-23T18:24:28.182484+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:24:28.182484+0000","last_undegraded":"2024-04-23T18:24:28.182484+0000","last_fullsized":"2024-04-23T18:24:28.182484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.332410+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:24:31.332410+0000","last_peered":"2024-04-23T18:24:31.332410+0000","last_clean":"2024-04-23T18:24:31.332410+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:24:31.332410+0000","last_undegraded":"2024-04-23T18:24:31.332410+0000","last_fullsized":"2024-04-23T18:24:31.332410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183728+0000","last_change":"2024-04-23T18:24:26.899273+0000","last_active":"2024-04-23T18:24:28.183728+0000","last_peered":"2024-04-23T18:24:28.183728+0000","last_clean":"2024-04-23T18:24:28.183728+0000","last_became_active":"2024-04-23T18:24:26.898967+0000","last_became_peered":"2024-04-23T18:24:26.898967+0000","last_unstale":"2024-04-23T18:24:28.183728+0000","last_undegraded":"2024-04-23T18:24:28.183728+0000","last_fullsized":"2024-04-23T18:24:28.183728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:03:30.309041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.339411+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:24:32.339411+0000","last_peered":"2024-04-23T18:24:32.339411+0000","last_clean":"2024-04-23T18:24:32.339411+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:24:32.339411+0000","last_undegraded":"2024-04-23T18:24:32.339411+0000","last_fullsized":"2024-04-23T18:24:32.339411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.263946+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:24:28.263946+0000","last_peered":"2024-04-23T18:24:28.263946+0000","last_clean":"2024-04-23T18:24:28.263946+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:24:28.263946+0000","last_undegraded":"2024-04-23T18:24:28.263946+0000","last_fullsized":"2024-04-23T18:24:28.263946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872112+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:24:27.872112+0000","last_peered":"2024-04-23T18:24:27.872112+0000","last_clean":"2024-04-23T18:24:27.872112+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:24:27.872112+0000","last_undegraded":"2024-04-23T18:24:27.872112+0000","last_fullsized":"2024-04-23T18:24:27.872112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.330445+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:24:31.330445+0000","last_peered":"2024-04-23T18:24:31.330445+0000","last_clean":"2024-04-23T18:24:31.330445+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:24:31.330445+0000","last_undegraded":"2024-04-23T18:24:31.330445+0000","last_fullsized":"2024-04-23T18:24:31.330445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183288+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:24:28.183288+0000","last_peered":"2024-04-23T18:24:28.183288+0000","last_clean":"2024-04-23T18:24:28.183288+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:24:28.183288+0000","last_undegraded":"2024-04-23T18:24:28.183288+0000","last_fullsized":"2024-04-23T18:24:28.183288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872370+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:24:27.872370+0000","last_peered":"2024-04-23T18:24:27.872370+0000","last_clean":"2024-04-23T18:24:27.872370+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:24:27.872370+0000","last_undegraded":"2024-04-23T18:24:27.872370+0000","last_fullsized":"2024-04-23T18:24:27.872370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.343885+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:24:32.343885+0000","last_peered":"2024-04-23T18:24:32.343885+0000","last_clean":"2024-04-23T18:24:32.343885+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:24:32.343885+0000","last_undegraded":"2024-04-23T18:24:32.343885+0000","last_fullsized":"2024-04-23T18:24:32.343885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183211+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:24:28.183211+0000","last_peered":"2024-04-23T18:24:28.183211+0000","last_clean":"2024-04-23T18:24:28.183211+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:24:28.183211+0000","last_undegraded":"2024-04-23T18:24:28.183211+0000","last_fullsized":"2024-04-23T18:24:28.183211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325746+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:24:30.325746+0000","last_peered":"2024-04-23T18:24:30.325746+0000","last_clean":"2024-04-23T18:24:30.325746+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:24:30.325746+0000","last_undegraded":"2024-04-23T18:24:30.325746+0000","last_fullsized":"2024-04-23T18:24:30.325746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183143+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:24:28.183143+0000","last_peered":"2024-04-23T18:24:28.183143+0000","last_clean":"2024-04-23T18:24:28.183143+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:24:28.183143+0000","last_undegraded":"2024-04-23T18:24:28.183143+0000","last_fullsized":"2024-04-23T18:24:28.183143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.873936+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:24:27.873936+0000","last_peered":"2024-04-23T18:24:27.873936+0000","last_clean":"2024-04-23T18:24:27.873936+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:24:27.873936+0000","last_undegraded":"2024-04-23T18:24:27.873936+0000","last_fullsized":"2024-04-23T18:24:27.873936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182455+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:24:28.182455+0000","last_peered":"2024-04-23T18:24:28.182455+0000","last_clean":"2024-04-23T18:24:28.182455+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:24:28.182455+0000","last_undegraded":"2024-04-23T18:24:28.182455+0000","last_fullsized":"2024-04-23T18:24:28.182455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.342726+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:24:32.342726+0000","last_peered":"2024-04-23T18:24:32.342726+0000","last_clean":"2024-04-23T18:24:32.342726+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:24:32.342726+0000","last_undegraded":"2024-04-23T18:24:32.342726+0000","last_fullsized":"2024-04-23T18:24:32.342726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182878+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:24:28.182878+0000","last_peered":"2024-04-23T18:24:28.182878+0000","last_clean":"2024-04-23T18:24:28.182878+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:24:28.182878+0000","last_undegraded":"2024-04-23T18:24:28.182878+0000","last_fullsized":"2024-04-23T18:24:28.182878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.320647+0000","last_change":"2024-04-23T18:24:26.302902+0000","last_active":"2024-04-23T18:24:31.320647+0000","last_peered":"2024-04-23T18:24:31.320647+0000","last_clean":"2024-04-23T18:24:31.320647+0000","last_became_active":"2024-04-23T18:24:26.302519+0000","last_became_peered":"2024-04-23T18:24:26.302519+0000","last_unstale":"2024-04-23T18:24:31.320647+0000","last_undegraded":"2024-04-23T18:24:31.320647+0000","last_fullsized":"2024-04-23T18:24:31.320647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:51.069269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183318+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:24:28.183318+0000","last_peered":"2024-04-23T18:24:28.183318+0000","last_clean":"2024-04-23T18:24:28.183318+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:24:28.183318+0000","last_undegraded":"2024-04-23T18:24:28.183318+0000","last_fullsized":"2024-04-23T18:24:28.183318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325935+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:24:30.325935+0000","last_peered":"2024-04-23T18:24:30.325935+0000","last_clean":"2024-04-23T18:24:30.325935+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:24:30.325935+0000","last_undegraded":"2024-04-23T18:24:30.325935+0000","last_fullsized":"2024-04-23T18:24:30.325935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264244+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:24:28.264244+0000","last_peered":"2024-04-23T18:24:28.264244+0000","last_clean":"2024-04-23T18:24:28.264244+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:24:28.264244+0000","last_undegraded":"2024-04-23T18:24:28.264244+0000","last_fullsized":"2024-04-23T18:24:28.264244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.341739+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:24:32.341739+0000","last_peered":"2024-04-23T18:24:32.341739+0000","last_clean":"2024-04-23T18:24:32.341739+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:24:32.341739+0000","last_undegraded":"2024-04-23T18:24:32.341739+0000","last_fullsized":"2024-04-23T18:24:32.341739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264360+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:24:28.264360+0000","last_peered":"2024-04-23T18:24:28.264360+0000","last_clean":"2024-04-23T18:24:28.264360+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:24:28.264360+0000","last_undegraded":"2024-04-23T18:24:28.264360+0000","last_fullsized":"2024-04-23T18:24:28.264360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183185+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:24:28.183185+0000","last_peered":"2024-04-23T18:24:28.183185+0000","last_clean":"2024-04-23T18:24:28.183185+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:24:28.183185+0000","last_undegraded":"2024-04-23T18:24:28.183185+0000","last_fullsized":"2024-04-23T18:24:28.183185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264343+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:24:28.264343+0000","last_peered":"2024-04-23T18:24:28.264343+0000","last_clean":"2024-04-23T18:24:28.264343+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:24:28.264343+0000","last_undegraded":"2024-04-23T18:24:28.264343+0000","last_fullsized":"2024-04-23T18:24:28.264343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872645+0000","last_change":"2024-04-23T18:24:26.289463+0000","last_active":"2024-04-23T18:24:27.872645+0000","last_peered":"2024-04-23T18:24:27.872645+0000","last_clean":"2024-04-23T18:24:27.872645+0000","last_became_active":"2024-04-23T18:24:26.288746+0000","last_became_peered":"2024-04-23T18:24:26.288746+0000","last_unstale":"2024-04-23T18:24:27.872645+0000","last_undegraded":"2024-04-23T18:24:27.872645+0000","last_fullsized":"2024-04-23T18:24:27.872645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:56.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264342+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:24:28.264342+0000","last_peered":"2024-04-23T18:24:28.264342+0000","last_clean":"2024-04-23T18:24:28.264342+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:24:28.264342+0000","last_undegraded":"2024-04-23T18:24:28.264342+0000","last_fullsized":"2024-04-23T18:24:28.264342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182973+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:24:28.182973+0000","last_peered":"2024-04-23T18:24:28.182973+0000","last_clean":"2024-04-23T18:24:28.182973+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:24:28.182973+0000","last_undegraded":"2024-04-23T18:24:28.182973+0000","last_fullsized":"2024-04-23T18:24:28.182973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872623+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:24:27.872623+0000","last_peered":"2024-04-23T18:24:27.872623+0000","last_clean":"2024-04-23T18:24:27.872623+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:24:27.872623+0000","last_undegraded":"2024-04-23T18:24:27.872623+0000","last_fullsized":"2024-04-23T18:24:27.872623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872424+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:24:27.872424+0000","last_peered":"2024-04-23T18:24:27.872424+0000","last_clean":"2024-04-23T18:24:27.872424+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:24:27.872424+0000","last_undegraded":"2024-04-23T18:24:27.872424+0000","last_fullsized":"2024-04-23T18:24:27.872424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182599+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:24:28.182599+0000","last_peered":"2024-04-23T18:24:28.182599+0000","last_clean":"2024-04-23T18:24:28.182599+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:24:28.182599+0000","last_undegraded":"2024-04-23T18:24:28.182599+0000","last_fullsized":"2024-04-23T18:24:28.182599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872361+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:24:27.872361+0000","last_peered":"2024-04-23T18:24:27.872361+0000","last_clean":"2024-04-23T18:24:27.872361+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:24:27.872361+0000","last_undegraded":"2024-04-23T18:24:27.872361+0000","last_fullsized":"2024-04-23T18:24:27.872361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183140+0000","last_change":"2024-04-23T18:24:26.892163+0000","last_active":"2024-04-23T18:24:28.183140+0000","last_peered":"2024-04-23T18:24:28.183140+0000","last_clean":"2024-04-23T18:24:28.183140+0000","last_became_active":"2024-04-23T18:24:26.891829+0000","last_became_peered":"2024-04-23T18:24:26.891829+0000","last_unstale":"2024-04-23T18:24:28.183140+0000","last_undegraded":"2024-04-23T18:24:28.183140+0000","last_fullsized":"2024-04-23T18:24:28.183140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:42.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872746+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:24:27.872746+0000","last_peered":"2024-04-23T18:24:27.872746+0000","last_clean":"2024-04-23T18:24:27.872746+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:24:27.872746+0000","last_undegraded":"2024-04-23T18:24:27.872746+0000","last_fullsized":"2024-04-23T18:24:27.872746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183142+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:24:28.183142+0000","last_peered":"2024-04-23T18:24:28.183142+0000","last_clean":"2024-04-23T18:24:28.183142+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:24:28.183142+0000","last_undegraded":"2024-04-23T18:24:28.183142+0000","last_fullsized":"2024-04-23T18:24:28.183142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182179+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:24:28.182179+0000","last_peered":"2024-04-23T18:24:28.182179+0000","last_clean":"2024-04-23T18:24:28.182179+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:24:28.182179+0000","last_undegraded":"2024-04-23T18:24:28.182179+0000","last_fullsized":"2024-04-23T18:24:28.182179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182880+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:24:28.182880+0000","last_peered":"2024-04-23T18:24:28.182880+0000","last_clean":"2024-04-23T18:24:28.182880+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:24:28.182880+0000","last_undegraded":"2024-04-23T18:24:28.182880+0000","last_fullsized":"2024-04-23T18:24:28.182880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874187+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:24:27.874187+0000","last_peered":"2024-04-23T18:24:27.874187+0000","last_clean":"2024-04-23T18:24:27.874187+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:24:27.874187+0000","last_undegraded":"2024-04-23T18:24:27.874187+0000","last_fullsized":"2024-04-23T18:24:27.874187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872483+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:24:27.872483+0000","last_peered":"2024-04-23T18:24:27.872483+0000","last_clean":"2024-04-23T18:24:27.872483+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:24:27.872483+0000","last_undegraded":"2024-04-23T18:24:27.872483+0000","last_fullsized":"2024-04-23T18:24:27.872483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182608+0000","last_change":"2024-04-23T18:24:26.891629+0000","last_active":"2024-04-23T18:24:28.182608+0000","last_peered":"2024-04-23T18:24:28.182608+0000","last_clean":"2024-04-23T18:24:28.182608+0000","last_became_active":"2024-04-23T18:24:26.891209+0000","last_became_peered":"2024-04-23T18:24:26.891209+0000","last_unstale":"2024-04-23T18:24:28.182608+0000","last_undegraded":"2024-04-23T18:24:28.182608+0000","last_fullsized":"2024-04-23T18:24:28.182608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:59.756258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280434,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5844,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":23017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1030695,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":438633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":449096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:24:35.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:24:35.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531543273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78100e90 msgr2=0x7f0c781012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531543273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78100e90 0x7f0c781012b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009900 tx=0x7f0c6c02a410 comp rx=0 tx=0).stop 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531543273 shutdown_connections 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531543273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c781029a0 0x7f0c781092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531543273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c78101fe0 0x7f0c78102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531543273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78100e90 0x7f0c781012b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531543273 >> 172.21.15.33:0/2531543273 conn(0x7f0c780fc400 msgr2=0x7f0c780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531543273 shutdown_connections 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.304+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531543273 wait complete. 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 Processor -- start 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 -- start start 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78100e90 0x7f0c78069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:35.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c781029a0 0x7f0c7819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810c9c0 con 0x7f0c78101fe0 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810c840 con 0x7f0c781029a0 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c7d5f7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7810cb40 con 0x7f0c78100e90 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44742/0 (socket says 172.21.15.33:44742) 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.305+0000 7f0c767fc640 1 -- 172.21.15.33:0/2531921876 learned_addr learned my addr 172.21.15.33:0/2531921876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c777fe640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c781029a0 0x7f0c7819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c76ffd640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78100e90 0x7f0c78069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 -- 172.21.15.33:0/2531921876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78100e90 msgr2=0x7f0c78069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78100e90 0x7f0c78069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 -- 172.21.15.33:0/2531921876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c781029a0 msgr2=0x7f0c7819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c781029a0 0x7f0c7819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 -- 172.21.15.33:0/2531921876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7819fda0 con 0x7f0c78101fe0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c767fc640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9c0 tx=0x7f0c6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6800cde0 con 0x7f0c78101fe0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c781a0090 con 0x7f0c78101fe0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c680108e0 con 0x7f0c78101fe0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c680188d0 con 0x7f0c78101fe0 2024-04-23T18:24:35.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.306+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c781a05e0 con 0x7f0c78101fe0 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.307+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c781081a0 con 0x7f0c78101fe0 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.308+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0c6801f020 con 0x7f0c78101fe0 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.309+0000 7f0c5ffff640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 0x7f0c600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.309+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0c680d0d00 con 0x7f0c78101fe0 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.309+0000 7f0c76ffd640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 0x7f0c600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.309+0000 7f0c76ffd640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 0x7f0c600a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000c00 tx=0x7f0c6c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:35.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.317+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c6809d630 con 0x7f0c78101fe0 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 17 op/s 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:active 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2630926046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:35 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1483588697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.504+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c78062390 con 0x7f0c600a3c40 2024-04-23T18:24:35.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.511+0000 7f0c5ffff640 1 -- 172.21.15.33:0/2531921876 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258927 (secure 0 0 0) 0x7f0c78062390 con 0x7f0c600a3c40 2024-04-23T18:24:35.508 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:24:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 msgr2=0x7f0c600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 0x7f0c600a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000c00 tx=0x7f0c6c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 msgr2=0x7f0c7819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800e9c0 tx=0x7f0c6800ee80 comp rx=0 tx=0).stop 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 shutdown_connections 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c600a3c40 0x7f0c600a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c781029a0 0x7f0c7819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c78101fe0 0x7f0c7819ce70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 --2- 172.21.15.33:0/2531921876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c78100e90 0x7f0c78069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 >> 172.21.15.33:0/2531921876 conn(0x7f0c780fc400 msgr2=0x7f0c78105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 shutdown_connections 2024-04-23T18:24:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:35.514+0000 7f0c7d5f7640 1 -- 172.21.15.33:0/2531921876 wait complete. 2024-04-23T18:24:35.519 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-23T18:24:34.150819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596212,"num_objects":65,"num_object_clones":0,"num_object_copies":195,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":316,"num_write_kb":2973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":269,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381048,"kb_used_data":11068,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560776,"statfs":{"total":1151940427776,"available":1151550234624,"internally_reserved":0,"allocated":11333632,"data_stored":7053511,"data_compressed":107010,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5844,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.878791"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183163+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:24:28.183163+0000","last_peered":"2024-04-23T18:24:28.183163+0000","last_clean":"2024-04-23T18:24:28.183163+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:24:28.183163+0000","last_undegraded":"2024-04-23T18:24:28.183163+0000","last_fullsized":"2024-04-23T18:24:28.183163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182969+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:24:28.182969+0000","last_peered":"2024-04-23T18:24:28.182969+0000","last_clean":"2024-04-23T18:24:28.182969+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:24:28.182969+0000","last_undegraded":"2024-04-23T18:24:28.182969+0000","last_fullsized":"2024-04-23T18:24:28.182969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182744+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:24:28.182744+0000","last_peered":"2024-04-23T18:24:28.182744+0000","last_clean":"2024-04-23T18:24:28.182744+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:24:28.182744+0000","last_undegraded":"2024-04-23T18:24:28.182744+0000","last_fullsized":"2024-04-23T18:24:28.182744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182882+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:24:28.182882+0000","last_peered":"2024-04-23T18:24:28.182882+0000","last_clean":"2024-04-23T18:24:28.182882+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:24:28.182882+0000","last_undegraded":"2024-04-23T18:24:28.182882+0000","last_fullsized":"2024-04-23T18:24:28.182882+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182447+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:24:28.182447+0000","last_peered":"2024-04-23T18:24:28.182447+0000","last_clean":"2024-04-23T18:24:28.182447+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:24:28.182447+0000","last_undegraded":"2024-04-23T18:24:28.182447+0000","last_fullsized":"2024-04-23T18:24:28.182447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182707+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:24:28.182707+0000","last_peered":"2024-04-23T18:24:28.182707+0000","last_clean":"2024-04-23T18:24:28.182707+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:24:28.182707+0000","last_undegraded":"2024-04-23T18:24:28.182707+0000","last_fullsized":"2024-04-23T18:24:28.182707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264088+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:24:28.264088+0000","last_peered":"2024-04-23T18:24:28.264088+0000","last_clean":"2024-04-23T18:24:28.264088+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:24:28.264088+0000","last_undegraded":"2024-04-23T18:24:28.264088+0000","last_fullsized":"2024-04-23T18:24:28.264088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182643+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:24:28.182643+0000","last_peered":"2024-04-23T18:24:28.182643+0000","last_clean":"2024-04-23T18:24:28.182643+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:24:28.182643+0000","last_undegraded":"2024-04-23T18:24:28.182643+0000","last_fullsized":"2024-04-23T18:24:28.182643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874419+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:24:27.874419+0000","last_peered":"2024-04-23T18:24:27.874419+0000","last_clean":"2024-04-23T18:24:27.874419+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:24:27.874419+0000","last_undegraded":"2024-04-23T18:24:27.874419+0000","last_fullsized":"2024-04-23T18:24:27.874419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183060+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:24:28.183060+0000","last_peered":"2024-04-23T18:24:28.183060+0000","last_clean":"2024-04-23T18:24:28.183060+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:24:28.183060+0000","last_undegraded":"2024-04-23T18:24:28.183060+0000","last_fullsized":"2024-04-23T18:24:28.183060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264332+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:24:28.264332+0000","last_peered":"2024-04-23T18:24:28.264332+0000","last_clean":"2024-04-23T18:24:28.264332+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:24:28.264332+0000","last_undegraded":"2024-04-23T18:24:28.264332+0000","last_fullsized":"2024-04-23T18:24:28.264332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182922+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:24:28.182922+0000","last_peered":"2024-04-23T18:24:28.182922+0000","last_clean":"2024-04-23T18:24:28.182922+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:24:28.182922+0000","last_undegraded":"2024-04-23T18:24:28.182922+0000","last_fullsized":"2024-04-23T18:24:28.182922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872464+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:24:27.872464+0000","last_peered":"2024-04-23T18:24:27.872464+0000","last_clean":"2024-04-23T18:24:27.872464+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:24:27.872464+0000","last_undegraded":"2024-04-23T18:24:27.872464+0000","last_fullsized":"2024-04-23T18:24:27.872464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182334+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:24:28.182334+0000","last_peered":"2024-04-23T18:24:28.182334+0000","last_clean":"2024-04-23T18:24:28.182334+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:24:28.182334+0000","last_undegraded":"2024-04-23T18:24:28.182334+0000","last_fullsized":"2024-04-23T18:24:28.182334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182199+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:24:28.182199+0000","last_peered":"2024-04-23T18:24:28.182199+0000","last_clean":"2024-04-23T18:24:28.182199+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:24:28.182199+0000","last_undegraded":"2024-04-23T18:24:28.182199+0000","last_fullsized":"2024-04-23T18:24:28.182199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.873962+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:24:27.873962+0000","last_peered":"2024-04-23T18:24:27.873962+0000","last_clean":"2024-04-23T18:24:27.873962+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:24:27.873962+0000","last_undegraded":"2024-04-23T18:24:27.873962+0000","last_fullsized":"2024-04-23T18:24:27.873962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183372+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:24:28.183372+0000","last_peered":"2024-04-23T18:24:28.183372+0000","last_clean":"2024-04-23T18:24:28.183372+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:24:28.183372+0000","last_undegraded":"2024-04-23T18:24:28.183372+0000","last_fullsized":"2024-04-23T18:24:28.183372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183305+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:24:28.183305+0000","last_peered":"2024-04-23T18:24:28.183305+0000","last_clean":"2024-04-23T18:24:28.183305+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:24:28.183305+0000","last_undegraded":"2024-04-23T18:24:28.183305+0000","last_fullsized":"2024-04-23T18:24:28.183305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872646+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:24:27.872646+0000","last_peered":"2024-04-23T18:24:27.872646+0000","last_clean":"2024-04-23T18:24:27.872646+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:24:27.872646+0000","last_undegraded":"2024-04-23T18:24:27.872646+0000","last_fullsized":"2024-04-23T18:24:27.872646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":28,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:33.348597+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:24:33.348597+0000","last_peered":"2024-04-23T18:24:33.348597+0000","last_clean":"2024-04-23T18:24:33.348597+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:24:33.348597+0000","last_undegraded":"2024-04-23T18:24:33.348597+0000","last_fullsized":"2024-04-23T18:24:33.348597+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182801+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:24:28.182801+0000","last_peered":"2024-04-23T18:24:28.182801+0000","last_clean":"2024-04-23T18:24:28.182801+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:24:28.182801+0000","last_undegraded":"2024-04-23T18:24:28.182801+0000","last_fullsized":"2024-04-23T18:24:28.182801+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183137+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:24:28.183137+0000","last_peered":"2024-04-23T18:24:28.183137+0000","last_clean":"2024-04-23T18:24:28.183137+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:24:28.183137+0000","last_undegraded":"2024-04-23T18:24:28.183137+0000","last_fullsized":"2024-04-23T18:24:28.183137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182930+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:24:28.182930+0000","last_peered":"2024-04-23T18:24:28.182930+0000","last_clean":"2024-04-23T18:24:28.182930+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:24:28.182930+0000","last_undegraded":"2024-04-23T18:24:28.182930+0000","last_fullsized":"2024-04-23T18:24:28.182930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183421+0000","last_change":"2024-04-23T18:24:26.289042+0000","last_active":"2024-04-23T18:24:28.183421+0000","last_peered":"2024-04-23T18:24:28.183421+0000","last_clean":"2024-04-23T18:24:28.183421+0000","last_became_active":"2024-04-23T18:24:26.288711+0000","last_became_peered":"2024-04-23T18:24:26.288711+0000","last_unstale":"2024-04-23T18:24:28.183421+0000","last_undegraded":"2024-04-23T18:24:28.183421+0000","last_fullsized":"2024-04-23T18:24:28.183421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:24.914645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874355+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:24:27.874355+0000","last_peered":"2024-04-23T18:24:27.874355+0000","last_clean":"2024-04-23T18:24:27.874355+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:24:27.874355+0000","last_undegraded":"2024-04-23T18:24:27.874355+0000","last_fullsized":"2024-04-23T18:24:27.874355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182880+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:24:28.182880+0000","last_peered":"2024-04-23T18:24:28.182880+0000","last_clean":"2024-04-23T18:24:28.182880+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:24:28.182880+0000","last_undegraded":"2024-04-23T18:24:28.182880+0000","last_fullsized":"2024-04-23T18:24:28.182880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183334+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:24:28.183334+0000","last_peered":"2024-04-23T18:24:28.183334+0000","last_clean":"2024-04-23T18:24:28.183334+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:24:28.183334+0000","last_undegraded":"2024-04-23T18:24:28.183334+0000","last_fullsized":"2024-04-23T18:24:28.183334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.326018+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:24:30.326018+0000","last_peered":"2024-04-23T18:24:30.326018+0000","last_clean":"2024-04-23T18:24:30.326018+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:24:30.326018+0000","last_undegraded":"2024-04-23T18:24:30.326018+0000","last_fullsized":"2024-04-23T18:24:30.326018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183267+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:24:28.183267+0000","last_peered":"2024-04-23T18:24:28.183267+0000","last_clean":"2024-04-23T18:24:28.183267+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:24:28.183267+0000","last_undegraded":"2024-04-23T18:24:28.183267+0000","last_fullsized":"2024-04-23T18:24:28.183267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182473+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:24:28.182473+0000","last_peered":"2024-04-23T18:24:28.182473+0000","last_clean":"2024-04-23T18:24:28.182473+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:24:28.182473+0000","last_undegraded":"2024-04-23T18:24:28.182473+0000","last_fullsized":"2024-04-23T18:24:28.182473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182826+0000","last_change":"2024-04-23T18:24:26.894333+0000","last_active":"2024-04-23T18:24:28.182826+0000","last_peered":"2024-04-23T18:24:28.182826+0000","last_clean":"2024-04-23T18:24:28.182826+0000","last_became_active":"2024-04-23T18:24:26.893843+0000","last_became_peered":"2024-04-23T18:24:26.893843+0000","last_unstale":"2024-04-23T18:24:28.182826+0000","last_undegraded":"2024-04-23T18:24:28.182826+0000","last_fullsized":"2024-04-23T18:24:28.182826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:57:44.102339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.333566+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:24:31.333566+0000","last_peered":"2024-04-23T18:24:31.333566+0000","last_clean":"2024-04-23T18:24:31.333566+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:24:31.333566+0000","last_undegraded":"2024-04-23T18:24:31.333566+0000","last_fullsized":"2024-04-23T18:24:31.333566+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182490+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:24:28.182490+0000","last_peered":"2024-04-23T18:24:28.182490+0000","last_clean":"2024-04-23T18:24:28.182490+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:24:28.182490+0000","last_undegraded":"2024-04-23T18:24:28.182490+0000","last_fullsized":"2024-04-23T18:24:28.182490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182245+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:24:28.182245+0000","last_peered":"2024-04-23T18:24:28.182245+0000","last_clean":"2024-04-23T18:24:28.182245+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:24:28.182245+0000","last_undegraded":"2024-04-23T18:24:28.182245+0000","last_fullsized":"2024-04-23T18:24:28.182245+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182645+0000","last_change":"2024-04-23T18:24:26.891966+0000","last_active":"2024-04-23T18:24:28.182645+0000","last_peered":"2024-04-23T18:24:28.182645+0000","last_clean":"2024-04-23T18:24:28.182645+0000","last_became_active":"2024-04-23T18:24:26.891579+0000","last_became_peered":"2024-04-23T18:24:26.891579+0000","last_unstale":"2024-04-23T18:24:28.182645+0000","last_undegraded":"2024-04-23T18:24:28.182645+0000","last_fullsized":"2024-04-23T18:24:28.182645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:10:18.633162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.324772+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:24:30.324772+0000","last_peered":"2024-04-23T18:24:30.324772+0000","last_clean":"2024-04-23T18:24:30.324772+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:24:30.324772+0000","last_undegraded":"2024-04-23T18:24:30.324772+0000","last_fullsized":"2024-04-23T18:24:30.324772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874334+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:24:27.874334+0000","last_peered":"2024-04-23T18:24:27.874334+0000","last_clean":"2024-04-23T18:24:27.874334+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:24:27.874334+0000","last_undegraded":"2024-04-23T18:24:27.874334+0000","last_fullsized":"2024-04-23T18:24:27.874334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.326117+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:24:30.326117+0000","last_peered":"2024-04-23T18:24:30.326117+0000","last_clean":"2024-04-23T18:24:30.326117+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:24:30.326117+0000","last_undegraded":"2024-04-23T18:24:30.326117+0000","last_fullsized":"2024-04-23T18:24:30.326117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182755+0000","last_change":"2024-04-23T18:24:26.892107+0000","last_active":"2024-04-23T18:24:28.182755+0000","last_peered":"2024-04-23T18:24:28.182755+0000","last_clean":"2024-04-23T18:24:28.182755+0000","last_became_active":"2024-04-23T18:24:26.891805+0000","last_became_peered":"2024-04-23T18:24:26.891805+0000","last_unstale":"2024-04-23T18:24:28.182755+0000","last_undegraded":"2024-04-23T18:24:28.182755+0000","last_fullsized":"2024-04-23T18:24:28.182755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:20:06.268770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183496+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:24:28.183496+0000","last_peered":"2024-04-23T18:24:28.183496+0000","last_clean":"2024-04-23T18:24:28.183496+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:24:28.183496+0000","last_undegraded":"2024-04-23T18:24:28.183496+0000","last_fullsized":"2024-04-23T18:24:28.183496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874315+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:24:27.874315+0000","last_peered":"2024-04-23T18:24:27.874315+0000","last_clean":"2024-04-23T18:24:27.874315+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:24:27.874315+0000","last_undegraded":"2024-04-23T18:24:27.874315+0000","last_fullsized":"2024-04-23T18:24:27.874315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325794+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:24:30.325794+0000","last_peered":"2024-04-23T18:24:30.325794+0000","last_clean":"2024-04-23T18:24:30.325794+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:24:30.325794+0000","last_undegraded":"2024-04-23T18:24:30.325794+0000","last_fullsized":"2024-04-23T18:24:30.325794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183129+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:24:28.183129+0000","last_peered":"2024-04-23T18:24:28.183129+0000","last_clean":"2024-04-23T18:24:28.183129+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:24:28.183129+0000","last_undegraded":"2024-04-23T18:24:28.183129+0000","last_fullsized":"2024-04-23T18:24:28.183129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'7","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.332935+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:24:31.332935+0000","last_peered":"2024-04-23T18:24:31.332935+0000","last_clean":"2024-04-23T18:24:31.332935+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:24:31.332935+0000","last_undegraded":"2024-04-23T18:24:31.332935+0000","last_fullsized":"2024-04-23T18:24:31.332935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872449+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:24:27.872449+0000","last_peered":"2024-04-23T18:24:27.872449+0000","last_clean":"2024-04-23T18:24:27.872449+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:24:27.872449+0000","last_undegraded":"2024-04-23T18:24:27.872449+0000","last_fullsized":"2024-04-23T18:24:27.872449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325656+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:24:30.325656+0000","last_peered":"2024-04-23T18:24:30.325656+0000","last_clean":"2024-04-23T18:24:30.325656+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:24:30.325656+0000","last_undegraded":"2024-04-23T18:24:30.325656+0000","last_fullsized":"2024-04-23T18:24:30.325656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264323+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:24:28.264323+0000","last_peered":"2024-04-23T18:24:28.264323+0000","last_clean":"2024-04-23T18:24:28.264323+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:24:28.264323+0000","last_undegraded":"2024-04-23T18:24:28.264323+0000","last_fullsized":"2024-04-23T18:24:28.264323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.339822+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:24:32.339822+0000","last_peered":"2024-04-23T18:24:32.339822+0000","last_clean":"2024-04-23T18:24:32.339822+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:24:32.339822+0000","last_undegraded":"2024-04-23T18:24:32.339822+0000","last_fullsized":"2024-04-23T18:24:32.339822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.263975+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:24:28.263975+0000","last_peered":"2024-04-23T18:24:28.263975+0000","last_clean":"2024-04-23T18:24:28.263975+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:24:28.263975+0000","last_undegraded":"2024-04-23T18:24:28.263975+0000","last_fullsized":"2024-04-23T18:24:28.263975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325235+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:24:30.325235+0000","last_peered":"2024-04-23T18:24:30.325235+0000","last_clean":"2024-04-23T18:24:30.325235+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:24:30.325235+0000","last_undegraded":"2024-04-23T18:24:30.325235+0000","last_fullsized":"2024-04-23T18:24:30.325235+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182405+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:24:28.182405+0000","last_peered":"2024-04-23T18:24:28.182405+0000","last_clean":"2024-04-23T18:24:28.182405+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:24:28.182405+0000","last_undegraded":"2024-04-23T18:24:28.182405+0000","last_fullsized":"2024-04-23T18:24:28.182405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182463+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:24:28.182463+0000","last_peered":"2024-04-23T18:24:28.182463+0000","last_clean":"2024-04-23T18:24:28.182463+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:24:28.182463+0000","last_undegraded":"2024-04-23T18:24:28.182463+0000","last_fullsized":"2024-04-23T18:24:28.182463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183305+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:24:28.183305+0000","last_peered":"2024-04-23T18:24:28.183305+0000","last_clean":"2024-04-23T18:24:28.183305+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:24:28.183305+0000","last_undegraded":"2024-04-23T18:24:28.183305+0000","last_fullsized":"2024-04-23T18:24:28.183305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.762107+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:24:28.762107+0000","last_peered":"2024-04-23T18:24:28.762107+0000","last_clean":"2024-04-23T18:24:28.762107+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:24:28.762107+0000","last_undegraded":"2024-04-23T18:24:28.762107+0000","last_fullsized":"2024-04-23T18:24:28.762107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183054+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:24:28.183054+0000","last_peered":"2024-04-23T18:24:28.183054+0000","last_clean":"2024-04-23T18:24:28.183054+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:24:28.183054+0000","last_undegraded":"2024-04-23T18:24:28.183054+0000","last_fullsized":"2024-04-23T18:24:28.183054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264190+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:24:28.264190+0000","last_peered":"2024-04-23T18:24:28.264190+0000","last_clean":"2024-04-23T18:24:28.264190+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:24:28.264190+0000","last_undegraded":"2024-04-23T18:24:28.264190+0000","last_fullsized":"2024-04-23T18:24:28.264190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183246+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:24:28.183246+0000","last_peered":"2024-04-23T18:24:28.183246+0000","last_clean":"2024-04-23T18:24:28.183246+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:24:28.183246+0000","last_undegraded":"2024-04-23T18:24:28.183246+0000","last_fullsized":"2024-04-23T18:24:28.183246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182484+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:24:28.182484+0000","last_peered":"2024-04-23T18:24:28.182484+0000","last_clean":"2024-04-23T18:24:28.182484+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:24:28.182484+0000","last_undegraded":"2024-04-23T18:24:28.182484+0000","last_fullsized":"2024-04-23T18:24:28.182484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.332410+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:24:31.332410+0000","last_peered":"2024-04-23T18:24:31.332410+0000","last_clean":"2024-04-23T18:24:31.332410+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:24:31.332410+0000","last_undegraded":"2024-04-23T18:24:31.332410+0000","last_fullsized":"2024-04-23T18:24:31.332410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183728+0000","last_change":"2024-04-23T18:24:26.899273+0000","last_active":"2024-04-23T18:24:28.183728+0000","last_peered":"2024-04-23T18:24:28.183728+0000","last_clean":"2024-04-23T18:24:28.183728+0000","last_became_active":"2024-04-23T18:24:26.898967+0000","last_became_peered":"2024-04-23T18:24:26.898967+0000","last_unstale":"2024-04-23T18:24:28.183728+0000","last_undegraded":"2024-04-23T18:24:28.183728+0000","last_fullsized":"2024-04-23T18:24:28.183728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:03:30.309041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.339411+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:24:32.339411+0000","last_peered":"2024-04-23T18:24:32.339411+0000","last_clean":"2024-04-23T18:24:32.339411+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:24:32.339411+0000","last_undegraded":"2024-04-23T18:24:32.339411+0000","last_fullsized":"2024-04-23T18:24:32.339411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.263946+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:24:28.263946+0000","last_peered":"2024-04-23T18:24:28.263946+0000","last_clean":"2024-04-23T18:24:28.263946+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:24:28.263946+0000","last_undegraded":"2024-04-23T18:24:28.263946+0000","last_fullsized":"2024-04-23T18:24:28.263946+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872112+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:24:27.872112+0000","last_peered":"2024-04-23T18:24:27.872112+0000","last_clean":"2024-04-23T18:24:27.872112+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:24:27.872112+0000","last_undegraded":"2024-04-23T18:24:27.872112+0000","last_fullsized":"2024-04-23T18:24:27.872112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.330445+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:24:31.330445+0000","last_peered":"2024-04-23T18:24:31.330445+0000","last_clean":"2024-04-23T18:24:31.330445+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:24:31.330445+0000","last_undegraded":"2024-04-23T18:24:31.330445+0000","last_fullsized":"2024-04-23T18:24:31.330445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183288+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:24:28.183288+0000","last_peered":"2024-04-23T18:24:28.183288+0000","last_clean":"2024-04-23T18:24:28.183288+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:24:28.183288+0000","last_undegraded":"2024-04-23T18:24:28.183288+0000","last_fullsized":"2024-04-23T18:24:28.183288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872370+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:24:27.872370+0000","last_peered":"2024-04-23T18:24:27.872370+0000","last_clean":"2024-04-23T18:24:27.872370+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:24:27.872370+0000","last_undegraded":"2024-04-23T18:24:27.872370+0000","last_fullsized":"2024-04-23T18:24:27.872370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.343885+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:24:32.343885+0000","last_peered":"2024-04-23T18:24:32.343885+0000","last_clean":"2024-04-23T18:24:32.343885+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:24:32.343885+0000","last_undegraded":"2024-04-23T18:24:32.343885+0000","last_fullsized":"2024-04-23T18:24:32.343885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183211+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:24:28.183211+0000","last_peered":"2024-04-23T18:24:28.183211+0000","last_clean":"2024-04-23T18:24:28.183211+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:24:28.183211+0000","last_undegraded":"2024-04-23T18:24:28.183211+0000","last_fullsized":"2024-04-23T18:24:28.183211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325746+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:24:30.325746+0000","last_peered":"2024-04-23T18:24:30.325746+0000","last_clean":"2024-04-23T18:24:30.325746+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:24:30.325746+0000","last_undegraded":"2024-04-23T18:24:30.325746+0000","last_fullsized":"2024-04-23T18:24:30.325746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183143+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:24:28.183143+0000","last_peered":"2024-04-23T18:24:28.183143+0000","last_clean":"2024-04-23T18:24:28.183143+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:24:28.183143+0000","last_undegraded":"2024-04-23T18:24:28.183143+0000","last_fullsized":"2024-04-23T18:24:28.183143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.873936+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:24:27.873936+0000","last_peered":"2024-04-23T18:24:27.873936+0000","last_clean":"2024-04-23T18:24:27.873936+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:24:27.873936+0000","last_undegraded":"2024-04-23T18:24:27.873936+0000","last_fullsized":"2024-04-23T18:24:27.873936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182455+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:24:28.182455+0000","last_peered":"2024-04-23T18:24:28.182455+0000","last_clean":"2024-04-23T18:24:28.182455+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:24:28.182455+0000","last_undegraded":"2024-04-23T18:24:28.182455+0000","last_fullsized":"2024-04-23T18:24:28.182455+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.342726+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:24:32.342726+0000","last_peered":"2024-04-23T18:24:32.342726+0000","last_clean":"2024-04-23T18:24:32.342726+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:24:32.342726+0000","last_undegraded":"2024-04-23T18:24:32.342726+0000","last_fullsized":"2024-04-23T18:24:32.342726+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182878+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:24:28.182878+0000","last_peered":"2024-04-23T18:24:28.182878+0000","last_clean":"2024-04-23T18:24:28.182878+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:24:28.182878+0000","last_undegraded":"2024-04-23T18:24:28.182878+0000","last_fullsized":"2024-04-23T18:24:28.182878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:31.320647+0000","last_change":"2024-04-23T18:24:26.302902+0000","last_active":"2024-04-23T18:24:31.320647+0000","last_peered":"2024-04-23T18:24:31.320647+0000","last_clean":"2024-04-23T18:24:31.320647+0000","last_became_active":"2024-04-23T18:24:26.302519+0000","last_became_peered":"2024-04-23T18:24:26.302519+0000","last_unstale":"2024-04-23T18:24:31.320647+0000","last_undegraded":"2024-04-23T18:24:31.320647+0000","last_fullsized":"2024-04-23T18:24:31.320647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:36:51.069269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183318+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:24:28.183318+0000","last_peered":"2024-04-23T18:24:28.183318+0000","last_clean":"2024-04-23T18:24:28.183318+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:24:28.183318+0000","last_undegraded":"2024-04-23T18:24:28.183318+0000","last_fullsized":"2024-04-23T18:24:28.183318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:30.325935+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:24:30.325935+0000","last_peered":"2024-04-23T18:24:30.325935+0000","last_clean":"2024-04-23T18:24:30.325935+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:24:30.325935+0000","last_undegraded":"2024-04-23T18:24:30.325935+0000","last_fullsized":"2024-04-23T18:24:30.325935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264244+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:24:28.264244+0000","last_peered":"2024-04-23T18:24:28.264244+0000","last_clean":"2024-04-23T18:24:28.264244+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:24:28.264244+0000","last_undegraded":"2024-04-23T18:24:28.264244+0000","last_fullsized":"2024-04-23T18:24:28.264244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:32.341739+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:24:32.341739+0000","last_peered":"2024-04-23T18:24:32.341739+0000","last_clean":"2024-04-23T18:24:32.341739+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:24:32.341739+0000","last_undegraded":"2024-04-23T18:24:32.341739+0000","last_fullsized":"2024-04-23T18:24:32.341739+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264360+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:24:28.264360+0000","last_peered":"2024-04-23T18:24:28.264360+0000","last_clean":"2024-04-23T18:24:28.264360+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:24:28.264360+0000","last_undegraded":"2024-04-23T18:24:28.264360+0000","last_fullsized":"2024-04-23T18:24:28.264360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183185+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:24:28.183185+0000","last_peered":"2024-04-23T18:24:28.183185+0000","last_clean":"2024-04-23T18:24:28.183185+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:24:28.183185+0000","last_undegraded":"2024-04-23T18:24:28.183185+0000","last_fullsized":"2024-04-23T18:24:28.183185+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264343+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:24:28.264343+0000","last_peered":"2024-04-23T18:24:28.264343+0000","last_clean":"2024-04-23T18:24:28.264343+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:24:28.264343+0000","last_undegraded":"2024-04-23T18:24:28.264343+0000","last_fullsized":"2024-04-23T18:24:28.264343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872645+0000","last_change":"2024-04-23T18:24:26.289463+0000","last_active":"2024-04-23T18:24:27.872645+0000","last_peered":"2024-04-23T18:24:27.872645+0000","last_clean":"2024-04-23T18:24:27.872645+0000","last_became_active":"2024-04-23T18:24:26.288746+0000","last_became_peered":"2024-04-23T18:24:26.288746+0000","last_unstale":"2024-04-23T18:24:27.872645+0000","last_undegraded":"2024-04-23T18:24:27.872645+0000","last_fullsized":"2024-04-23T18:24:27.872645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:58:56.310614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.264342+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:24:28.264342+0000","last_peered":"2024-04-23T18:24:28.264342+0000","last_clean":"2024-04-23T18:24:28.264342+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:24:28.264342+0000","last_undegraded":"2024-04-23T18:24:28.264342+0000","last_fullsized":"2024-04-23T18:24:28.264342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182973+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:24:28.182973+0000","last_peered":"2024-04-23T18:24:28.182973+0000","last_clean":"2024-04-23T18:24:28.182973+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:24:28.182973+0000","last_undegraded":"2024-04-23T18:24:28.182973+0000","last_fullsized":"2024-04-23T18:24:28.182973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872623+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:24:27.872623+0000","last_peered":"2024-04-23T18:24:27.872623+0000","last_clean":"2024-04-23T18:24:27.872623+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:24:27.872623+0000","last_undegraded":"2024-04-23T18:24:27.872623+0000","last_fullsized":"2024-04-23T18:24:27.872623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872424+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:24:27.872424+0000","last_peered":"2024-04-23T18:24:27.872424+0000","last_clean":"2024-04-23T18:24:27.872424+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:24:27.872424+0000","last_undegraded":"2024-04-23T18:24:27.872424+0000","last_fullsized":"2024-04-23T18:24:27.872424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182599+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:24:28.182599+0000","last_peered":"2024-04-23T18:24:28.182599+0000","last_clean":"2024-04-23T18:24:28.182599+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:24:28.182599+0000","last_undegraded":"2024-04-23T18:24:28.182599+0000","last_fullsized":"2024-04-23T18:24:28.182599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872361+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:24:27.872361+0000","last_peered":"2024-04-23T18:24:27.872361+0000","last_clean":"2024-04-23T18:24:27.872361+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:24:27.872361+0000","last_undegraded":"2024-04-23T18:24:27.872361+0000","last_fullsized":"2024-04-23T18:24:27.872361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183140+0000","last_change":"2024-04-23T18:24:26.892163+0000","last_active":"2024-04-23T18:24:28.183140+0000","last_peered":"2024-04-23T18:24:28.183140+0000","last_clean":"2024-04-23T18:24:28.183140+0000","last_became_active":"2024-04-23T18:24:26.891829+0000","last_became_peered":"2024-04-23T18:24:26.891829+0000","last_unstale":"2024-04-23T18:24:28.183140+0000","last_undegraded":"2024-04-23T18:24:28.183140+0000","last_fullsized":"2024-04-23T18:24:28.183140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:25:42.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872746+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:24:27.872746+0000","last_peered":"2024-04-23T18:24:27.872746+0000","last_clean":"2024-04-23T18:24:27.872746+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:24:27.872746+0000","last_undegraded":"2024-04-23T18:24:27.872746+0000","last_fullsized":"2024-04-23T18:24:27.872746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.183142+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:24:28.183142+0000","last_peered":"2024-04-23T18:24:28.183142+0000","last_clean":"2024-04-23T18:24:28.183142+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:24:28.183142+0000","last_undegraded":"2024-04-23T18:24:28.183142+0000","last_fullsized":"2024-04-23T18:24:28.183142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182179+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:24:28.182179+0000","last_peered":"2024-04-23T18:24:28.182179+0000","last_clean":"2024-04-23T18:24:28.182179+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:24:28.182179+0000","last_undegraded":"2024-04-23T18:24:28.182179+0000","last_fullsized":"2024-04-23T18:24:28.182179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182880+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:24:28.182880+0000","last_peered":"2024-04-23T18:24:28.182880+0000","last_clean":"2024-04-23T18:24:28.182880+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:24:28.182880+0000","last_undegraded":"2024-04-23T18:24:28.182880+0000","last_fullsized":"2024-04-23T18:24:28.182880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.874187+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:24:27.874187+0000","last_peered":"2024-04-23T18:24:27.874187+0000","last_clean":"2024-04-23T18:24:27.874187+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:24:27.874187+0000","last_undegraded":"2024-04-23T18:24:27.874187+0000","last_fullsized":"2024-04-23T18:24:27.874187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:27.872483+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:24:27.872483+0000","last_peered":"2024-04-23T18:24:27.872483+0000","last_clean":"2024-04-23T18:24:27.872483+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:24:27.872483+0000","last_undegraded":"2024-04-23T18:24:27.872483+0000","last_fullsized":"2024-04-23T18:24:27.872483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-23T18:24:28.182608+0000","last_change":"2024-04-23T18:24:26.891629+0000","last_active":"2024-04-23T18:24:28.182608+0000","last_peered":"2024-04-23T18:24:28.182608+0000","last_clean":"2024-04-23T18:24:28.182608+0000","last_became_active":"2024-04-23T18:24:26.891209+0000","last_became_peered":"2024-04-23T18:24:26.891209+0000","last_unstale":"2024-04-23T18:24:28.182608+0000","last_undegraded":"2024-04-23T18:24:28.182608+0000","last_fullsized":"2024-04-23T18:24:28.182608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:17:59.756258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280434,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5844,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":135,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":23017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":913408,"data_stored":438521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":1030695,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":439643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070493,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":1030093,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":921600,"data_stored":438633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":925696,"data_stored":439215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":449096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":929792,"data_stored":438805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757291,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":1030365,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:24:35.523 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-23T18:24:35.524 INFO:tasks.cephadm:Setting up client nodes... 2024-04-23T18:24:35.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 17 op/s 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:active 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2630926046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.600 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:35 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1483588697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: pgmap v290: 97 pgs: 97 active+clean; 582 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s wr, 17 op/s 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] up:active 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:24:35.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2630926046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:35 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1483588697' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:36.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:36 smithi106 ceph-mon[32716]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:36.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:36 smithi106 ceph-mon[32716]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:36.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:36 smithi122 ceph-mon[32546]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:36.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:36 smithi122 ceph-mon[32546]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:36.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:36 smithi033 ceph-mon[29274]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:36.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:36 smithi033 ceph-mon[29274]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:24:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 -- 172.21.15.33:0/910493039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 msgr2=0x7fd16c109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 --2- 172.21.15.33:0/910493039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c109e10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd1640099c0 tx=0x7fd164009cd0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 -- 172.21.15.33:0/910493039 shutdown_connections 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 --2- 172.21.15.33:0/910493039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c109e10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 --2- 172.21.15.33:0/910493039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c1019c0 0x7fd16c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 --2- 172.21.15.33:0/910493039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c102bc0 0x7fd16c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 -- 172.21.15.33:0/910493039 >> 172.21.15.33:0/910493039 conn(0x7fd16c0fcf60 msgr2=0x7fd16c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 -- 172.21.15.33:0/910493039 shutdown_connections 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.516+0000 7fd1735da700 1 -- 172.21.15.33:0/910493039 wait complete. 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 Processor -- start 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 -- start start 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c1019c0 0x7fd16c105900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 0x7fd16c103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c10d460 con 0x7fd16c103560 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c10d2e0 con 0x7fd16c102bc0 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd1735da700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c10d5e0 con 0x7fd16c1019c0 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53518/0 (socket says 172.21.15.33:53518) 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 -- 172.21.15.33:0/3342124560 learned_addr learned my addr 172.21.15.33:0/3342124560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171376700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c1019c0 0x7fd16c105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd170b75700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 0x7fd16c103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 -- 172.21.15.33:0/3342124560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c1019c0 msgr2=0x7fd16c105900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c1019c0 0x7fd16c105900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 -- 172.21.15.33:0/3342124560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 msgr2=0x7fd16c103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 0x7fd16c103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd171b77700 1 -- 172.21.15.33:0/3342124560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c104db0 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.517+0000 7fd170b75700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 0x7fd16c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd171b77700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1640059d0 tx=0x7fd1640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164027070 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1640040e0 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c1a2b60 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16400f460 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.518+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c1a3060 con 0x7fd16c103560 2024-04-23T18:24:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.519+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16c050f80 con 0x7fd16c103560 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.520+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd16402a070 con 0x7fd16c103560 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.521+0000 7fd16a7fc700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 0x7fd1540a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.521+0000 7fd171376700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 0x7fd1540a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.521+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd1640d12d0 con 0x7fd16c103560 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.521+0000 7fd171376700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 0x7fd1540a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd16c102a20 tx=0x7fd15c007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:37.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.525+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd16405eff0 con 0x7fd16c103560 2024-04-23T18:24:37.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:37 smithi106 ceph-mon[32716]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-04-23T18:24:37.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:37 smithi122 ceph-mon[32546]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-04-23T18:24:37.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:37 smithi033 ceph-mon[29274]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-04-23T18:24:37.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.821+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd16c1a3460 con 0x7fd16c103560 2024-04-23T18:24:37.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.824+0000 7fd16a7fc700 1 -- 172.21.15.33:0/3342124560 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd16c1a3460 con 0x7fd16c103560 2024-04-23T18:24:37.817 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-23T18:24:37.817 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDl/Cdm2BIRMRAARfXMtY5G5UU8txMVyAtc1w== 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 msgr2=0x7fd1540a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 0x7fd1540a62c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd16c102a20 tx=0x7fd15c007a30 comp rx=0 tx=0).stop 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 msgr2=0x7fd16c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1640059d0 tx=0x7fd1640047b0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 shutdown_connections 2024-04-23T18:24:37.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd1540a3e00 0x7fd1540a62c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd16c103560 0x7fd16c1044e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c1019c0 0x7fd16c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 --2- 172.21.15.33:0/3342124560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd16c102bc0 0x7fd16c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 >> 172.21.15.33:0/3342124560 conn(0x7fd16c0fcf60 msgr2=0x7fd16c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 shutdown_connections 2024-04-23T18:24:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:37.826+0000 7fd1735da700 1 -- 172.21.15.33:0/3342124560 wait complete. 2024-04-23T18:24:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:38.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:38 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:38.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:38 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:38.680 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:24:38.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-23T18:24:38.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-23T18:24:38.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:38 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-23T18:24:38.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:38 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3342124560' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-23T18:24:38.720 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-23T18:24:38.720 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-23T18:24:38.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-23T18:24:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:39 smithi106 ceph-mon[32716]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-23T18:24:40.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:39 smithi033 ceph-mon[29274]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-23T18:24:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:39 smithi122 ceph-mon[32546]: pgmap v292: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 26 op/s 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- 172.21.15.33:0/2098283128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 msgr2=0x7f4a70102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2098283128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70102f10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009990 tx=0x7f4a60009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- 172.21.15.33:0/2098283128 shutdown_connections 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2098283128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 0x7f4a7010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2098283128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a70108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2098283128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70102f10 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- 172.21.15.33:0/2098283128 >> 172.21.15.33:0/2098283128 conn(0x7f4a70079ba0 msgr2=0x7f4a70079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- 172.21.15.33:0/2098283128 shutdown_connections 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- 172.21.15.33:0/2098283128 wait complete. 2024-04-23T18:24:40.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 Processor -- start 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.782+0000 7f4a777e1700 1 -- start start 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 0x7f4a70075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a700757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70112c80 con 0x7f4a70102a90 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70112b00 con 0x7f4a70108a50 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a70112e00 con 0x7f4a70103450 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53542/0 (socket says 172.21.15.33:53542) 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 -- 172.21.15.33:0/2049523475 learned_addr learned my addr 172.21.15.33:0/2049523475 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a74d7c700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 0x7f4a70075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a75d7e700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a700757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 -- 172.21.15.33:0/2049523475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 msgr2=0x7f4a70075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 0x7f4a70075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 -- 172.21.15.33:0/2049523475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 msgr2=0x7f4a700757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a700757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 -- 172.21.15.33:0/2049523475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a701a7700 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a75d7e700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a700757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a7557d700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00b530 tx=0x7f4a6c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c015020 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a701a79f0 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a6c011ce0 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.783+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c010e40 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.784+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a7f50 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.785+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a70069730 con 0x7f4a70102a90 2024-04-23T18:24:40.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.786+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4a6c018020 con 0x7f4a70102a90 2024-04-23T18:24:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.786+0000 7f4a667fc700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 0x7f4a5c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.786+0000 7f4a74d7c700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 0x7f4a5c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.786+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f4a6c0cfb30 con 0x7f4a70102a90 2024-04-23T18:24:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.787+0000 7f4a74d7c700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 0x7f4a5c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b5c0 tx=0x7f4a60005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:40.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:40.791+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a6c01f020 con 0x7f4a70102a90 2024-04-23T18:24:41.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.061+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f4a7002df80 con 0x7f4a70102a90 2024-04-23T18:24:41.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.065+0000 7f4a667fc700 1 -- 172.21.15.33:0/2049523475 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f4a6c09c670 con 0x7f4a70102a90 2024-04-23T18:24:41.062 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 msgr2=0x7f4a5c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 0x7f4a5c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000b5c0 tx=0x7f4a60005960 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 msgr2=0x7f4a70074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00b530 tx=0x7f4a6c00b840 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 shutdown_connections 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4a5c0a3cd0 0x7f4a5c0a6190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a70108a50 0x7f4a700757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a70102a90 0x7f4a70074d40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 --2- 172.21.15.33:0/2049523475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70103450 0x7f4a70075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 >> 172.21.15.33:0/2049523475 conn(0x7f4a70079ba0 msgr2=0x7f4a701017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:41.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 shutdown_connections 2024-04-23T18:24:41.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:41.068+0000 7f4a777e1700 1 -- 172.21.15.33:0/2049523475 wait complete. 2024-04-23T18:24:41.942 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":2158874591},{"type":"v1","addr":"172.21.15.33:6801","nonce":2158874591}]},"active_addr":"172.21.15.33:6801/2158874591","active_change":"2024-04-23T18:17:24.068042+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":851272344}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2988933181}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":4092751446}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2434561620}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3971011298}]}]} 2024-04-23T18:24:41.949 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-23T18:24:41.949 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-23T18:24:41.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:24:42.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:41 smithi106 ceph-mon[32716]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-23T18:24:42.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:41 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2049523475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:41 smithi122 ceph-mon[32546]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-23T18:24:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:41 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2049523475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:42.202 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:42.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:41 smithi033 ceph-mon[29274]: pgmap v293: 97 pgs: 97 active+clean; 621 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-04-23T18:24:42.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:41 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2049523475' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-23T18:24:43.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 -- 172.21.15.33:0/3080141523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f378006bbb0 msgr2=0x7f378010f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 --2- 172.21.15.33:0/3080141523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f378006bbb0 0x7f378010f930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f377800b0a0 tx=0x7f377800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 -- 172.21.15.33:0/3080141523 shutdown_connections 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 --2- 172.21.15.33:0/3080141523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f378006bbb0 0x7f378010f930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 --2- 172.21.15.33:0/3080141523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3780108a60 0x7f3780108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 --2- 172.21.15.33:0/3080141523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 0x7f378006b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 -- 172.21.15.33:0/3080141523 >> 172.21.15.33:0/3080141523 conn(0x7f37800789e0 msgr2=0x7f3780078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 -- 172.21.15.33:0/3080141523 shutdown_connections 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.814+0000 7f37856ca700 1 -- 172.21.15.33:0/3080141523 wait complete. 2024-04-23T18:24:43.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 Processor -- start 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 -- start start 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 0x7f37801019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378006bbb0 0x7f3780101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3780112c60 con 0x7f3780108a60 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3780112ae0 con 0x7f378006bbb0 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f37856ca700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3780112de0 con 0x7f378006b1f0 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f377f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f377f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53566/0 (socket says 172.21.15.33:53566) 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f377f7fe700 1 -- 172.21.15.33:0/3551969384 learned_addr learned my addr 172.21.15.33:0/3551969384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:43.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.815+0000 7f377effd700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 0x7f37801019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 -- 172.21.15.33:0/3551969384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 msgr2=0x7f37801019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f3777fff700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378006bbb0 0x7f3780101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 0x7f37801019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 -- 172.21.15.33:0/3551969384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378006bbb0 msgr2=0x7f3780101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378006bbb0 0x7f3780101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 -- 172.21.15.33:0/3551969384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37801a7640 con 0x7f3780108a60 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377f7fe700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f377800b830 tx=0x7f3778004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778021070 con 0x7f3780108a60 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37801a78d0 con 0x7f3780108a60 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778007e30 con 0x7f3780108a60 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37780047d0 con 0x7f3780108a60 2024-04-23T18:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.816+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a7d80 con 0x7f3780108a60 2024-04-23T18:24:43.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.817+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37801041a0 con 0x7f3780108a60 2024-04-23T18:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.818+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3778007980 con 0x7f3780108a60 2024-04-23T18:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.819+0000 7f377cff9700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 0x7f376c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.819+0000 7f377effd700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 0x7f376c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.819+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f37780d13b0 con 0x7f3780108a60 2024-04-23T18:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.819+0000 7f377effd700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 0x7f376c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3770005950 tx=0x7f377000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:43.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:43.822+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f377809de70 con 0x7f3780108a60 2024-04-23T18:24:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.034+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f378002df80 con 0x7f3780108a60 2024-04-23T18:24:44.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.035+0000 7f377cff9700 1 -- 172.21.15.33:0/3551969384 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7f377809d770 con 0x7f3780108a60 2024-04-23T18:24:44.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:44.029 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:24:27.868758+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.037+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 msgr2=0x7f376c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.037+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 0x7f376c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3770005950 tx=0x7f377000a300 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.037+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 msgr2=0x7f3780102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.037+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f377800b830 tx=0x7f3778004ce0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.037+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 shutdown_connections 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f376c0a3d50 0x7f376c0a6210 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3780108a60 0x7f3780102430 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378006b1f0 0x7f37801019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 --2- 172.21.15.33:0/3551969384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378006bbb0 0x7f3780101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 >> 172.21.15.33:0/3551969384 conn(0x7f37800789e0 msgr2=0x7f37800ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 shutdown_connections 2024-04-23T18:24:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:44.038+0000 7f37856ca700 1 -- 172.21.15.33:0/3551969384 wait complete. 2024-04-23T18:24:44.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:43 smithi106 ceph-mon[32716]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-23T18:24:44.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:43 smithi122 ceph-mon[32546]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-23T18:24:44.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:43 smithi033 ceph-mon[29274]: pgmap v294: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 9.3 KiB/s wr, 25 op/s 2024-04-23T18:24:44.758 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-23T18:24:44.758 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-23T18:24:44.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:44 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3551969384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:45.011 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:45.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:44 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3551969384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:45.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:44 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3551969384' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:46.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:45 smithi106 ceph-mon[32716]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-23T18:24:46.162 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:45 smithi033 ceph-mon[29274]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-23T18:24:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:45 smithi122 ceph-mon[32546]: pgmap v295: 97 pgs: 97 active+clean; 630 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 8.4 KiB/s wr, 22 op/s 2024-04-23T18:24:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 -- 172.21.15.33:0/511261377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c108a60 msgr2=0x7fa10c108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 --2- 172.21.15.33:0/511261377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c108a60 0x7fa10c108e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009990 tx=0x7fa0fc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 -- 172.21.15.33:0/511261377 shutdown_connections 2024-04-23T18:24:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 --2- 172.21.15.33:0/511261377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06bbb0 0x7fa10c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 --2- 172.21.15.33:0/511261377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c108a60 0x7fa10c108e40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 --2- 172.21.15.33:0/511261377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c06b1f0 0x7fa10c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 -- 172.21.15.33:0/511261377 >> 172.21.15.33:0/511261377 conn(0x7fa10c0789e0 msgr2=0x7fa10c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 -- 172.21.15.33:0/511261377 shutdown_connections 2024-04-23T18:24:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 -- 172.21.15.33:0/511261377 wait complete. 2024-04-23T18:24:46.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.711+0000 7fa111e10700 1 Processor -- start 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 -- start start 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06b1f0 0x7fa10c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c108a60 0x7fa10c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112ca0 con 0x7fa10c06bbb0 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112b20 con 0x7fa10c108a60 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa111e10700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c112e20 con 0x7fa10c06b1f0 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41162/0 (socket says 172.21.15.33:41162) 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 -- 172.21.15.33:0/4161626023 learned_addr learned my addr 172.21.15.33:0/4161626023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:46.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10bfff700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c108a60 0x7fa10c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10b7fe700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06b1f0 0x7fa10c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 -- 172.21.15.33:0/4161626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06b1f0 msgr2=0x7fa10c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06b1f0 0x7fa10c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 -- 172.21.15.33:0/4161626023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c108a60 msgr2=0x7fa10c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c108a60 0x7fa10c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 -- 172.21.15.33:0/4161626023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c19e140 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.712+0000 7fa10affd700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c920 tx=0x7fa10000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.713+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100007b00 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.713+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa100004ce0 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.713+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c19e430 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.713+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100005650 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.713+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a5cb0 con 0x7fa10c06bbb0 2024-04-23T18:24:46.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.714+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10c10b060 con 0x7fa10c06bbb0 2024-04-23T18:24:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.715+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa100004750 con 0x7fa10c06bbb0 2024-04-23T18:24:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.715+0000 7fa108ff9700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 0x7fa0f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:46.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.716+0000 7fa10b7fe700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 0x7fa0f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.716+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fa1000d0500 con 0x7fa10c06bbb0 2024-04-23T18:24:46.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.716+0000 7fa10b7fe700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 0x7fa0f40a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009750 tx=0x7fa0fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.720+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa100014030 con 0x7fa10c06bbb0 2024-04-23T18:24:46.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.930+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa10c10aa40 con 0x7fa10c06bbb0 2024-04-23T18:24:46.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.932+0000 7fa108ff9700 1 -- 172.21.15.33:0/4161626023 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22244 (secure 0 0 0) 0x7fa10009cf30 con 0x7fa10c06bbb0 2024-04-23T18:24:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:24:46.925 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:24:27.868758+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 msgr2=0x7fa0f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 0x7fa0f40a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009750 tx=0x7fa0fc01a040 comp rx=0 tx=0).stop 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 msgr2=0x7fa10c199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c920 tx=0x7fa10000cce0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 shutdown_connections 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa0f40a3cd0 0x7fa0f40a6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa10c108a60 0x7fa10c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa10c06b1f0 0x7fa10c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 --2- 172.21.15.33:0/4161626023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa10c06bbb0 0x7fa10c199670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 >> 172.21.15.33:0/4161626023 conn(0x7fa10c0789e0 msgr2=0x7fa10c0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 shutdown_connections 2024-04-23T18:24:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:46.934+0000 7fa111e10700 1 -- 172.21.15.33:0/4161626023 wait complete. 2024-04-23T18:24:47.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-23T18:24:47.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-23T18:24:47.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-23T18:24:47.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-23T18:24:47.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-23T18:24:47.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-23T18:24:47.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-23T18:24:47.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-23T18:24:47.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-23T18:24:47.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-23T18:24:47.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-23T18:24:47.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-23T18:24:47.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:47 smithi033 ceph-mon[29274]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-23T18:24:47.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:47 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4161626023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:48.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:47 smithi106 ceph-mon[32716]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-23T18:24:48.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:47 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4161626023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:48.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:47 smithi122 ceph-mon[32546]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-23T18:24:48.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:47 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4161626023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:24:48.313 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.860 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:49.503 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:49.504 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:49.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:49 smithi033 ceph-mon[29274]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-04-23T18:24:50.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:49 smithi106 ceph-mon[32716]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-04-23T18:24:50.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:49 smithi122 ceph-mon[32546]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-04-23T18:24:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 -- 172.21.15.33:0/3257978219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188102a90 msgr2=0x7f8188102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 --2- 172.21.15.33:0/3257978219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188102a90 0x7f8188102f10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f817c009990 tx=0x7f817c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 -- 172.21.15.33:0/3257978219 shutdown_connections 2024-04-23T18:24:51.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 --2- 172.21.15.33:0/3257978219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8188103450 0x7f818810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 --2- 172.21.15.33:0/3257978219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8188108a50 0x7f8188108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 --2- 172.21.15.33:0/3257978219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188102a90 0x7f8188102f10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 -- 172.21.15.33:0/3257978219 >> 172.21.15.33:0/3257978219 conn(0x7f8188079ba0 msgr2=0x7f8188079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 -- 172.21.15.33:0/3257978219 shutdown_connections 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.396+0000 7f818cd86700 1 -- 172.21.15.33:0/3257978219 wait complete. 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 Processor -- start 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 -- start start 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8188102a90 0x7f8188199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8188103450 0x7f8188199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 0x7f818819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112be0 con 0x7f8188108a50 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112a60 con 0x7f8188102a90 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818cd86700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112d60 con 0x7f8188103450 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f8186d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 0x7f818819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f8186d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 0x7f818819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41166/0 (socket says 172.21.15.33:41166) 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f8186d9d700 1 -- 172.21.15.33:0/1679783982 learned_addr learned my addr 172.21.15.33:0/1679783982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f818659c700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8188102a90 0x7f8188199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.397+0000 7f8185d9b700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8188103450 0x7f8188199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 -- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8188103450 msgr2=0x7f8188199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8188103450 0x7f8188199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 -- 172.21.15.33:0/1679783982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8188102a90 msgr2=0x7f8188199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8188102a90 0x7f8188199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818819e160 con 0x7f8188108a50 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f8186d9d700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 0x7f818819da20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f818000ada0 tx=0x7f818000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180017020 con 0x7f8188108a50 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818000cd80 con 0x7f8188108a50 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180005260 con 0x7f8188108a50 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819e450 con 0x7f8188108a50 2024-04-23T18:24:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.398+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a5cd0 con 0x7f8188108a50 2024-04-23T18:24:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.399+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8188063150 con 0x7f8188108a50 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.400+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f818000a080 con 0x7f8188108a50 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f81777fe700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81700a3d50 0x7f81700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f818659c700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81700a3d50 0x7f81700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f81800d0540 con 0x7f8188108a50 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f81777fe700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] conn(0x7f81700ad530 0x7f81700af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8180023e50 con 0x7f81700ad530 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f81800d08f0 con 0x7f8188108a50 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f818659c700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81700a3d50 0x7f81700a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f817800abd0 tx=0x7f817800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.401+0000 7f8185d9b700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] conn(0x7f81700ad530 0x7f81700af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.402+0000 7f8185d9b700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] conn(0x7f81700ad530 0x7f81700af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.403+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== osd.8 v2:172.21.15.122:6816/497939734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8180023e50 con 0x7f81700ad530 2024-04-23T18:24:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.419+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 --> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f81880635d0 con 0x7f81700ad530 2024-04-23T18:24:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f81777fe700 1 -- 172.21.15.33:0/1679783982 <== osd.8 v2:172.21.15.122:6816/497939734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f81880635d0 con 0x7f81700ad530 2024-04-23T18:24:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] conn(0x7f81700ad530 msgr2=0x7f81700af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] conn(0x7f81700ad530 0x7f81700af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81700a3d50 msgr2=0x7f81700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81700a3d50 0x7f81700a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f817800abd0 tx=0x7f817800a380 comp rx=0 tx=0).stop 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 msgr2=0x7f818819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 --2- 172.21.15.33:0/1679783982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8188108a50 0x7f818819da20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f818000ada0 tx=0x7f818000c330 comp rx=0 tx=0).stop 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f8186d9d700 1 -- 172.21.15.33:0/1679783982 reap_dead start 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 shutdown_connections 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 >> 172.21.15.33:0/1679783982 conn(0x7f8188079ba0 msgr2=0x7f8188101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 shutdown_connections 2024-04-23T18:24:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:51.420+0000 7f818cd86700 1 -- 172.21.15.33:0/1679783982 wait complete. 2024-04-23T18:24:52.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:51 smithi106 ceph-mon[32716]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-23T18:24:52.123 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:51 smithi033 ceph-mon[29274]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-23T18:24:52.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:51 smithi122 ceph-mon[32546]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-23T18:24:54.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:53 smithi033 ceph-mon[29274]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:24:54.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:53 smithi106 ceph-mon[32716]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:24:54.150 INFO:teuthology.orchestra.run.smithi033.stdout:253403070498 2024-04-23T18:24:54.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-23T18:24:54.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:53 smithi122 ceph-mon[32546]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-04-23T18:24:55.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.345+0000 7f28a8e20700 1 -- 172.21.15.33:0/2032787494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 msgr2=0x7f28a410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.345+0000 7f28a8e20700 1 --2- 172.21.15.33:0/2032787494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a410e830 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f289c00b0a0 tx=0x7f289c00b3b0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.345+0000 7f28a8e20700 1 -- 172.21.15.33:0/2032787494 shutdown_connections 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 --2- 172.21.15.33:0/2032787494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a410e830 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 --2- 172.21.15.33:0/2032787494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28a4107960 0x7f28a4107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 --2- 172.21.15.33:0/2032787494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 -- 172.21.15.33:0/2032787494 >> 172.21.15.33:0/2032787494 conn(0x7f28a40789e0 msgr2=0x7f28a4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 -- 172.21.15.33:0/2032787494 shutdown_connections 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 -- 172.21.15.33:0/2032787494 wait complete. 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 Processor -- start 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.346+0000 7f28a8e20700 1 -- start start 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a40fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:55.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a40feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28a4107960 0x7f28a40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4111ba0 con 0x7f28a4102320 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4111a20 con 0x7f28a4101960 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a8e20700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f28a4111d20 con 0x7f28a4107960 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a40feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a259c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a40fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a40feed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41176/0 (socket says 172.21.15.33:41176) 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a259c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a40fe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:38944/0 (socket says 172.21.15.33:38944) 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 -- 172.21.15.33:0/3965744162 learned_addr learned my addr 172.21.15.33:0/3965744162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a2d9d700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28a4107960 0x7f28a40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 -- 172.21.15.33:0/3965744162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28a4107960 msgr2=0x7f28a40ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f28a4107960 0x7f28a40ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 -- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 msgr2=0x7f28a40fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a40fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a41a7610 con 0x7f28a4102320 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a259c700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28a4101960 0x7f28a40fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28a1d9b700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a40feed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9a0 tx=0x7f289800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.347+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289800ca10 con 0x7f28a4102320 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.348+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a41a7900 con 0x7f28a4102320 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.348+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289800cbb0 con 0x7f28a4102320 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.348+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a41a7e90 con 0x7f28a4102320 2024-04-23T18:24:55.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.348+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898018680 con 0x7f28a4102320 2024-04-23T18:24:55.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.348+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2884001030 con 0x7f28a4102320 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.350+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2898018820 con 0x7f28a4102320 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28937fe700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f288c0a3cd0 0x7f288c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28a259c700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f288c0a3cd0 0x7f288c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f28980d0990 con 0x7f28a4102320 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28937fe700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] conn(0x7f288c0ad4b0 0x7f288c0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2898018e50 con 0x7f288c0ad4b0 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f28980d0d40 con 0x7f28a4102320 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28a259c700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f288c0a3cd0 0x7f288c0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2894005950 tx=0x7f289400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.351+0000 7f28a2d9d700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] conn(0x7f288c0ad4b0 0x7f288c0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:24:55.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.352+0000 7f28a2d9d700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] conn(0x7f288c0ad4b0 0x7f288c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:24:55.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.352+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== osd.10 v2:172.21.15.106:6824/4120078808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2898018e50 con 0x7f288c0ad4b0 2024-04-23T18:24:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.369+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 --> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2884002e60 con 0x7f288c0ad4b0 2024-04-23T18:24:55.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28937fe700 1 -- 172.21.15.33:0/3965744162 <== osd.10 v2:172.21.15.106:6824/4120078808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2884002e60 con 0x7f288c0ad4b0 2024-04-23T18:24:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] conn(0x7f288c0ad4b0 msgr2=0x7f288c0af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] conn(0x7f288c0ad4b0 0x7f288c0af8d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f288c0a3cd0 msgr2=0x7f288c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f288c0a3cd0 0x7f288c0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2894005950 tx=0x7f289400a300 comp rx=0 tx=0).stop 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 msgr2=0x7f28a40feed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 --2- 172.21.15.33:0/3965744162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a4102320 0x7f28a40feed0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f289800e9a0 tx=0x7f289800ecb0 comp rx=0 tx=0).stop 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a2d9d700 1 -- 172.21.15.33:0/3965744162 reap_dead start 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 shutdown_connections 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 >> 172.21.15.33:0/3965744162 conn(0x7f28a40789e0 msgr2=0x7f28a406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 shutdown_connections 2024-04-23T18:24:55.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:24:55.370+0000 7f28a8e20700 1 -- 172.21.15.33:0/3965744162 wait complete. 2024-04-23T18:24:55.695 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:56.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:55 smithi106 ceph-mon[32716]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:56.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:55 smithi122 ceph-mon[32546]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:55 smithi033 ceph-mon[29274]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:56.592 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:24:58.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:57 smithi106 ceph-mon[32716]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:58.166 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:57 smithi033 ceph-mon[29274]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:58.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:57 smithi122 ceph-mon[32546]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-23T18:24:58.202 INFO:teuthology.orchestra.run.smithi033.stdout:309237645337 2024-04-23T18:24:58.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-23T18:25:00.124 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:24:59 smithi106 ceph-mon[32716]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:00.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:24:59 smithi122 ceph-mon[32546]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:00.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:24:59 smithi033 ceph-mon[29274]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 -- 172.21.15.33:0/2406860227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 msgr2=0x7f268c107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 --2- 172.21.15.33:0/2406860227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c107e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f267c0099c0 tx=0x7f267c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 -- 172.21.15.33:0/2406860227 shutdown_connections 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 --2- 172.21.15.33:0/2406860227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f268c101950 0x7f268c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 --2- 172.21.15.33:0/2406860227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c107e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 --2- 172.21.15.33:0/2406860227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 0x7f268c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.330+0000 7f26930e8700 1 -- 172.21.15.33:0/2406860227 >> 172.21.15.33:0/2406860227 conn(0x7f268c077f40 msgr2=0x7f268c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- 172.21.15.33:0/2406860227 shutdown_connections 2024-04-23T18:25:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- 172.21.15.33:0/2406860227 wait complete. 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 Processor -- start 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- start start 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 0x7f268c102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f268c101950 0x7f268c1026b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f268c112d70 con 0x7f268c107a60 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f268c112bf0 con 0x7f268c100f90 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.331+0000 7f26930e8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f268c112ef0 con 0x7f268c101950 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2690e84700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 0x7f268c102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f268bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f268c101950 0x7f268c1026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2690e84700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 0x7f268c102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:45138/0 (socket says 172.21.15.33:45138) 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2690e84700 1 -- 172.21.15.33:0/3821260502 learned_addr learned my addr 172.21.15.33:0/3821260502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 -- 172.21.15.33:0/3821260502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f268c101950 msgr2=0x7f268c1026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f268c101950 0x7f268c1026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 -- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 msgr2=0x7f268c102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f268c100f90 0x7f268c102170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c1a7700 con 0x7f268c107a60 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2691685700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c102bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f268400be20 tx=0x7f268400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:01.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684019070 con 0x7f268c107a60 2024-04-23T18:25:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2684014440 con 0x7f268c107a60 2024-04-23T18:25:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c1a79f0 con 0x7f268c107a60 2024-04-23T18:25:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.332+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26840125a0 con 0x7f268c107a60 2024-04-23T18:25:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.333+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a7f30 con 0x7f268c107a60 2024-04-23T18:25:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.333+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f268c0630d0 con 0x7f268c107a60 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.335+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f268401c070 con 0x7f268c107a60 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.335+0000 7f2689ffb700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26740a3cd0 0x7f26740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2690e84700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26740a3cd0 0x7f26740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f26840cfff0 con 0x7f268c107a60 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2689ffb700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] conn(0x7f26740ad4b0 0x7f26740af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26740aff80 con 0x7f26740ad4b0 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f26840d0420 con 0x7f268c107a60 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f2690e84700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26740a3cd0 0x7f26740a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f267c009750 tx=0x7f267c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.336+0000 7f268bfff700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] conn(0x7f26740ad4b0 0x7f26740af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.337+0000 7f268bfff700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] conn(0x7f26740ad4b0 0x7f26740af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.337+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== osd.1 v2:172.21.15.106:6800/1202694989 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26740aff80 con 0x7f26740ad4b0 2024-04-23T18:25:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.354+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 --> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f268c0634d0 con 0x7f26740ad4b0 2024-04-23T18:25:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.354+0000 7f2689ffb700 1 -- 172.21.15.33:0/3821260502 <== osd.1 v2:172.21.15.106:6800/1202694989 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f268c0634d0 con 0x7f26740ad4b0 2024-04-23T18:25:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] conn(0x7f26740ad4b0 msgr2=0x7f26740af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] conn(0x7f26740ad4b0 0x7f26740af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26740a3cd0 msgr2=0x7f26740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26740a3cd0 0x7f26740a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f267c009750 tx=0x7f267c005ab0 comp rx=0 tx=0).stop 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 msgr2=0x7f268c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 --2- 172.21.15.33:0/3821260502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f268c107a60 0x7f268c102bf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f268400be20 tx=0x7f268400be50 comp rx=0 tx=0).stop 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f2691685700 1 -- 172.21.15.33:0/3821260502 reap_dead start 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 shutdown_connections 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 >> 172.21.15.33:0/3821260502 conn(0x7f268c077f40 msgr2=0x7f268c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 shutdown_connections 2024-04-23T18:25:01.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.355+0000 7f26930e8700 1 -- 172.21.15.33:0/3821260502 wait complete. 2024-04-23T18:25:02.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:01 smithi106 ceph-mon[32716]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:25:02.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:01 smithi122 ceph-mon[32546]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:25:02.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:01 smithi033 ceph-mon[29274]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 -- 172.21.15.33:0/3356033744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8107c30 msgr2=0x7f92c8108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/3356033744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8107c30 0x7f92c8108010 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f92b80099c0 tx=0x7f92b8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 -- 172.21.15.33:0/3356033744 shutdown_connections 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/3356033744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c8102ed0 0x7f92c810e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/3356033744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8107c30 0x7f92c8108010 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/3356033744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92c8102510 0x7f92c8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 -- 172.21.15.33:0/3356033744 >> 172.21.15.33:0/3356033744 conn(0x7f92c8076c60 msgr2=0x7f92c8077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 -- 172.21.15.33:0/3356033744 shutdown_connections 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.762+0000 7f92ce8a3700 1 -- 172.21.15.33:0/3356033744 wait complete. 2024-04-23T18:25:03.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 Processor -- start 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 -- start start 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 0x7f92c81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92c8102ed0 0x7f92c8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c8107c30 0x7f92c819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8111c50 con 0x7f92c8102510 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8111ad0 con 0x7f92c8102ed0 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.763+0000 7f92ce8a3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8111dd0 con 0x7f92c8107c30 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 0x7f92c81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 0x7f92c81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41910/0 (socket says 172.21.15.33:41910) 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 -- 172.21.15.33:0/880688146 learned_addr learned my addr 172.21.15.33:0/880688146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92cce40700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c8107c30 0x7f92c819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c77fe700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92c8102ed0 0x7f92c8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 -- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c8107c30 msgr2=0x7f92c819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f92c8107c30 0x7f92c819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 -- 172.21.15.33:0/880688146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92c8102ed0 msgr2=0x7f92c8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92c8102ed0 0x7f92c8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c819e1f0 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c7fff700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 0x7f92c81991e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b580 tx=0x7f92b80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8027070 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c819e480 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92b8003f20 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8022d70 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.764+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c81a5d60 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.765+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92c8063150 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.766+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f92b8022500 con 0x7f92c8102510 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c57fa700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92b00a3cd0 0x7f92b00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f92b80d16d0 con 0x7f92c8102510 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c77fe700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92b00a3cd0 0x7f92b00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c57fa700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] conn(0x7f92b00ad380 0x7f92b00af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92b8019e80 con 0x7f92b00ad380 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.767+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f92b80d1bb0 con 0x7f92c8102510 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.768+0000 7f92c77fe700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92b00a3cd0 0x7f92b00a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92c8077340 tx=0x7f92bc00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.768+0000 7f92cce40700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] conn(0x7f92b00ad380 0x7f92b00af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:03.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.768+0000 7f92cce40700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] conn(0x7f92b00ad380 0x7f92b00af7a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.769+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== osd.5 v2:172.21.15.122:6808/2410305978 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92b8019e80 con 0x7f92b00ad380 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.785+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 --> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92c8063550 con 0x7f92b00ad380 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.786+0000 7f92c57fa700 1 -- 172.21.15.33:0/880688146 <== osd.5 v2:172.21.15.122:6808/2410305978 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92c8063550 con 0x7f92b00ad380 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.786+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] conn(0x7f92b00ad380 msgr2=0x7f92b00af7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.786+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] conn(0x7f92b00ad380 0x7f92b00af7a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92b00a3cd0 msgr2=0x7f92b00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92b00a3cd0 0x7f92b00a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f92c8077340 tx=0x7f92bc00b5b0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 msgr2=0x7f92c81991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 --2- 172.21.15.33:0/880688146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92c8102510 0x7f92c81991e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92b800b580 tx=0x7f92b80047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92cce40700 1 -- 172.21.15.33:0/880688146 reap_dead start 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 shutdown_connections 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 >> 172.21.15.33:0/880688146 conn(0x7f92c8076c60 msgr2=0x7f92c8101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 shutdown_connections 2024-04-23T18:25:03.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:01.787+0000 7f92ce8a3700 1 -- 172.21.15.33:0/880688146 wait complete. 2024-04-23T18:25:04.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:03 smithi106 ceph-mon[32716]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:04.126 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:03 smithi033 ceph-mon[29274]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:04.127 INFO:teuthology.orchestra.run.smithi033.stdout:55834574918 2024-04-23T18:25:04.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-23T18:25:04.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:03 smithi122 ceph-mon[32546]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:25:04.861 INFO:teuthology.orchestra.run.smithi033.stdout:163208757298 2024-04-23T18:25:04.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-23T18:25:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 -- 172.21.15.33:0/705313919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4194108a20 msgr2=0x7f4194108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 --2- 172.21.15.33:0/705313919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4194108a20 0x7f4194108e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4184009990 tx=0x7f4184009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 -- 172.21.15.33:0/705313919 shutdown_connections 2024-04-23T18:25:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 --2- 172.21.15.33:0/705313919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f419406b980 0x7f419410f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 --2- 172.21.15.33:0/705313919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4194108a20 0x7f4194108e00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 --2- 172.21.15.33:0/705313919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f419406afe0 0x7f419406b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 -- 172.21.15.33:0/705313919 >> 172.21.15.33:0/705313919 conn(0x7f41940788b0 msgr2=0x7f4194078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 -- 172.21.15.33:0/705313919 shutdown_connections 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.042+0000 7f419b4a1700 1 -- 172.21.15.33:0/705313919 wait complete. 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 Processor -- start 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 -- start start 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f419406afe0 0x7f41941990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 0x7f4194199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4194108a20 0x7f419419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4194112bb0 con 0x7f419406b980 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4194112a30 con 0x7f4194108a20 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419b4a1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4194112d30 con 0x7f419406afe0 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f4198a3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 0x7f4194199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f419923d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f419406afe0 0x7f41941990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f4198a3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 0x7f4194199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41934/0 (socket says 172.21.15.33:41934) 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f4198a3c700 1 -- 172.21.15.33:0/1450065076 learned_addr learned my addr 172.21.15.33:0/1450065076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f4199a3e700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4194108a20 0x7f419419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.043+0000 7f4198a3c700 1 -- 172.21.15.33:0/1450065076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f419406afe0 msgr2=0x7f41941990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4198a3c700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f419406afe0 0x7f41941990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4198a3c700 1 -- 172.21.15.33:0/1450065076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4194108a20 msgr2=0x7f419419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4198a3c700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4194108a20 0x7f419419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4198a3c700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419419e0f0 con 0x7f419406b980 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4199a3e700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4194108a20 0x7f419419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f4198a3c700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 0x7f4194199620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f419000c920 tx=0x7f419000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190007b00 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4190004ce0 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419419e3e0 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190005550 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.044+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941a5c60 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.045+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4194063190 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.046+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4190007ca0 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f418a7fc700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f41800a3d50 0x7f41800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f419923d700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f41800a3d50 0x7f41800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f41900d0500 con 0x7f419406b980 2024-04-23T18:25:06.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f418a7fc700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] conn(0x7f41800ad530 0x7f41800af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41800b0000 con 0x7f41800ad530 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f41900d08b0 con 0x7f419406b980 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f4199a3e700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] conn(0x7f41800ad530 0x7f41800af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.047+0000 7f419923d700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f41800a3d50 0x7f41800a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f418400b5c0 tx=0x7f4184005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.048+0000 7f4199a3e700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] conn(0x7f41800ad530 0x7f41800af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.048+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== osd.3 v2:172.21.15.33:6810/1512542347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41800b0000 con 0x7f41800ad530 2024-04-23T18:25:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.068+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 --> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f41940635c0 con 0x7f41800ad530 2024-04-23T18:25:06.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.068+0000 7f418a7fc700 1 -- 172.21.15.33:0/1450065076 <== osd.3 v2:172.21.15.33:6810/1512542347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f41940635c0 con 0x7f41800ad530 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] conn(0x7f41800ad530 msgr2=0x7f41800af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] conn(0x7f41800ad530 0x7f41800af950 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f41800a3d50 msgr2=0x7f41800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f41800a3d50 0x7f41800a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f418400b5c0 tx=0x7f4184005960 comp rx=0 tx=0).stop 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 msgr2=0x7f4194199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 --2- 172.21.15.33:0/1450065076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f419406b980 0x7f4194199620 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f419000c920 tx=0x7f419000cc30 comp rx=0 tx=0).stop 2024-04-23T18:25:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f4199a3e700 1 -- 172.21.15.33:0/1450065076 reap_dead start 2024-04-23T18:25:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 shutdown_connections 2024-04-23T18:25:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 >> 172.21.15.33:0/1450065076 conn(0x7f41940788b0 msgr2=0x7f41940ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 shutdown_connections 2024-04-23T18:25:06.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.069+0000 7f419b4a1700 1 -- 172.21.15.33:0/1450065076 wait complete. 2024-04-23T18:25:06.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:05 smithi106 ceph-mon[32716]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:06.161 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:05 smithi033 ceph-mon[29274]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:06.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:05 smithi122 ceph-mon[32546]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 -- 172.21.15.33:0/993095555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 msgr2=0x7f0504107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 --2- 172.21.15.33:0/993095555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f0504107d70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f04f8009990 tx=0x7f04f8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 -- 172.21.15.33:0/993095555 shutdown_connections 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 --2- 172.21.15.33:0/993095555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 0x7f050410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 --2- 172.21.15.33:0/993095555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f0504107d70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 --2- 172.21.15.33:0/993095555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 0x7f0504101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 -- 172.21.15.33:0/993095555 >> 172.21.15.33:0/993095555 conn(0x7f05040788f0 msgr2=0x7f0504078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 -- 172.21.15.33:0/993095555 shutdown_connections 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.580+0000 7f050afc0700 1 -- 172.21.15.33:0/993095555 wait complete. 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 Processor -- start 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 -- start start 2024-04-23T18:25:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 0x7f0504197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 0x7f0504198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f050419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0504111ae0 con 0x7f0504107990 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0504111960 con 0x7f0504101940 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050afc0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0504111c60 con 0x7f0504102300 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050955d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f050419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f0503fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 0x7f0504198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050955d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f050419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41956/0 (socket says 172.21.15.33:41956) 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f050955d700 1 -- 172.21.15.33:0/3257717856 learned_addr learned my addr 172.21.15.33:0/3257717856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.581+0000 7f0508d5c700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 0x7f0504197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 -- 172.21.15.33:0/3257717856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 msgr2=0x7f0504198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 0x7f0504198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 -- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 msgr2=0x7f0504197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 0x7f0504197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050419d000 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f0508d5c700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0504101940 0x7f0504197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f0503fff700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0504102300 0x7f0504198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050955d700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f050419c8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00ef30 tx=0x7f04fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0092e0 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc004bb0 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050419d290 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc004280 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.582+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050406a720 con 0x7f0504107990 2024-04-23T18:25:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.583+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f050406ac90 con 0x7f0504107990 2024-04-23T18:25:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.584+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04fc01b030 con 0x7f0504107990 2024-04-23T18:25:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0501ffb700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04f40a3cd0 0x7f04f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f04fc0d0830 con 0x7f0504107990 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0501ffb700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] conn(0x7f04f40ad4b0 0x7f04f40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04f40aff80 con 0x7f04f40ad4b0 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f04fc0d0d10 con 0x7f0504107990 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.585+0000 7f0508d5c700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04f40a3cd0 0x7f04f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.586+0000 7f0503fff700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] conn(0x7f04f40ad4b0 0x7f04f40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.586+0000 7f0508d5c700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04f40a3cd0 0x7f04f40a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b2e0 tx=0x7f04f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.587+0000 7f0503fff700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] conn(0x7f04f40ad4b0 0x7f04f40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.588+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== osd.7 v2:172.21.15.106:6816/4041578359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04f40aff80 con 0x7f04f40ad4b0 2024-04-23T18:25:06.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.605+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 --> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f050406b110 con 0x7f04f40ad4b0 2024-04-23T18:25:06.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.605+0000 7f0501ffb700 1 -- 172.21.15.33:0/3257717856 <== osd.7 v2:172.21.15.106:6816/4041578359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f050406b110 con 0x7f04f40ad4b0 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] conn(0x7f04f40ad4b0 msgr2=0x7f04f40af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] conn(0x7f04f40ad4b0 0x7f04f40af8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04f40a3cd0 msgr2=0x7f04f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04f40a3cd0 0x7f04f40a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f04f800b2e0 tx=0x7f04f801a040 comp rx=0 tx=0).stop 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 msgr2=0x7f050419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 --2- 172.21.15.33:0/3257717856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0504107990 0x7f050419c8c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00ef30 tx=0x7f04fc00c3a0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050955d700 1 -- 172.21.15.33:0/3257717856 reap_dead start 2024-04-23T18:25:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 shutdown_connections 2024-04-23T18:25:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 >> 172.21.15.33:0/3257717856 conn(0x7f05040788f0 msgr2=0x7f0504100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 shutdown_connections 2024-04-23T18:25:06.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.606+0000 7f050afc0700 1 -- 172.21.15.33:0/3257717856 wait complete. 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1492124218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06a750 msgr2=0x7f2e9c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1492124218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06a750 0x7f2e9c06abb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009990 tx=0x7f2e8c009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1492124218 shutdown_connections 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1492124218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1492124218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e9c108a30 0x7f2e9c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1492124218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06a750 0x7f2e9c06abb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1492124218 >> 172.21.15.33:0/1492124218 conn(0x7f2e9c0788f0 msgr2=0x7f2e9c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1492124218 shutdown_connections 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1492124218 wait complete. 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 Processor -- start 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.847+0000 7f2ea3ff9700 1 -- start start 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06a750 0x7f2e9c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e9c108a30 0x7f2e9c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112c20 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112aa0 con 0x7f2e9c06a750 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112da0 con 0x7f2e9c108a30 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43592/0 (socket says 172.21.15.33:43592) 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 -- 172.21.15.33:0/1495623793 learned_addr learned my addr 172.21.15.33:0/1495623793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea2596700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e9c108a30 0x7f2e9c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1d95700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06a750 0x7f2e9c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 -- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e9c108a30 msgr2=0x7f2e9c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e9c108a30 0x7f2e9c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 -- 172.21.15.33:0/1495623793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06a750 msgr2=0x7f2e9c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06a750 0x7f2e9c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c19e190 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1d95700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e9c06a750 0x7f2e9c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea1594700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c199710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c000c00 tx=0x7f2e8c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c027070 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e8c022470 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9c19e420 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c017650 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.848+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9c1a5d00 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.849+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2e9c0631d0 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.851+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2e8c02a070 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.851+0000 7f2e92ffd700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2e880a3cd0 0x7f2e880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f2e8c0d13f0 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2ea1d95700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2e880a3cd0 0x7f2e880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2e92ffd700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] conn(0x7f2e880ad4b0 0x7f2e880af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e880aff80 con 0x7f2e880ad4b0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f2e8c0d18d0 con 0x7f2e9c06b0f0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2ea2596700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] conn(0x7f2e880ad4b0 0x7f2e880af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2ea1d95700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2e880a3cd0 0x7f2e880a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009c90 tx=0x7f2e98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.852+0000 7f2ea2596700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] conn(0x7f2e880ad4b0 0x7f2e880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:06.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.853+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== osd.11 v2:172.21.15.122:6824/3755562411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2e880aff80 con 0x7f2e880ad4b0 2024-04-23T18:25:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 --> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2e9c063650 con 0x7f2e880ad4b0 2024-04-23T18:25:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2e92ffd700 1 -- 172.21.15.33:0/1495623793 <== osd.11 v2:172.21.15.122:6824/3755562411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2e9c063650 con 0x7f2e880ad4b0 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] conn(0x7f2e880ad4b0 msgr2=0x7f2e880af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] conn(0x7f2e880ad4b0 0x7f2e880af8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2e880a3cd0 msgr2=0x7f2e880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2e880a3cd0 0x7f2e880a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009c90 tx=0x7f2e98009260 comp rx=0 tx=0).stop 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 msgr2=0x7f2e9c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 --2- 172.21.15.33:0/1495623793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e9c06b0f0 0x7f2e9c199710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c000c00 tx=0x7f2e8c0057c0 comp rx=0 tx=0).stop 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea2596700 1 -- 172.21.15.33:0/1495623793 reap_dead start 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 shutdown_connections 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 >> 172.21.15.33:0/1495623793 conn(0x7f2e9c0788f0 msgr2=0x7f2e9c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 shutdown_connections 2024-04-23T18:25:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:06.870+0000 7f2ea3ff9700 1 -- 172.21.15.33:0/1495623793 wait complete. 2024-04-23T18:25:07.519 INFO:teuthology.orchestra.run.smithi033.stdout:219043332136 2024-04-23T18:25:07.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-23T18:25:08.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:07 smithi033 ceph-mon[29274]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:08.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:07 smithi106 ceph-mon[32716]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:08.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:07 smithi122 ceph-mon[32546]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 -- 172.21.15.33:0/3365399947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c108ab0 msgr2=0x7f485c108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 --2- 172.21.15.33:0/3365399947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c108ab0 0x7f485c108e90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4854009990 tx=0x7f4854009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 -- 172.21.15.33:0/3365399947 shutdown_connections 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 --2- 172.21.15.33:0/3365399947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f485c103480 0x7f485c10f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 --2- 172.21.15.33:0/3365399947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c108ab0 0x7f485c108e90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 --2- 172.21.15.33:0/3365399947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f485c102ac0 0x7f485c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 -- 172.21.15.33:0/3365399947 >> 172.21.15.33:0/3365399947 conn(0x7f485c079bf0 msgr2=0x7f485c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 -- 172.21.15.33:0/3365399947 shutdown_connections 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.916+0000 7f4864035700 1 -- 172.21.15.33:0/3365399947 wait complete. 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 Processor -- start 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 -- start start 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 0x7f485c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f485c103480 0x7f485c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f485c108ab0 0x7f485c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f485c112cb0 con 0x7f485c102ac0 2024-04-23T18:25:08.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f485c112b30 con 0x7f485c108ab0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4864035700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f485c112e30 con 0x7f485c103480 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 0x7f485c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 0x7f485c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43622/0 (socket says 172.21.15.33:43622) 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 -- 172.21.15.33:0/4153239646 learned_addr learned my addr 172.21.15.33:0/4153239646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f48625d2700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f485c108ab0 0x7f485c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f48615d0700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f485c103480 0x7f485c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 -- 172.21.15.33:0/4153239646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f485c103480 msgr2=0x7f485c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f485c103480 0x7f485c075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 -- 172.21.15.33:0/4153239646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f485c108ab0 msgr2=0x7f485c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f485c108ab0 0x7f485c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485c1a76b0 con 0x7f485c102ac0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.917+0000 7f4861dd1700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 0x7f485c074d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f485400b5c0 tx=0x7f48540058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854027070 con 0x7f485c102ac0 2024-04-23T18:25:08.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854003fb0 con 0x7f485c102ac0 2024-04-23T18:25:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485c1a7940 con 0x7f485c102ac0 2024-04-23T18:25:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854022ce0 con 0x7f485c102ac0 2024-04-23T18:25:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f485c1a7df0 con 0x7f485c102ac0 2024-04-23T18:25:08.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.918+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f485c0630d0 con 0x7f485c102ac0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.920+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f485402a070 con 0x7f485c102ac0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.920+0000 7f485affd700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f48440a3d50 0x7f48440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f48615d0700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f48440a3d50 0x7f48440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f48540d1cb0 con 0x7f485c102ac0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f485affd700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] conn(0x7f48440ad530 0x7f48440af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48440b0000 con 0x7f48440ad530 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f48540d2190 con 0x7f485c102ac0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f48625d2700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] conn(0x7f48440ad530 0x7f48440af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:08.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f48615d0700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f48440a3d50 0x7f48440a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f485c07a260 tx=0x7f4850005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.921+0000 7f48625d2700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] conn(0x7f48440ad530 0x7f48440af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:08.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.922+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== osd.6 v2:172.21.15.33:6818/3512939411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f48440b0000 con 0x7f48440ad530 2024-04-23T18:25:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.938+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 --> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f485c0634d0 con 0x7f48440ad530 2024-04-23T18:25:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f485affd700 1 -- 172.21.15.33:0/4153239646 <== osd.6 v2:172.21.15.33:6818/3512939411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f485c0634d0 con 0x7f48440ad530 2024-04-23T18:25:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] conn(0x7f48440ad530 msgr2=0x7f48440af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] conn(0x7f48440ad530 0x7f48440af950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f48440a3d50 msgr2=0x7f48440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f48440a3d50 0x7f48440a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f485c07a260 tx=0x7f4850005cf0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 msgr2=0x7f485c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 --2- 172.21.15.33:0/4153239646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c102ac0 0x7f485c074d40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f485400b5c0 tx=0x7f48540058e0 comp rx=0 tx=0).stop 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f48625d2700 1 -- 172.21.15.33:0/4153239646 reap_dead start 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 shutdown_connections 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 >> 172.21.15.33:0/4153239646 conn(0x7f485c079bf0 msgr2=0x7f485c100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 shutdown_connections 2024-04-23T18:25:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:08.939+0000 7f4864035700 1 -- 172.21.15.33:0/4153239646 wait complete. 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.142+0000 7f1e6d551700 1 -- 172.21.15.33:0/2875895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e681022f0 msgr2=0x7f1e6810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.142+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2875895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e681022f0 0x7f1e6810e7e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b0a0 tx=0x7f1e6000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 -- 172.21.15.33:0/2875895500 shutdown_connections 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2875895500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e681022f0 0x7f1e6810e7e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2875895500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68107980 0x7f1e68107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2875895500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e68101930 0x7f1e68101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 -- 172.21.15.33:0/2875895500 >> 172.21.15.33:0/2875895500 conn(0x7f1e680788b0 msgr2=0x7f1e68078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 -- 172.21.15.33:0/2875895500 shutdown_connections 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 -- 172.21.15.33:0/2875895500 wait complete. 2024-04-23T18:25:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 Processor -- start 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.143+0000 7f1e6d551700 1 -- start start 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68101930 0x7f1e68198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e681022f0 0x7f1e68198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 0x7f1e6819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68111af0 con 0x7f1e68107980 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68111970 con 0x7f1e681022f0 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e68111c70 con 0x7f1e68101930 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 0x7f1e6819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 0x7f1e6819c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43640/0 (socket says 172.21.15.33:43640) 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 -- 172.21.15.33:0/2355388650 learned_addr learned my addr 172.21.15.33:0/2355388650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e667fc700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e681022f0 0x7f1e68198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e66ffd700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68101930 0x7f1e68198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 -- 172.21.15.33:0/2355388650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68101930 msgr2=0x7f1e68198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e68101930 0x7f1e68198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 -- 172.21.15.33:0/2355388650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e681022f0 msgr2=0x7f1e68198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e681022f0 0x7f1e68198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6819d010 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e677fe700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 0x7f1e6819c8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b830 tx=0x7f1e60004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60021070 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6819d2a0 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e60007e30 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60004700 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.144+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e681a4b80 con 0x7f1e68107980 2024-04-23T18:25:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.145+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e68063150 con 0x7f1e68107980 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.146+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e60003d70 con 0x7f1e68107980 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e5ffff700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e540a3cd0 0x7f1e540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e66ffd700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e540a3cd0 0x7f1e540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f1e6000f020 con 0x7f1e68107980 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e5ffff700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] conn(0x7f1e540ad4b0 0x7f1e540af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e540aff80 con 0x7f1e540ad4b0 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.147+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f1e600d1f40 con 0x7f1e68107980 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.148+0000 7f1e667fc700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] conn(0x7f1e540ad4b0 0x7f1e540af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.148+0000 7f1e66ffd700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e540a3cd0 0x7f1e540a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e580045b0 tx=0x7f1e5800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.148+0000 7f1e667fc700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] conn(0x7f1e540ad4b0 0x7f1e540af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.149+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== osd.9 v2:172.21.15.33:6826/3157866333 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e540aff80 con 0x7f1e540ad4b0 2024-04-23T18:25:09.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.165+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 --> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e680635d0 con 0x7f1e540ad4b0 2024-04-23T18:25:09.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e5ffff700 1 -- 172.21.15.33:0/2355388650 <== osd.9 v2:172.21.15.33:6826/3157866333 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e680635d0 con 0x7f1e540ad4b0 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] conn(0x7f1e540ad4b0 msgr2=0x7f1e540af8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] conn(0x7f1e540ad4b0 0x7f1e540af8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e540a3cd0 msgr2=0x7f1e540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1e540a3cd0 0x7f1e540a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1e580045b0 tx=0x7f1e5800b3c0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 msgr2=0x7f1e6819c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 --2- 172.21.15.33:0/2355388650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e68107980 0x7f1e6819c8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000b830 tx=0x7f1e60004ce0 comp rx=0 tx=0).stop 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e677fe700 1 -- 172.21.15.33:0/2355388650 reap_dead start 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 shutdown_connections 2024-04-23T18:25:09.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.166+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 >> 172.21.15.33:0/2355388650 conn(0x7f1e680788b0 msgr2=0x7f1e680ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.167+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 shutdown_connections 2024-04-23T18:25:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:09.167+0000 7f1e6d551700 1 -- 172.21.15.33:0/2355388650 wait complete. 2024-04-23T18:25:10.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:09 smithi033 ceph-mon[29274]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:10.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:09 smithi122 ceph-mon[32546]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:10.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:09 smithi106 ceph-mon[32716]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 -- 172.21.15.33:0/3289950356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 msgr2=0x7fbf98111860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 --2- 172.21.15.33:0/3289950356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf98111860 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009990 tx=0x7fbf88009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 -- 172.21.15.33:0/3289950356 shutdown_connections 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 --2- 172.21.15.33:0/3289950356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf98111860 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 --2- 172.21.15.33:0/3289950356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98074c70 0x7fbf98075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 --2- 172.21.15.33:0/3289950356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf98075590 0x7fbf9810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.285+0000 7fbf9f686700 1 -- 172.21.15.33:0/3289950356 >> 172.21.15.33:0/3289950356 conn(0x7fbf98079bf0 msgr2=0x7fbf9807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- 172.21.15.33:0/3289950356 shutdown_connections 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- 172.21.15.33:0/3289950356 wait complete. 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 Processor -- start 2024-04-23T18:25:10.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- start start 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf98074c70 0x7fbf9819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98075590 0x7fbf9819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf9819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98114eb0 con 0x7fbf9810f470 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98114d30 con 0x7fbf98074c70 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.286+0000 7fbf9f686700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf98115030 con 0x7fbf98075590 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9dc23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf9819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9dc23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf9819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43666/0 (socket says 172.21.15.33:43666) 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9dc23700 1 -- 172.21.15.33:0/466889438 learned_addr learned my addr 172.21.15.33:0/466889438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98075590 0x7fbf9819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9d422700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf98074c70 0x7fbf9819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 -- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf98074c70 msgr2=0x7fbf9819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf98074c70 0x7fbf9819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 -- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 msgr2=0x7fbf9819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf9819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf981a0390 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9dc23700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf9810f470 0x7fbf9819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf9cc21700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98075590 0x7fbf9819b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000ada0 tx=0x7fbf9000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.287+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90017020 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.288+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9000cd80 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.288+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf981a0680 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.288+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf90005280 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.288+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf981a7f00 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.289+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf98063110 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.290+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf9000a080 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.290+0000 7fbf8e7fc700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf840a3d50 0x7fbf840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf9d422700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf840a3d50 0x7fbf840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbf900cfde0 con 0x7fbf98075590 2024-04-23T18:25:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf8e7fc700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] conn(0x7fbf840ad530 0x7fbf840af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf840b0000 con 0x7fbf840ad530 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fbf900d0190 con 0x7fbf98075590 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf9d422700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf840a3d50 0x7fbf840a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.291+0000 7fbf9dc23700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] conn(0x7fbf840ad530 0x7fbf840af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.292+0000 7fbf9dc23700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] conn(0x7fbf840ad530 0x7fbf840af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.292+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== osd.4 v2:172.21.15.106:6808/2880981819 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf840b0000 con 0x7fbf840ad530 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.309+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 --> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf98063510 con 0x7fbf840ad530 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.309+0000 7fbf8e7fc700 1 -- 172.21.15.33:0/466889438 <== osd.4 v2:172.21.15.106:6808/2880981819 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf98063510 con 0x7fbf840ad530 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] conn(0x7fbf840ad530 msgr2=0x7fbf840af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] conn(0x7fbf840ad530 0x7fbf840af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf840a3d50 msgr2=0x7fbf840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf840a3d50 0x7fbf840a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400a300 comp rx=0 tx=0).stop 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98075590 msgr2=0x7fbf9819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 --2- 172.21.15.33:0/466889438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf98075590 0x7fbf9819b8c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000ada0 tx=0x7fbf9000c330 comp rx=0 tx=0).stop 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9dc23700 1 -- 172.21.15.33:0/466889438 reap_dead start 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 shutdown_connections 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 >> 172.21.15.33:0/466889438 conn(0x7fbf98079bf0 msgr2=0x7fbf980ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 shutdown_connections 2024-04-23T18:25:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.310+0000 7fbf9f686700 1 -- 172.21.15.33:0/466889438 wait complete. 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 -- 172.21.15.33:0/1526261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 msgr2=0x7f04dc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1526261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc102f10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009990 tx=0x7f04c8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 -- 172.21.15.33:0/1526261040 shutdown_connections 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1526261040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04dc103450 0x7f04dc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1526261040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc108a50 0x7f04dc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1526261040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc102f10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 -- 172.21.15.33:0/1526261040 >> 172.21.15.33:0/1526261040 conn(0x7f04dc079ba0 msgr2=0x7f04dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.908+0000 7f04e2adb700 1 -- 172.21.15.33:0/1526261040 shutdown_connections 2024-04-23T18:25:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 -- 172.21.15.33:0/1526261040 wait complete. 2024-04-23T18:25:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 Processor -- start 2024-04-23T18:25:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 -- start start 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04dc103450 0x7f04dc19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc108a50 0x7f04dc19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc112c60 con 0x7f04dc102a90 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc112ae0 con 0x7f04dc103450 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e2adb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc112de0 con 0x7f04dc108a50 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.909+0000 7f04e0877700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43676/0 (socket says 172.21.15.33:43676) 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 -- 172.21.15.33:0/1123030721 learned_addr learned my addr 172.21.15.33:0/1123030721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e1078700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc108a50 0x7f04dc19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04dbfff700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04dc103450 0x7f04dc19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 -- 172.21.15.33:0/1123030721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc108a50 msgr2=0x7f04dc19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc108a50 0x7f04dc19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 -- 172.21.15.33:0/1123030721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04dc103450 msgr2=0x7f04dc19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04dc103450 0x7f04dc19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc1a03e0 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e0877700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc19b3d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b530 tx=0x7f04d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0015020 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc1a06d0 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04d0011ce0 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0010e40 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.910+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a7f50 con 0x7f04dc102a90 2024-04-23T18:25:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.911+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04dc063150 con 0x7f04dc102a90 2024-04-23T18:25:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.912+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04d0018020 con 0x7f04dc102a90 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04d9ffb700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04cc0a3d50 0x7f04cc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04dbfff700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04cc0a3d50 0x7f04cc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f04d00d0180 con 0x7f04dc102a90 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04d9ffb700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] conn(0x7f04cc0ad530 0x7f04cc0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04cc0b0000 con 0x7f04cc0ad530 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f04d00d0660 con 0x7f04dc102a90 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.913+0000 7f04e1078700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] conn(0x7f04cc0ad530 0x7f04cc0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.914+0000 7f04dbfff700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04cc0a3d50 0x7f04cc0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006010 tx=0x7f04c801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.914+0000 7f04e1078700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] conn(0x7f04cc0ad530 0x7f04cc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:10.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.914+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== osd.0 v2:172.21.15.33:6802/3729126407 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04cc0b0000 con 0x7f04cc0ad530 2024-04-23T18:25:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.931+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 --> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04dc063550 con 0x7f04cc0ad530 2024-04-23T18:25:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.931+0000 7f04d9ffb700 1 -- 172.21.15.33:0/1123030721 <== osd.0 v2:172.21.15.33:6802/3729126407 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f04dc063550 con 0x7f04cc0ad530 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.931+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] conn(0x7f04cc0ad530 msgr2=0x7f04cc0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.931+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] conn(0x7f04cc0ad530 0x7f04cc0af950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04cc0a3d50 msgr2=0x7f04cc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f04cc0a3d50 0x7f04cc0a6210 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f04c8006010 tx=0x7f04c801a040 comp rx=0 tx=0).stop 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 msgr2=0x7f04dc19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 --2- 172.21.15.33:0/1123030721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04dc102a90 0x7f04dc19b3d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f04d000b530 tx=0x7f04d000b840 comp rx=0 tx=0).stop 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e1078700 1 -- 172.21.15.33:0/1123030721 reap_dead start 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 shutdown_connections 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 >> 172.21.15.33:0/1123030721 conn(0x7f04dc079ba0 msgr2=0x7f04dc10df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 shutdown_connections 2024-04-23T18:25:10.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:10.932+0000 7f04e2adb700 1 -- 172.21.15.33:0/1123030721 wait complete. 2024-04-23T18:25:11.668 INFO:teuthology.orchestra.run.smithi033.stdout:287762808865 2024-04-23T18:25:11.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-23T18:25:11.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:11 smithi033 ceph-mon[29274]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:25:12.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:11 smithi122 ceph-mon[32546]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:25:12.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:11 smithi106 ceph-mon[32716]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:25:12.425 INFO:teuthology.orchestra.run.smithi033.stdout:141733920826 2024-04-23T18:25:12.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-23T18:25:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 -- 172.21.15.33:0/2795298299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84103480 msgr2=0x7fbf8410f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/2795298299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84103480 0x7fbf8410f980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009990 tx=0x7fbf74009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 -- 172.21.15.33:0/2795298299 shutdown_connections 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/2795298299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84103480 0x7fbf8410f980 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/2795298299 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf84108ab0 0x7fbf84108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/2795298299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf84102ac0 0x7fbf84102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 -- 172.21.15.33:0/2795298299 >> 172.21.15.33:0/2795298299 conn(0x7fbf84079bf0 msgr2=0x7fbf8407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 -- 172.21.15.33:0/2795298299 shutdown_connections 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.872+0000 7fbf8c31f700 1 -- 172.21.15.33:0/2795298299 wait complete. 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 Processor -- start 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 -- start start 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 0x7fbf84074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf84103480 0x7fbf84075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf84108ab0 0x7fbf840757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84112ca0 con 0x7fbf84102ac0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84112b20 con 0x7fbf84108ab0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8c31f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf84112e20 con 0x7fbf84103480 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8a0bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 0x7fbf84074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8a0bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 0x7fbf84074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43700/0 (socket says 172.21.15.33:43700) 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf8a0bb700 1 -- 172.21.15.33:0/1198051229 learned_addr learned my addr 172.21.15.33:0/1198051229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.873+0000 7fbf898ba700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf84103480 0x7fbf84075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a8bc700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf84108ab0 0x7fbf840757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 -- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf84103480 msgr2=0x7fbf84075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf84103480 0x7fbf84075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 -- 172.21.15.33:0/1198051229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf84108ab0 msgr2=0x7fbf840757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf84108ab0 0x7fbf840757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf841a77e0 con 0x7fbf84102ac0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8a0bb700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 0x7fbf84074d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0060b0 tx=0x7fbf7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c01a070 con 0x7fbf84102ac0 2024-04-23T18:25:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7c00ddd0 con 0x7fbf84102ac0 2024-04-23T18:25:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf841a7ad0 con 0x7fbf84102ac0 2024-04-23T18:25:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf841a8060 con 0x7fbf84102ac0 2024-04-23T18:25:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.874+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c014720 con 0x7fbf84102ac0 2024-04-23T18:25:12.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.875+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf84063150 con 0x7fbf84102ac0 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.876+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbf7c01d070 con 0x7fbf84102ac0 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf837fe700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf6c0a3d50 0x7fbf6c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf898ba700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf6c0a3d50 0x7fbf6c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fbf7c0d0340 con 0x7fbf84102ac0 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf837fe700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] conn(0x7fbf6c0ad530 0x7fbf6c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf6c0b0000 con 0x7fbf6c0ad530 2024-04-23T18:25:12.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fbf7c0d0820 con 0x7fbf84102ac0 2024-04-23T18:25:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf898ba700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf6c0a3d50 0x7fbf6c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf8407a180 tx=0x7fbf7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.877+0000 7fbf8a8bc700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] conn(0x7fbf6c0ad530 0x7fbf6c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.878+0000 7fbf8a8bc700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] conn(0x7fbf6c0ad530 0x7fbf6c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:12.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.879+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== osd.2 v2:172.21.15.122:6800/2314180839 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf6c0b0000 con 0x7fbf6c0ad530 2024-04-23T18:25:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.895+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 --> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf84063550 con 0x7fbf6c0ad530 2024-04-23T18:25:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.896+0000 7fbf837fe700 1 -- 172.21.15.33:0/1198051229 <== osd.2 v2:172.21.15.122:6800/2314180839 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbf84063550 con 0x7fbf6c0ad530 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.896+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] conn(0x7fbf6c0ad530 msgr2=0x7fbf6c0af950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.896+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] conn(0x7fbf6c0ad530 0x7fbf6c0af950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.896+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf6c0a3d50 msgr2=0x7fbf6c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.896+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbf6c0a3d50 0x7fbf6c0a6210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbf8407a180 tx=0x7fbf7800b410 comp rx=0 tx=0).stop 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 msgr2=0x7fbf84074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 --2- 172.21.15.33:0/1198051229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf84102ac0 0x7fbf84074d40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0060b0 tx=0x7fbf7c00d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8a8bc700 1 -- 172.21.15.33:0/1198051229 reap_dead start 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 shutdown_connections 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 >> 172.21.15.33:0/1198051229 conn(0x7fbf84079bf0 msgr2=0x7fbf84100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 shutdown_connections 2024-04-23T18:25:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:12.897+0000 7fbf8c31f700 1 -- 172.21.15.33:0/1198051229 wait complete. 2024-04-23T18:25:13.715 INFO:teuthology.orchestra.run.smithi033.stdout:197568495663 2024-04-23T18:25:13.716 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-23T18:25:13.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:13 smithi033 ceph-mon[29274]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:25:14.118 INFO:teuthology.orchestra.run.smithi033.stdout:343597383701 2024-04-23T18:25:14.119 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-23T18:25:14.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:13 smithi122 ceph-mon[32546]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:25:14.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:13 smithi106 ceph-mon[32716]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-23T18:25:14.573 INFO:teuthology.orchestra.run.smithi033.stdout:107374182461 2024-04-23T18:25:14.574 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-23T18:25:14.636 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:15.069 INFO:teuthology.orchestra.run.smithi033.stdout:34359738444 2024-04-23T18:25:15.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-23T18:25:15.815 INFO:teuthology.orchestra.run.smithi033.stdout:77309411396 2024-04-23T18:25:15.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-23T18:25:15.883 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:16.094 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:15 smithi033 ceph-mon[29274]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:15 smithi122 ceph-mon[32546]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:15 smithi106 ceph-mon[32716]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:17.970 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:17 smithi033 ceph-mon[29274]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:18.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:17 smithi122 ceph-mon[32546]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:18.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:17 smithi106 ceph-mon[32716]: pgmap v311: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:19.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 -- 172.21.15.33:0/1528146215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 msgr2=0x7f078c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 --2- 172.21.15.33:0/1528146215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c102f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0780009990 tx=0x7f0780009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 -- 172.21.15.33:0/1528146215 shutdown_connections 2024-04-23T18:25:19.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 --2- 172.21.15.33:0/1528146215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c103480 0x7f078c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 --2- 172.21.15.33:0/1528146215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c108a70 0x7f078c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 --2- 172.21.15.33:0/1528146215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c102f40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 -- 172.21.15.33:0/1528146215 >> 172.21.15.33:0/1528146215 conn(0x7f078c079bb0 msgr2=0x7f078c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 -- 172.21.15.33:0/1528146215 shutdown_connections 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.369+0000 7f07911f0700 1 -- 172.21.15.33:0/1528146215 wait complete. 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.370+0000 7f07911f0700 1 Processor -- start 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.370+0000 7f07911f0700 1 -- start start 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c103480 0x7f078c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:19.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c108a70 0x7f078c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f078c112c40 con 0x7f078c102ac0 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f078c112ac0 con 0x7f078c103480 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f07911f0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f078c112dc0 con 0x7f078c108a70 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52798/0 (socket says 172.21.15.33:52798) 2024-04-23T18:25:19.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 -- 172.21.15.33:0/2384147878 learned_addr learned my addr 172.21.15.33:0/2384147878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078a59c700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c103480 0x7f078c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078b59e700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c108a70 0x7f078c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 -- 172.21.15.33:0/2384147878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c108a70 msgr2=0x7f078c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c108a70 0x7f078c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 -- 172.21.15.33:0/2384147878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c103480 msgr2=0x7f078c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c103480 0x7f078c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 -- 172.21.15.33:0/2384147878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f078c19e0d0 con 0x7f078c102ac0 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f078ad9d700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d950 tx=0x7f077c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.371+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c016070 con 0x7f078c102ac0 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.372+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f078c19e3c0 con 0x7f078c102ac0 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.372+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077c00fd40 con 0x7f078c102ac0 2024-04-23T18:25:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.372+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00e3b0 con 0x7f078c102ac0 2024-04-23T18:25:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.372+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f078c1a5c40 con 0x7f078c102ac0 2024-04-23T18:25:19.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.373+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f078c069730 con 0x7f078c102ac0 2024-04-23T18:25:19.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.374+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f077c019070 con 0x7f078c102ac0 2024-04-23T18:25:19.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.375+0000 7f0773fff700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 0x7f07740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.375+0000 7f078a59c700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 0x7f07740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.375+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f077c0cfec0 con 0x7f078c102ac0 2024-04-23T18:25:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.376+0000 7f078a59c700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 0x7f07740a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f078000b5c0 tx=0x7f0780005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:19.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.379+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f077c09c7f0 con 0x7f078c102ac0 2024-04-23T18:25:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.590+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f078c02df80 con 0x7f078c102ac0 2024-04-23T18:25:19.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.590+0000 7f0773fff700 1 -- 172.21.15.33:0/2384147878 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f077c09c0f0 con 0x7f078c102ac0 2024-04-23T18:25:19.583 INFO:teuthology.orchestra.run.smithi033.stdout:309237645341 2024-04-23T18:25:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 msgr2=0x7f07740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 0x7f07740a6190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f078000b5c0 tx=0x7f0780005960 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 msgr2=0x7f078c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f077c00d950 tx=0x7f077c00dc60 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 shutdown_connections 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07740a3cd0 0x7f07740a6190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f078c108a70 0x7f078c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c102ac0 0x7f078c199110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 --2- 172.21.15.33:0/2384147878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f078c103480 0x7f078c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.592+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 >> 172.21.15.33:0/2384147878 conn(0x7f078c079bb0 msgr2=0x7f078c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.593+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 shutdown_connections 2024-04-23T18:25:19.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:19.593+0000 7f07911f0700 1 -- 172.21.15.33:0/2384147878 wait complete. 2024-04-23T18:25:19.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 -- 172.21.15.33:0/4235647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538103450 msgr2=0x7f253810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 --2- 172.21.15.33:0/4235647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538103450 0x7f253810f8b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f253000b0a0 tx=0x7f253000b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 -- 172.21.15.33:0/4235647169 shutdown_connections 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 --2- 172.21.15.33:0/4235647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538103450 0x7f253810f8b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 --2- 172.21.15.33:0/4235647169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538108a50 0x7f2538108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 --2- 172.21.15.33:0/4235647169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538102a90 0x7f2538102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 -- 172.21.15.33:0/4235647169 >> 172.21.15.33:0/4235647169 conn(0x7f2538079ba0 msgr2=0x7f2538079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 -- 172.21.15.33:0/4235647169 shutdown_connections 2024-04-23T18:25:19.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.005+0000 7f253ed13700 1 -- 172.21.15.33:0/4235647169 wait complete. 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.006+0000 7f253ed13700 1 Processor -- start 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.006+0000 7f253ed13700 1 -- start start 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.006+0000 7f253ed13700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538102a90 0x7f25381991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538103450 0x7f2538199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112c80 con 0x7f2538108a50 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112b00 con 0x7f2538102a90 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112e00 con 0x7f2538103450 2024-04-23T18:25:20.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52830/0 (socket says 172.21.15.33:52830) 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 -- 172.21.15.33:0/3848038976 learned_addr learned my addr 172.21.15.33:0/3848038976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f2537fff700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538103450 0x7f2538199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253caaf700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538102a90 0x7f25381991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 -- 172.21.15.33:0/3848038976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538103450 msgr2=0x7f2538199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538103450 0x7f2538199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 -- 172.21.15.33:0/3848038976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538102a90 msgr2=0x7f25381991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538102a90 0x7f25381991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 -- 172.21.15.33:0/3848038976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253819e200 con 0x7f2538108a50 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253d2b0700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f253000b830 tx=0x7f2530004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530021070 con 0x7f2538108a50 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2530007e30 con 0x7f2538108a50 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253819e490 con 0x7f2538108a50 2024-04-23T18:25:20.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.007+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25300047d0 con 0x7f2538108a50 2024-04-23T18:25:20.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.008+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a5d70 con 0x7f2538108a50 2024-04-23T18:25:20.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.008+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538069730 con 0x7f2538108a50 2024-04-23T18:25:20.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.010+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2530007980 con 0x7f2538108a50 2024-04-23T18:25:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.010+0000 7f2535ffb700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 0x7f25280a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.010+0000 7f253caaf700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 0x7f25280a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.010+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f253000f020 con 0x7f2538108a50 2024-04-23T18:25:20.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.011+0000 7f253caaf700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 0x7f25280a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:20.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.014+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f253002a020 con 0x7f2538108a50 2024-04-23T18:25:20.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:19 smithi033 ceph-mon[29274]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:20.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2384147878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:25:20.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:19 smithi122 ceph-mon[32546]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:20.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2384147878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:25:20.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.231+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f253802df80 con 0x7f2538108a50 2024-04-23T18:25:20.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.231+0000 7f2535ffb700 1 -- 172.21.15.33:0/3848038976 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f253009e820 con 0x7f2538108a50 2024-04-23T18:25:20.224 INFO:teuthology.orchestra.run.smithi033.stdout:253403070503 2024-04-23T18:25:20.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 msgr2=0x7f25280a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 0x7f25280a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c008040 comp rx=0 tx=0).stop 2024-04-23T18:25:20.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 msgr2=0x7f253819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f253000b830 tx=0x7f2530004ce0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 shutdown_connections 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f25280a3cd0 0x7f25280a6190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2538108a50 0x7f253819dac0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2538102a90 0x7f25381991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 --2- 172.21.15.33:0/3848038976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2538103450 0x7f2538199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 >> 172.21.15.33:0/3848038976 conn(0x7f2538079ba0 msgr2=0x7f2538101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 shutdown_connections 2024-04-23T18:25:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:20.233+0000 7f253ed13700 1 -- 172.21.15.33:0/3848038976 wait complete. 2024-04-23T18:25:20.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:19 smithi106 ceph-mon[32716]: pgmap v312: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:20.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2384147878' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-23T18:25:21.028 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:20 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3848038976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:25:21.063 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070498 got 253403070503 for osd.8 2024-04-23T18:25:21.063 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:21.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:20 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3848038976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:25:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:20 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3848038976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-23T18:25:21.744 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645337 got 309237645341 for osd.10 2024-04-23T18:25:21.744 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:22.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:21 smithi122 ceph-mon[32546]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:22.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:21 smithi033 ceph-mon[29274]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:22.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:21 smithi106 ceph-mon[32716]: pgmap v313: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:22.803 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:24.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:23 smithi122 ceph-mon[32546]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:25:24.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:23 smithi033 ceph-mon[29274]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:25:24.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:23 smithi106 ceph-mon[32716]: pgmap v314: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-23T18:25:25.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-23T18:25:25.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:25:25.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-23T18:25:25.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:24 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-23T18:25:25.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-23T18:25:25.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:24 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-23T18:25:25.333 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:25.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-23T18:25:25.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-04-23T18:25:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:24 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-23T18:25:26.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:25 smithi033 ceph-mon[29274]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:25:26.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-23T18:25:26.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:25 smithi122 ceph-mon[32546]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:25:26.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2", "id": [3, 9]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 7]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-23T18:25:26.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:25 smithi106 ceph-mon[32716]: osdmap e86: 12 total, 12 up, 12 in 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 -- 172.21.15.33:0/373860558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8101970 msgr2=0x7fc5f8101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/373860558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8101970 0x7fc5f8101df0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009990 tx=0x7fc5ec009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 -- 172.21.15.33:0/373860558 shutdown_connections 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/373860558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8102330 0x7fc5f810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/373860558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8107920 0x7fc5f8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/373860558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8101970 0x7fc5f8101df0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 -- 172.21.15.33:0/373860558 >> 172.21.15.33:0/373860558 conn(0x7fc5f80788f0 msgr2=0x7fc5f8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 -- 172.21.15.33:0/373860558 shutdown_connections 2024-04-23T18:25:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.402+0000 7fc5fe4db700 1 -- 172.21.15.33:0/373860558 wait complete. 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 Processor -- start 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 -- start start 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8101970 0x7fc5f8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8102330 0x7fc5f819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8111b60 con 0x7fc5f8107920 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f81119e0 con 0x7fc5f8101970 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fe4db700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f8111ce0 con 0x7fc5f8102330 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fca78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fca78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52856/0 (socket says 172.21.15.33:52856) 2024-04-23T18:25:26.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5fca78700 1 -- 172.21.15.33:0/4020936067 learned_addr learned my addr 172.21.15.33:0/4020936067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.403+0000 7fc5f77fe700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8102330 0x7fc5f819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5f7fff700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8101970 0x7fc5f8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 -- 172.21.15.33:0/4020936067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8102330 msgr2=0x7fc5f819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8102330 0x7fc5f819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 -- 172.21.15.33:0/4020936067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8101970 msgr2=0x7fc5f8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8101970 0x7fc5f8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 -- 172.21.15.33:0/4020936067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f81a5540 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fca78700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0007ee0 tx=0x7fc5f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f000dbd0 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5f000dd70 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f81a5770 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0005990 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.404+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81a5c80 con 0x7fc5f8107920 2024-04-23T18:25:26.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.405+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f8109eb0 con 0x7fc5f8107920 2024-04-23T18:25:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.406+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc5f001a020 con 0x7fc5f8107920 2024-04-23T18:25:26.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.407+0000 7fc5f57fa700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 0x7fc5e00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.407+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc5f00cfd20 con 0x7fc5f8107920 2024-04-23T18:25:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.407+0000 7fc5f7fff700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 0x7fc5e00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.408+0000 7fc5f7fff700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 0x7fc5e00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009bd0 tx=0x7fc5e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:26.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.416+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5f0010070 con 0x7fc5f8107920 2024-04-23T18:25:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.629+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc5f8109890 con 0x7fc5f8107920 2024-04-23T18:25:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.629+0000 7fc5f57fa700 1 -- 172.21.15.33:0/4020936067 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5f009c5a0 con 0x7fc5f8107920 2024-04-23T18:25:26.623 INFO:teuthology.orchestra.run.smithi033.stdout:55834574922 2024-04-23T18:25:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 msgr2=0x7fc5e00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 0x7fc5e00a6190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8009bd0 tx=0x7fc5e8009260 comp rx=0 tx=0).stop 2024-04-23T18:25:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 msgr2=0x7fc5f81a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0007ee0 tx=0x7fc5f000d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 shutdown_connections 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc5e00a3cd0 0x7fc5e00a6190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f8107920 0x7fc5f81a5000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f8101970 0x7fc5f8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 --2- 172.21.15.33:0/4020936067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f8102330 0x7fc5f819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.631+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 >> 172.21.15.33:0/4020936067 conn(0x7fc5f80788f0 msgr2=0x7fc5f80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.632+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 shutdown_connections 2024-04-23T18:25:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:26.632+0000 7fc5fe4db700 1 -- 172.21.15.33:0/4020936067 wait complete. 2024-04-23T18:25:27.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:26 smithi122 ceph-mon[32546]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:25:27.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:26 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4020936067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:27.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:26 smithi033 ceph-mon[29274]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:25:27.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:26 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4020936067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:27.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:26 smithi106 ceph-mon[32716]: osdmap e87: 12 total, 12 up, 12 in 2024-04-23T18:25:27.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:26 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4020936067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-23T18:25:28.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:27 smithi122 ceph-mon[32546]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:28.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:27 smithi033 ceph-mon[29274]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:28.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:27 smithi106 ceph-mon[32716]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-23T18:25:29.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:28 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:28 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:28 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:25:29.369 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574922 for osd.1 2024-04-23T18:25:29.369 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 -- 172.21.15.33:0/3167142012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4107960 msgr2=0x7f88c4107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 --2- 172.21.15.33:0/3167142012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4107960 0x7f88c4107d40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f88b4009990 tx=0x7f88b4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 -- 172.21.15.33:0/3167142012 shutdown_connections 2024-04-23T18:25:29.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 --2- 172.21.15.33:0/3167142012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 0x7f88c410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 --2- 172.21.15.33:0/3167142012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4107960 0x7f88c4107d40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 --2- 172.21.15.33:0/3167142012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4101960 0x7f88c4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 -- 172.21.15.33:0/3167142012 >> 172.21.15.33:0/3167142012 conn(0x7f88c40789e0 msgr2=0x7f88c4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 -- 172.21.15.33:0/3167142012 shutdown_connections 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.936+0000 7f88ca89c700 1 -- 172.21.15.33:0/3167142012 wait complete. 2024-04-23T18:25:29.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 Processor -- start 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 -- start start 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 0x7f88c4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 0x7f88c419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4111b70 con 0x7f88c4101960 2024-04-23T18:25:29.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f88c41119f0 con 0x7f88c4107960 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88ca89c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88c4111cf0 con 0x7f88c4102320 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42240/0 (socket says 172.21.15.33:42240) 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c3fff700 1 -- 172.21.15.33:0/1843899405 learned_addr learned my addr 172.21.15.33:0/1843899405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c8e39700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 0x7f88c419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c37fe700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 0x7f88c4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.937+0000 7f88c3fff700 1 -- 172.21.15.33:0/1843899405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 msgr2=0x7f88c4199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c3fff700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 0x7f88c4199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c3fff700 1 -- 172.21.15.33:0/1843899405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 msgr2=0x7f88c419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c3fff700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 0x7f88c419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c3fff700 1 -- 172.21.15.33:0/1843899405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c419e100 con 0x7f88c4101960 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c8e39700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 0x7f88c419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:29.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c3fff700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b2e0 tx=0x7f88b40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4026070 con 0x7f88c4101960 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c419e390 con 0x7f88c4101960 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88b4021410 con 0x7f88c4101960 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4017650 con 0x7f88c4101960 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.938+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c41a5c70 con 0x7f88c4101960 2024-04-23T18:25:29.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.939+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4069730 con 0x7f88c4101960 2024-04-23T18:25:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.940+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f88b4029070 con 0x7f88c4101960 2024-04-23T18:25:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.941+0000 7f88c17fa700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 0x7f88ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:29.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.941+0000 7f88c37fe700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 0x7f88ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.941+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f88b40d1a30 con 0x7f88c4101960 2024-04-23T18:25:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.942+0000 7f88c37fe700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 0x7f88ac0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88c40790c0 tx=0x7f88b8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:29.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:29.945+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88b4030070 con 0x7f88c4101960 2024-04-23T18:25:29.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:29 smithi033 ceph-mon[29274]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:29.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:29 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.156+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f88c402df80 con 0x7f88c4101960 2024-04-23T18:25:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.156+0000 7f88c17fa700 1 -- 172.21.15.33:0/1843899405 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f88b4013070 con 0x7f88c4101960 2024-04-23T18:25:30.149 INFO:teuthology.orchestra.run.smithi033.stdout:163208757303 2024-04-23T18:25:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 msgr2=0x7f88ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 0x7f88ac0a6190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88c40790c0 tx=0x7f88b8009260 comp rx=0 tx=0).stop 2024-04-23T18:25:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 msgr2=0x7f88c41990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b2e0 tx=0x7f88b40047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 shutdown_connections 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f88ac0a3cd0 0x7f88ac0a6190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f88c4107960 0x7f88c419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88c4101960 0x7f88c41990f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 --2- 172.21.15.33:0/1843899405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88c4102320 0x7f88c4199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 >> 172.21.15.33:0/1843899405 conn(0x7f88c40789e0 msgr2=0x7f88c406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 shutdown_connections 2024-04-23T18:25:30.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:30.158+0000 7f88ca89c700 1 -- 172.21.15.33:0/1843899405 wait complete. 2024-04-23T18:25:30.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:29 smithi122 ceph-mon[32546]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:30.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:29 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:29 smithi106 ceph-mon[32716]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 11 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:30.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:29 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:31.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:30 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1843899405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:30 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1843899405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:31.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:30 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1843899405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-23T18:25:31.830 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:32.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:31 smithi122 ceph-mon[32546]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:32.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:31 smithi033 ceph-mon[29274]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:32.246 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757298 got 163208757303 for osd.5 2024-04-23T18:25:32.246 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:32.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:31 smithi106 ceph-mon[32716]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:32.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.440+0000 7f213e647700 1 -- 172.21.15.33:0/1770666416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 msgr2=0x7f2138107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.440+0000 7f213e647700 1 --2- 172.21.15.33:0/1770666416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138107d70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2128009990 tx=0x7f2128009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 -- 172.21.15.33:0/1770666416 shutdown_connections 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 --2- 172.21.15.33:0/1770666416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f213810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 --2- 172.21.15.33:0/1770666416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138107d70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 --2- 172.21.15.33:0/1770666416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 0x7f2138101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 -- 172.21.15.33:0/1770666416 >> 172.21.15.33:0/1770666416 conn(0x7f21380788f0 msgr2=0x7f2138078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 -- 172.21.15.33:0/1770666416 shutdown_connections 2024-04-23T18:25:32.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 -- 172.21.15.33:0/1770666416 wait complete. 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 Processor -- start 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.441+0000 7f213e647700 1 -- start start 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 0x7f21381034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f2138103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2138111b60 con 0x7f2138107990 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21381119e0 con 0x7f2138102300 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213e647700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2138111ce0 con 0x7f2138101940 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42268/0 (socket says 172.21.15.33:42268) 2024-04-23T18:25:32.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 -- 172.21.15.33:0/537717223 learned_addr learned my addr 172.21.15.33:0/537717223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f21377fe700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f2138103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f2137fff700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 0x7f21381034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 -- 172.21.15.33:0/537717223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 msgr2=0x7f21381034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 0x7f21381034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 -- 172.21.15.33:0/537717223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 msgr2=0x7f2138103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f2138103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 -- 172.21.15.33:0/537717223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21381a76f0 con 0x7f2138107990 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f21377fe700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f2138103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f213cbe4700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2130007d70 tx=0x7f213000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.442+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2130016020 con 0x7f2138107990 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.443+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21381a79e0 con 0x7f2138107990 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.443+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f213000ad40 con 0x7f2138107990 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.443+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300115e0 con 0x7f2138107990 2024-04-23T18:25:32.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.443+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21381a7f20 con 0x7f2138107990 2024-04-23T18:25:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.444+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2138069730 con 0x7f2138107990 2024-04-23T18:25:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.445+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2130019020 con 0x7f2138107990 2024-04-23T18:25:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.445+0000 7f21357fa700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 0x7f21200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.446+0000 7f2137fff700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 0x7f21200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:32.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.446+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f21300d0910 con 0x7f2138107990 2024-04-23T18:25:32.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.446+0000 7f2137fff700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 0x7f21200a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2128006010 tx=0x7f212801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:32.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.449+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2130020020 con 0x7f2138107990 2024-04-23T18:25:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.659+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f21381a8250 con 0x7f2138107990 2024-04-23T18:25:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.660+0000 7f21357fa700 1 -- 172.21.15.33:0/537717223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f213001ae40 con 0x7f2138107990 2024-04-23T18:25:32.653 INFO:teuthology.orchestra.run.smithi033.stdout:141733920830 2024-04-23T18:25:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 msgr2=0x7f21200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 0x7f21200a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2128006010 tx=0x7f212801a040 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 msgr2=0x7f2138103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2130007d70 tx=0x7f213000a340 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 shutdown_connections 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f21200a3cd0 0x7f21200a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2138107990 0x7f2138103f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2138101940 0x7f21381034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 --2- 172.21.15.33:0/537717223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2138102300 0x7f2138103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 >> 172.21.15.33:0/537717223 conn(0x7f21380788f0 msgr2=0x7f213806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 shutdown_connections 2024-04-23T18:25:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:32.662+0000 7f213e647700 1 -- 172.21.15.33:0/537717223 wait complete. 2024-04-23T18:25:33.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:32 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/537717223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:33.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:32 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/537717223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:33.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:32 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/537717223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-23T18:25:33.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 -- 172.21.15.33:0/2142750458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec102a90 msgr2=0x7f79ec102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2142750458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec102a90 0x7f79ec102f10 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009990 tx=0x7f79dc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 -- 172.21.15.33:0/2142750458 shutdown_connections 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2142750458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec103450 0x7f79ec10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2142750458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec108a50 0x7f79ec108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2142750458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec102a90 0x7f79ec102f10 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 -- 172.21.15.33:0/2142750458 >> 172.21.15.33:0/2142750458 conn(0x7f79ec079ba0 msgr2=0x7f79ec079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 -- 172.21.15.33:0/2142750458 shutdown_connections 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.482+0000 7f79f43bb700 1 -- 172.21.15.33:0/2142750458 wait complete. 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.483+0000 7f79f43bb700 1 Processor -- start 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.483+0000 7f79f43bb700 1 -- start start 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.483+0000 7f79f43bb700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec102a90 0x7f79ec1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.483+0000 7f79f43bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec108a50 0x7f79ec19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec112c80 con 0x7f79ec103450 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec112b00 con 0x7f79ec102a90 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79ec112e00 con 0x7f79ec108a50 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42278/0 (socket says 172.21.15.33:42278) 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 -- 172.21.15.33:0/2005382621 learned_addr learned my addr 172.21.15.33:0/2005382621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f2958700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec108a50 0x7f79ec19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f2157700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec102a90 0x7f79ec1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 -- 172.21.15.33:0/2005382621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec108a50 msgr2=0x7f79ec19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec108a50 0x7f79ec19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 -- 172.21.15.33:0/2005382621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec102a90 msgr2=0x7f79ec1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec102a90 0x7f79ec1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 -- 172.21.15.33:0/2005382621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79ec19e1c0 con 0x7f79ec103450 2024-04-23T18:25:33.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f1956700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc027070 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79ec19e450 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79dc003fa0 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc022db0 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.484+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79ec1a5d30 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.485+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79ec069730 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.486+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f79dc02a070 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.487+0000 7f79e37fe700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 0x7f79d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.487+0000 7f79f2157700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 0x7f79d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.487+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f79dc0d1680 con 0x7f79ec103450 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.487+0000 7f79f2157700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 0x7f79d80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79e8006fd0 tx=0x7f79e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:33.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.491+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79dc013070 con 0x7f79ec103450 2024-04-23T18:25:33.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.701+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f79ec02df80 con 0x7f79ec103450 2024-04-23T18:25:33.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.701+0000 7f79e37fe700 1 -- 172.21.15.33:0/2005382621 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f79dc09df90 con 0x7f79ec103450 2024-04-23T18:25:33.694 INFO:teuthology.orchestra.run.smithi033.stdout:287762808869 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 msgr2=0x7f79d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 0x7f79d80a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f79e8006fd0 tx=0x7f79e8008040 comp rx=0 tx=0).stop 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 msgr2=0x7f79ec1996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f79dc000c00 tx=0x7f79dc0047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 shutdown_connections 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79d80a3cd0 0x7f79d80a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79ec108a50 0x7f79ec19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79ec102a90 0x7f79ec1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 --2- 172.21.15.33:0/2005382621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79ec103450 0x7f79ec1996f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 >> 172.21.15.33:0/2005382621 conn(0x7f79ec079ba0 msgr2=0x7f79ec1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 shutdown_connections 2024-04-23T18:25:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:33.704+0000 7f79f43bb700 1 -- 172.21.15.33:0/2005382621 wait complete. 2024-04-23T18:25:33.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:33 smithi033 ceph-mon[29274]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:33.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2005382621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:34.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:33 smithi122 ceph-mon[32546]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:34.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2005382621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:34.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:33 smithi106 ceph-mon[32716]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:34.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2005382621' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-23T18:25:35.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808865 got 287762808869 for osd.9 2024-04-23T18:25:35.061 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 -- 172.21.15.33:0/2193849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 msgr2=0x7fc05c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 --2- 172.21.15.33:0/2193849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c10f930 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc05400b0a0 tx=0x7fc05400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 -- 172.21.15.33:0/2193849477 shutdown_connections 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 --2- 172.21.15.33:0/2193849477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c10f930 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 --2- 172.21.15.33:0/2193849477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c108a60 0x7fc05c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 --2- 172.21.15.33:0/2193849477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c06b1f0 0x7fc05c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 -- 172.21.15.33:0/2193849477 >> 172.21.15.33:0/2193849477 conn(0x7fc05c0789e0 msgr2=0x7fc05c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 -- 172.21.15.33:0/2193849477 shutdown_connections 2024-04-23T18:25:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.251+0000 7fc063f60700 1 -- 172.21.15.33:0/2193849477 wait complete. 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 Processor -- start 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 -- start start 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c06b1f0 0x7fc05c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c108a60 0x7fc05c19d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c112c60 con 0x7fc05c06bbb0 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c112ae0 con 0x7fc05c06b1f0 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc063f60700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c112de0 con 0x7fc05c108a60 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc0614fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc0614fb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42286/0 (socket says 172.21.15.33:42286) 2024-04-23T18:25:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.252+0000 7fc0614fb700 1 -- 172.21.15.33:0/1204997250 learned_addr learned my addr 172.21.15.33:0/1204997250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0624fd700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c108a60 0x7fc05c19d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc061cfc700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c06b1f0 0x7fc05c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 -- 172.21.15.33:0/1204997250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c108a60 msgr2=0x7fc05c19d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c108a60 0x7fc05c19d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 -- 172.21.15.33:0/1204997250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c06b1f0 msgr2=0x7fc05c199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c06b1f0 0x7fc05c199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 -- 172.21.15.33:0/1204997250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c19e0a0 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc0614fb700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00d980 tx=0x7fc04c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c017070 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c19e390 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc04c005020 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c009b50 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.253+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c1a5c10 con 0x7fc05c06bbb0 2024-04-23T18:25:35.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.254+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c069730 con 0x7fc05c06bbb0 2024-04-23T18:25:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.255+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc04c0051d0 con 0x7fc05c06bbb0 2024-04-23T18:25:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.256+0000 7fc052ffd700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 0x7fc0480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.256+0000 7fc061cfc700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 0x7fc0480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.256+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc04c0d08b0 con 0x7fc05c06bbb0 2024-04-23T18:25:35.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.256+0000 7fc061cfc700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 0x7fc0480a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc058005fd0 tx=0x7fc058005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.260+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc04c020020 con 0x7fc05c06bbb0 2024-04-23T18:25:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.474+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc05c02df80 con 0x7fc05c06bbb0 2024-04-23T18:25:35.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.474+0000 7fc052ffd700 1 -- 172.21.15.33:0/1204997250 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc04c09d130 con 0x7fc05c06bbb0 2024-04-23T18:25:35.469 INFO:teuthology.orchestra.run.smithi033.stdout:219043332142 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 msgr2=0x7fc0480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 0x7fc0480a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc058005fd0 tx=0x7fc058005db0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 msgr2=0x7fc05c1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00d980 tx=0x7fc04c00dc90 comp rx=0 tx=0).stop 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 shutdown_connections 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc0480a3cd0 0x7fc0480a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc05c108a60 0x7fc05c19d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc05c06b1f0 0x7fc05c199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 --2- 172.21.15.33:0/1204997250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc05c06bbb0 0x7fc05c1995d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 >> 172.21.15.33:0/1204997250 conn(0x7fc05c0789e0 msgr2=0x7fc05c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 shutdown_connections 2024-04-23T18:25:35.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.477+0000 7fc063f60700 1 -- 172.21.15.33:0/1204997250 wait complete. 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- 172.21.15.33:0/3024183561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 msgr2=0x7f058010e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 --2- 172.21.15.33:0/3024183561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f058010e830 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f057800b0a0 tx=0x7f057800b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- 172.21.15.33:0/3024183561 shutdown_connections 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 --2- 172.21.15.33:0/3024183561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f058010e830 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 --2- 172.21.15.33:0/3024183561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f0580107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 --2- 172.21.15.33:0/3024183561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- 172.21.15.33:0/3024183561 >> 172.21.15.33:0/3024183561 conn(0x7f05800789e0 msgr2=0x7f0580078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- 172.21.15.33:0/3024183561 shutdown_connections 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- 172.21.15.33:0/3024183561 wait complete. 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 Processor -- start 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.889+0000 7f0586572700 1 -- start start 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f058019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0580111b40 con 0x7f0580102320 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05801119c0 con 0x7f0580107960 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0586572700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0580111cc0 con 0x7f0580101960 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42304/0 (socket says 172.21.15.33:42304) 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 -- 172.21.15.33:0/1448841465 learned_addr learned my addr 172.21.15.33:0/1448841465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057ffff700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0584b0f700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f058019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 -- 172.21.15.33:0/1448841465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 msgr2=0x7f0580197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 -- 172.21.15.33:0/1448841465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 msgr2=0x7f058019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:35.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f058019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 -- 172.21.15.33:0/1448841465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058019cf00 con 0x7f0580102320 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057ffff700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f0584b0f700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f058019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.890+0000 7f057f7fe700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f057000d980 tx=0x7f057000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.891+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570016070 con 0x7f0580102320 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.891+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05700044d0 con 0x7f0580102320 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.891+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570004d00 con 0x7f0580102320 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.891+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058019d1f0 con 0x7f0580102320 2024-04-23T18:25:35.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.891+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05801a4a70 con 0x7f0580102320 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.892+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0580069730 con 0x7f0580102320 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.893+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0570004020 con 0x7f0580102320 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.894+0000 7f057d7fa700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 0x7f05680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.894+0000 7f057ffff700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 0x7f05680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.894+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f05700d0950 con 0x7f0580102320 2024-04-23T18:25:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.894+0000 7f057ffff700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 0x7f05680a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f057400baa0 tx=0x7f0574005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:35.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:35.898+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f057001f020 con 0x7f0580102320 2024-04-23T18:25:36.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.110+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f05801a4cd0 con 0x7f0580102320 2024-04-23T18:25:36.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:35 smithi033 ceph-mon[29274]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:36.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:35 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1204997250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:36.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.111+0000 7f057d7fa700 1 -- 172.21.15.33:0/1448841465 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f057009d1d0 con 0x7f0580102320 2024-04-23T18:25:36.105 INFO:teuthology.orchestra.run.smithi033.stdout:34359738449 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 msgr2=0x7f05680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 0x7f05680a6210 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f057400baa0 tx=0x7f0574005db0 comp rx=0 tx=0).stop 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 msgr2=0x7f0580198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f057000d980 tx=0x7f057000dd40 comp rx=0 tx=0).stop 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 shutdown_connections 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f05680a3d50 0x7f05680a6210 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:36.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0580107960 0x7f058019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0580101960 0x7f0580197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 --2- 172.21.15.33:0/1448841465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0580102320 0x7f0580198430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 >> 172.21.15.33:0/1448841465 conn(0x7f05800789e0 msgr2=0x7f05801043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 shutdown_connections 2024-04-23T18:25:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:36.113+0000 7f0586572700 1 -- 172.21.15.33:0/1448841465 wait complete. 2024-04-23T18:25:36.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:35 smithi122 ceph-mon[32546]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:36.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:35 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1204997250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:36.267 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332136 got 219043332142 for osd.7 2024-04-23T18:25:36.267 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:36.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:35 smithi106 ceph-mon[32716]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-04-23T18:25:36.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:35 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1204997250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-23T18:25:36.891 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738449 for osd.0 2024-04-23T18:25:36.891 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:37.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:36 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1448841465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:37.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:36 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1448841465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/1897102730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8102af0 msgr2=0x7f2fb8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/1897102730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8102af0 0x7f2fb8102f70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2fac009990 tx=0x7f2fac009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/1897102730 shutdown_connections 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/1897102730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb81034b0 0x7f2fb810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/1897102730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8108a80 0x7f2fb8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/1897102730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb8102af0 0x7f2fb8102f70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/1897102730 >> 172.21.15.33:0/1897102730 conn(0x7f2fb8079b20 msgr2=0x7f2fb8079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/1897102730 shutdown_connections 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.210+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/1897102730 wait complete. 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 Processor -- start 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 -- start start 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 0x7f2fb81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb8108a80 0x7f2fb819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112c50 con 0x7f2fb81034b0 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112ad0 con 0x7f2fb8102af0 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.211+0000 7f2fbcb4e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb8112dd0 con 0x7f2fb8108a80 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb659c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 0x7f2fb81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33508/0 (socket says 172.21.15.33:33508) 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 -- 172.21.15.33:0/583924785 learned_addr learned my addr 172.21.15.33:0/583924785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb6d9d700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb8108a80 0x7f2fb819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 -- 172.21.15.33:0/583924785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb8108a80 msgr2=0x7f2fb819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb8108a80 0x7f2fb819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 -- 172.21.15.33:0/583924785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 msgr2=0x7f2fb81990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 0x7f2fb81990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 -- 172.21.15.33:0/583924785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb819e0f0 con 0x7f2fb81034b0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb659c700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 0x7f2fb81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fb5d9b700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fac000c00 tx=0x7f2fac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac027070 con 0x7f2fb81034b0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb819e380 con 0x7f2fb81034b0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fac003f20 con 0x7f2fb81034b0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.212+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac022ca0 con 0x7f2fb81034b0 2024-04-23T18:25:37.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.213+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb8074c70 con 0x7f2fb81034b0 2024-04-23T18:25:37.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.214+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb81041b0 con 0x7f2fb81034b0 2024-04-23T18:25:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.215+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2fac02a070 con 0x7f2fb81034b0 2024-04-23T18:25:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.216+0000 7f2fa77fe700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 0x7f2fa00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.217+0000 7f2fb659c700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 0x7f2fa00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.217+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f2fac0d17a0 con 0x7f2fb81034b0 2024-04-23T18:25:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.217+0000 7f2fb659c700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 0x7f2fa00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005950 tx=0x7f2fa800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.220+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fac031070 con 0x7f2fb81034b0 2024-04-23T18:25:37.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:36 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1448841465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-23T18:25:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2261228867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc108a20 msgr2=0x7f1adc108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2261228867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc108a20 0x7f1adc108e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009990 tx=0x7f1ad8009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2261228867 shutdown_connections 2024-04-23T18:25:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2261228867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06b980 0x7f1adc10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2261228867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc108a20 0x7f1adc108e00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2261228867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc06afe0 0x7f1adc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2261228867 >> 172.21.15.33:0/2261228867 conn(0x7f1adc0788b0 msgr2=0x7f1adc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2261228867 shutdown_connections 2024-04-23T18:25:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.424+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2261228867 wait complete. 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 Processor -- start 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 -- start start 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06afe0 0x7f1adc101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 0x7f1adc102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc112bb0 con 0x7f1adc06b980 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc112a30 con 0x7f1adc108a20 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae40ca700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc112d30 con 0x7f1adc06afe0 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33524/0 (socket says 172.21.15.33:33524) 2024-04-23T18:25:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 -- 172.21.15.33:0/2876401821 learned_addr learned my addr 172.21.15.33:0/2876401821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1e66700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06afe0 0x7f1adc101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae2667700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 0x7f1adc102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 -- 172.21.15.33:0/2876401821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06afe0 msgr2=0x7f1adc101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06afe0 0x7f1adc101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 -- 172.21.15.33:0/2876401821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 msgr2=0x7f1adc102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 0x7f1adc102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae1665700 1 -- 172.21.15.33:0/2876401821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc1a7610 con 0x7f1adc06b980 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.425+0000 7f1ae2667700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 0x7f1adc102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ae1665700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00d950 tx=0x7f1acc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc016070 con 0x7f1adc06b980 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc1a7900 con 0x7f1adc06b980 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1acc009df0 con 0x7f1adc06b980 2024-04-23T18:25:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00f3c0 con 0x7f1adc06b980 2024-04-23T18:25:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.426+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc1a7e40 con 0x7f1adc06b980 2024-04-23T18:25:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.427+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1adc069730 con 0x7f1adc06b980 2024-04-23T18:25:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.428+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1acc019070 con 0x7f1adc06b980 2024-04-23T18:25:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.429+0000 7f1ad2ffd700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 0x7f1ac80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:37.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.429+0000 7f1ae1e66700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 0x7f1ac80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.429+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f1acc0d0140 con 0x7f1adc06b980 2024-04-23T18:25:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.429+0000 7f1ae1e66700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 0x7f1ac80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800b2e0 tx=0x7f1ad8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.432+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1acc09c9c0 con 0x7f1adc06b980 2024-04-23T18:25:37.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.436+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2fb802df80 con 0x7f2fb81034b0 2024-04-23T18:25:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.436+0000 7f2fa77fe700 1 -- 172.21.15.33:0/583924785 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2fac013070 con 0x7f2fb81034b0 2024-04-23T18:25:37.430 INFO:teuthology.orchestra.run.smithi033.stdout:343597383707 2024-04-23T18:25:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 msgr2=0x7f2fa00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 0x7f2fa00a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8005950 tx=0x7f2fa800b410 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 msgr2=0x7f2fb8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2fac000c00 tx=0x7f2fac0047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 shutdown_connections 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2fa00a3cd0 0x7f2fa00a6190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fb8108a80 0x7f2fb819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb8102af0 0x7f2fb81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 --2- 172.21.15.33:0/583924785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fb81034b0 0x7f2fb8199620 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 >> 172.21.15.33:0/583924785 conn(0x7f2fb8079b20 msgr2=0x7f2fb81017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 shutdown_connections 2024-04-23T18:25:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.438+0000 7f2fbcb4e700 1 -- 172.21.15.33:0/583924785 wait complete. 2024-04-23T18:25:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.657+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1adc02e140 con 0x7f1adc06b980 2024-04-23T18:25:37.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.657+0000 7f1ad2ffd700 1 -- 172.21.15.33:0/2876401821 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1acc09c2c0 con 0x7f1adc06b980 2024-04-23T18:25:37.651 INFO:teuthology.orchestra.run.smithi033.stdout:107374182467 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 msgr2=0x7f1ac80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 0x7f1ac80a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800b2e0 tx=0x7f1ad8005960 comp rx=0 tx=0).stop 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 msgr2=0x7f1adc101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00d950 tx=0x7f1acc00dc60 comp rx=0 tx=0).stop 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 shutdown_connections 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1ac80a3cd0 0x7f1ac80a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1adc108a20 0x7f1adc102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1adc06afe0 0x7f1adc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 --2- 172.21.15.33:0/2876401821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1adc06b980 0x7f1adc101f40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 >> 172.21.15.33:0/2876401821 conn(0x7f1adc0788b0 msgr2=0x7f1adc0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 shutdown_connections 2024-04-23T18:25:37.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:37.660+0000 7f1ae40ca700 1 -- 172.21.15.33:0/2876401821 wait complete. 2024-04-23T18:25:38.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:37 smithi122 ceph-mon[32546]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:25:38.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/583924785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:38.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2876401821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:38.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:37 smithi033 ceph-mon[29274]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:25:38.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/583924785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:38.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2876401821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:38.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182467 for osd.3 2024-04-23T18:25:38.303 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:37 smithi106 ceph-mon[32716]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s; 0 B/s, 0 keys/s, 1 objects/s recovering 2024-04-23T18:25:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/583924785' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-23T18:25:38.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2876401821' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-23T18:25:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 -- 172.21.15.33:0/2593765551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806b0f0 msgr2=0x7fead810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 --2- 172.21.15.33:0/2593765551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806b0f0 0x7fead810f900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fead0009990 tx=0x7fead0009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 -- 172.21.15.33:0/2593765551 shutdown_connections 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 --2- 172.21.15.33:0/2593765551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806b0f0 0x7fead810f900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 --2- 172.21.15.33:0/2593765551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 0x7fead8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 --2- 172.21.15.33:0/2593765551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806a750 0x7fead806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 -- 172.21.15.33:0/2593765551 >> 172.21.15.33:0/2593765551 conn(0x7fead80788f0 msgr2=0x7fead8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 -- 172.21.15.33:0/2593765551 shutdown_connections 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.497+0000 7feadf820700 1 -- 172.21.15.33:0/2593765551 wait complete. 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 Processor -- start 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 -- start start 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 0x7fead8199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 0x7fead819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fead8112c20 con 0x7fead806a750 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fead8112aa0 con 0x7fead806b0f0 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.498+0000 7feadf820700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fead8112da0 con 0x7fead8108a30 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33550/0 (socket says 172.21.15.33:33550) 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 -- 172.21.15.33:0/3376908663 learned_addr learned my addr 172.21.15.33:0/3376908663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadcdbb700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 0x7fead8199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadddbd700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 0x7fead819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 -- 172.21.15.33:0/3376908663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 msgr2=0x7fead819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 0x7fead819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 -- 172.21.15.33:0/3376908663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 msgr2=0x7fead8199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 0x7fead8199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 -- 172.21.15.33:0/3376908663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead819e160 con 0x7fead806a750 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadcdbb700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 0x7fead8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadd5bc700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feac80060b0 tx=0x7feac800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac801a070 con 0x7fead806a750 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feac800dd20 con 0x7fead806a750 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead819e450 con 0x7fead806a750 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8014600 con 0x7fead806a750 2024-04-23T18:25:39.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.499+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead81a5cd0 con 0x7fead806a750 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.501+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead8069730 con 0x7fead806a750 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.502+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feac801d070 con 0x7fead806a750 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.502+0000 7fead67fc700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 0x7feac00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.503+0000 7feadcdbb700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 0x7feac00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.503+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7feac80d01b0 con 0x7fead806a750 2024-04-23T18:25:39.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.503+0000 7feadcdbb700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 0x7feac00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fead8078f60 tx=0x7feacc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:39.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.506+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feac8010070 con 0x7fead806a750 2024-04-23T18:25:39.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383707 for osd.11 2024-04-23T18:25:39.603 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:39.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.717+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fead802df80 con 0x7fead806a750 2024-04-23T18:25:39.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.717+0000 7fead67fc700 1 -- 172.21.15.33:0/3376908663 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feac809ca30 con 0x7fead806a750 2024-04-23T18:25:39.711 INFO:teuthology.orchestra.run.smithi033.stdout:197568495669 2024-04-23T18:25:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 msgr2=0x7feac00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 0x7feac00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fead8078f60 tx=0x7feacc00b410 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 msgr2=0x7fead8199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feac80060b0 tx=0x7feac800d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 shutdown_connections 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feac00a3cd0 0x7feac00a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.718+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fead8108a30 0x7fead819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.719+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fead806a750 0x7fead8199150 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.719+0000 7feadf820700 1 --2- 172.21.15.33:0/3376908663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead806b0f0 0x7fead8199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.719+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 >> 172.21.15.33:0/3376908663 conn(0x7fead80788f0 msgr2=0x7fead80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.719+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 shutdown_connections 2024-04-23T18:25:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:39.719+0000 7feadf820700 1 -- 172.21.15.33:0/3376908663 wait complete. 2024-04-23T18:25:39.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:39 smithi033 ceph-mon[29274]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:25:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:39 smithi122 ceph-mon[32546]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:25:40.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:39 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3376908663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:40.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:39 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3376908663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:40.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:39 smithi106 ceph-mon[32716]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-23T18:25:40.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:39 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3376908663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-23T18:25:40.692 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920830 for osd.4 2024-04-23T18:25:40.692 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:41.159 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495669 for osd.6 2024-04-23T18:25:41.159 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:41.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 -- 172.21.15.33:0/2141341270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 msgr2=0x7f5134108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 --2- 172.21.15.33:0/2141341270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f5134108eb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5124009990 tx=0x7f5124009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 -- 172.21.15.33:0/2141341270 shutdown_connections 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 --2- 172.21.15.33:0/2141341270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 0x7f513410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 --2- 172.21.15.33:0/2141341270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f5134108eb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 --2- 172.21.15.33:0/2141341270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 0x7f5134101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 -- 172.21.15.33:0/2141341270 >> 172.21.15.33:0/2141341270 conn(0x7f5134077fb0 msgr2=0x7f51340ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 -- 172.21.15.33:0/2141341270 shutdown_connections 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.626+0000 7f5139f30700 1 -- 172.21.15.33:0/2141341270 wait complete. 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.627+0000 7f5139f30700 1 Processor -- start 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.627+0000 7f5139f30700 1 -- start start 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.627+0000 7f5139f30700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 0x7f51341991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.627+0000 7f5139f30700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 0x7f51341996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.627+0000 7f5139f30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5139f30700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5134112c30 con 0x7f5134108ad0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5139f30700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5134112ab0 con 0x7f51341019d0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5139f30700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5134112db0 con 0x7f5134101030 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33570/0 (socket says 172.21.15.33:33570) 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 -- 172.21.15.33:0/3681301737 learned_addr learned my addr 172.21.15.33:0/3681301737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5132ffd700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 0x7f51341996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f51337fe700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 0x7f51341991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 -- 172.21.15.33:0/3681301737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 msgr2=0x7f51341991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 0x7f51341991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 -- 172.21.15.33:0/3681301737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 msgr2=0x7f51341996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 0x7f51341996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 -- 172.21.15.33:0/3681301737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513419e1b0 con 0x7f5134108ad0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5133fff700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f512c007ee0 tx=0x7f512c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:41.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c004bb0 con 0x7f5134108ad0 2024-04-23T18:25:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.628+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c004d50 con 0x7f5134108ad0 2024-04-23T18:25:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.629+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513419e4a0 con 0x7f5134108ad0 2024-04-23T18:25:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.629+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c005920 con 0x7f5134108ad0 2024-04-23T18:25:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.629+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51341a5d20 con 0x7f5134108ad0 2024-04-23T18:25:41.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.630+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5134104350 con 0x7f5134108ad0 2024-04-23T18:25:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.631+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f512c01a020 con 0x7f5134108ad0 2024-04-23T18:25:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.632+0000 7f5130ff9700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 0x7f511c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.632+0000 7f51337fe700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 0x7f511c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.632+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f512c0cfbe0 con 0x7f5134108ad0 2024-04-23T18:25:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.632+0000 7f51337fe700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 0x7f511c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f512400b5c0 tx=0x7f5124005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.637+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f512c010070 con 0x7f5134108ad0 2024-04-23T18:25:41.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.856+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f513402df80 con 0x7f5134108ad0 2024-04-23T18:25:41.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.856+0000 7f5130ff9700 1 -- 172.21.15.33:0/3681301737 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f512c09c420 con 0x7f5134108ad0 2024-04-23T18:25:41.850 INFO:teuthology.orchestra.run.smithi033.stdout:77309411402 2024-04-23T18:25:41.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 msgr2=0x7f511c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 0x7f511c0a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f512400b5c0 tx=0x7f5124005fd0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 msgr2=0x7f513419da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f512c007ee0 tx=0x7f512c00d3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 shutdown_connections 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.858+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f511c0a3cd0 0x7f511c0a6190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5134108ad0 0x7f513419da70 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5134101030 0x7f51341991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 --2- 172.21.15.33:0/3681301737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51341019d0 0x7f51341996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 >> 172.21.15.33:0/3681301737 conn(0x7f5134077fb0 msgr2=0x7f51340ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 shutdown_connections 2024-04-23T18:25:41.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:41.859+0000 7f5139f30700 1 -- 172.21.15.33:0/3681301737 wait complete. 2024-04-23T18:25:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:41 smithi122 ceph-mon[32546]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:41 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3681301737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:42.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:41 smithi033 ceph-mon[29274]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:42.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:41 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3681301737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:42.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:41 smithi106 ceph-mon[32716]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:42.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:41 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3681301737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-23T18:25:42.544 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411402 for osd.2 2024-04-23T18:25:42.544 DEBUG:teuthology.parallel:result is None 2024-04-23T18:25:42.545 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-23T18:25:42.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:25:43.132 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:44.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:43 smithi033 ceph-mon[29274]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:44.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:43 smithi106 ceph-mon[32716]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:44.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:43 smithi122 ceph-mon[32546]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 -- 172.21.15.33:0/4165772707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101dc0 msgr2=0x7f9b2c10f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 --2- 172.21.15.33:0/4165772707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c10f830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400b0a0 tx=0x7f9b2400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 -- 172.21.15.33:0/4165772707 shutdown_connections 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 --2- 172.21.15.33:0/4165772707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c10f830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 --2- 172.21.15.33:0/4165772707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 0x7f9b2c107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 --2- 172.21.15.33:0/4165772707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101400 0x7f9b2c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 -- 172.21.15.33:0/4165772707 >> 172.21.15.33:0/4165772707 conn(0x7f9b2c076c60 msgr2=0x7f9b2c077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 -- 172.21.15.33:0/4165772707 shutdown_connections 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.835+0000 7f9b331b4700 1 -- 172.21.15.33:0/4165772707 wait complete. 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 Processor -- start 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 -- start start 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 0x7f9b2c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c112d30 con 0x7f9b2c101400 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c112bb0 con 0x7f9b2c101dc0 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b331b4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c112eb0 con 0x7f9b2c107b20 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b30f50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b30f50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33594/0 (socket says 172.21.15.33:33594) 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.836+0000 7f9b30f50700 1 -- 172.21.15.33:0/279024527 learned_addr learned my addr 172.21.15.33:0/279024527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b31751700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 0x7f9b2c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b2bfff700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 -- 172.21.15.33:0/279024527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 msgr2=0x7f9b2c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 0x7f9b2c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 -- 172.21.15.33:0/279024527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101dc0 msgr2=0x7f9b2c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 -- 172.21.15.33:0/279024527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c19e1f0 con 0x7f9b2c101400 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b30f50700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00b500 tx=0x7f9b1c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c015020 con 0x7f9b2c101400 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1c011ce0 con 0x7f9b2c101400 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c19e4e0 con 0x7f9b2c101400 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c010d60 con 0x7f9b2c101400 2024-04-23T18:25:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.837+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c1a5d60 con 0x7f9b2c101400 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.838+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c069730 con 0x7f9b2c101400 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.839+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b1c018020 con 0x7f9b2c101400 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.840+0000 7f9b29ffb700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 0x7f9b140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.840+0000 7f9b2bfff700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 0x7f9b140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.840+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f9b1c0cfc90 con 0x7f9b2c101400 2024-04-23T18:25:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.840+0000 7f9b2bfff700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 0x7f9b140a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006fd0 tx=0x7f9b20008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:44.845+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b1c0d5020 con 0x7f9b2c101400 2024-04-23T18:25:45.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.066+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b2c062e80 con 0x7f9b140a3cd0 2024-04-23T18:25:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.080+0000 7f9b29ffb700 1 -- 172.21.15.33:0/279024527 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259022 (secure 0 0 0) 0x7f9b2c062e80 con 0x7f9b140a3cd0 2024-04-23T18:25:45.077 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 msgr2=0x7f9b140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 0x7f9b140a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006fd0 tx=0x7f9b20008040 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 msgr2=0x7f9b2c1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00b500 tx=0x7f9b1c00b8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 shutdown_connections 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f9b140a3cd0 0x7f9b140a6190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b2c107b20 0x7f9b2c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.083+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b2c101400 0x7f9b2c1991e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.084+0000 7f9b331b4700 1 --2- 172.21.15.33:0/279024527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9b2c101dc0 0x7f9b2c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:45.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.084+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 >> 172.21.15.33:0/279024527 conn(0x7f9b2c076c60 msgr2=0x7f9b2c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:45.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.084+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 shutdown_connections 2024-04-23T18:25:45.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:45.084+0000 7f9b331b4700 1 -- 172.21.15.33:0/279024527 wait complete. 2024-04-23T18:25:45.080 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:25:45.793 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-23T18:25:44.173894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657829,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":847,"num_read_kb":545,"num_write":422,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381748,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560076,"statfs":{"total":1151940427776,"available":1151549517824,"internally_reserved":0,"allocated":11993088,"data_stored":7441911,"data_compressed":176862,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":60,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002988"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912449+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:25:25.912449+0000","last_peered":"2024-04-23T18:25:25.912449+0000","last_clean":"2024-04-23T18:25:25.912449+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:25:25.912449+0000","last_undegraded":"2024-04-23T18:25:25.912449+0000","last_fullsized":"2024-04-23T18:25:25.912449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909607+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:25:25.909607+0000","last_peered":"2024-04-23T18:25:25.909607+0000","last_clean":"2024-04-23T18:25:25.909607+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:25:25.909607+0000","last_undegraded":"2024-04-23T18:25:25.909607+0000","last_fullsized":"2024-04-23T18:25:25.909607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909532+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:25:25.909532+0000","last_peered":"2024-04-23T18:25:25.909532+0000","last_clean":"2024-04-23T18:25:25.909532+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:25:25.909532+0000","last_undegraded":"2024-04-23T18:25:25.909532+0000","last_fullsized":"2024-04-23T18:25:25.909532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101564+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:25:26.101564+0000","last_peered":"2024-04-23T18:25:26.101564+0000","last_clean":"2024-04-23T18:25:26.101564+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:25:26.101564+0000","last_undegraded":"2024-04-23T18:25:26.101564+0000","last_fullsized":"2024-04-23T18:25:26.101564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910158+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:25:25.910158+0000","last_peered":"2024-04-23T18:25:25.910158+0000","last_clean":"2024-04-23T18:25:25.910158+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:25:25.910158+0000","last_undegraded":"2024-04-23T18:25:25.910158+0000","last_fullsized":"2024-04-23T18:25:25.910158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152686+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:25:26.152686+0000","last_peered":"2024-04-23T18:25:26.152686+0000","last_clean":"2024-04-23T18:25:26.152686+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:25:26.152686+0000","last_undegraded":"2024-04-23T18:25:26.152686+0000","last_fullsized":"2024-04-23T18:25:26.152686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910122+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:25:25.910122+0000","last_peered":"2024-04-23T18:25:25.910122+0000","last_clean":"2024-04-23T18:25:25.910122+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:25:25.910122+0000","last_undegraded":"2024-04-23T18:25:25.910122+0000","last_fullsized":"2024-04-23T18:25:25.910122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915474+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:25:25.915474+0000","last_peered":"2024-04-23T18:25:25.915474+0000","last_clean":"2024-04-23T18:25:25.915474+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:25:25.915474+0000","last_undegraded":"2024-04-23T18:25:25.915474+0000","last_fullsized":"2024-04-23T18:25:25.915474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090668+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:25:26.090668+0000","last_peered":"2024-04-23T18:25:26.090668+0000","last_clean":"2024-04-23T18:25:26.090668+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:25:26.090668+0000","last_undegraded":"2024-04-23T18:25:26.090668+0000","last_fullsized":"2024-04-23T18:25:26.090668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.153093+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:25:26.153093+0000","last_peered":"2024-04-23T18:25:26.153093+0000","last_clean":"2024-04-23T18:25:26.153093+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:25:26.153093+0000","last_undegraded":"2024-04-23T18:25:26.153093+0000","last_fullsized":"2024-04-23T18:25:26.153093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910649+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:25:25.910649+0000","last_peered":"2024-04-23T18:25:25.910649+0000","last_clean":"2024-04-23T18:25:25.910649+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:25:25.910649+0000","last_undegraded":"2024-04-23T18:25:25.910649+0000","last_fullsized":"2024-04-23T18:25:25.910649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101498+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:25:26.101498+0000","last_peered":"2024-04-23T18:25:26.101498+0000","last_clean":"2024-04-23T18:25:26.101498+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:25:26.101498+0000","last_undegraded":"2024-04-23T18:25:26.101498+0000","last_fullsized":"2024-04-23T18:25:26.101498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912688+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:25:25.912688+0000","last_peered":"2024-04-23T18:25:25.912688+0000","last_clean":"2024-04-23T18:25:25.912688+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:25:25.912688+0000","last_undegraded":"2024-04-23T18:25:25.912688+0000","last_fullsized":"2024-04-23T18:25:25.912688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909244+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:25:25.909244+0000","last_peered":"2024-04-23T18:25:25.909244+0000","last_clean":"2024-04-23T18:25:25.909244+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:25:25.909244+0000","last_undegraded":"2024-04-23T18:25:25.909244+0000","last_fullsized":"2024-04-23T18:25:25.909244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909207+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:25:25.909207+0000","last_peered":"2024-04-23T18:25:25.909207+0000","last_clean":"2024-04-23T18:25:25.909207+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:25:25.909207+0000","last_undegraded":"2024-04-23T18:25:25.909207+0000","last_fullsized":"2024-04-23T18:25:25.909207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090180+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:25:26.090180+0000","last_peered":"2024-04-23T18:25:26.090180+0000","last_clean":"2024-04-23T18:25:26.090180+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:25:26.090180+0000","last_undegraded":"2024-04-23T18:25:26.090180+0000","last_fullsized":"2024-04-23T18:25:26.090180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912562+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:25:25.912562+0000","last_peered":"2024-04-23T18:25:25.912562+0000","last_clean":"2024-04-23T18:25:25.912562+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:25:25.912562+0000","last_undegraded":"2024-04-23T18:25:25.912562+0000","last_fullsized":"2024-04-23T18:25:25.912562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912960+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:25:25.912960+0000","last_peered":"2024-04-23T18:25:25.912960+0000","last_clean":"2024-04-23T18:25:25.912960+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:25:25.912960+0000","last_undegraded":"2024-04-23T18:25:25.912960+0000","last_fullsized":"2024-04-23T18:25:25.912960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.917216+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:25:25.917216+0000","last_peered":"2024-04-23T18:25:25.917216+0000","last_clean":"2024-04-23T18:25:25.917216+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:25:25.917216+0000","last_undegraded":"2024-04-23T18:25:25.917216+0000","last_fullsized":"2024-04-23T18:25:25.917216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:42.462471+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:25:42.462471+0000","last_peered":"2024-04-23T18:25:42.462471+0000","last_clean":"2024-04-23T18:25:42.462471+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:25:42.462471+0000","last_undegraded":"2024-04-23T18:25:42.462471+0000","last_fullsized":"2024-04-23T18:25:42.462471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915556+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:25:25.915556+0000","last_peered":"2024-04-23T18:25:25.915556+0000","last_clean":"2024-04-23T18:25:25.915556+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:25:25.915556+0000","last_undegraded":"2024-04-23T18:25:25.915556+0000","last_fullsized":"2024-04-23T18:25:25.915556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915769+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:25:25.915769+0000","last_peered":"2024-04-23T18:25:25.915769+0000","last_clean":"2024-04-23T18:25:25.915769+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:25:25.915769+0000","last_undegraded":"2024-04-23T18:25:25.915769+0000","last_fullsized":"2024-04-23T18:25:25.915769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909738+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:25:25.909738+0000","last_peered":"2024-04-23T18:25:25.909738+0000","last_clean":"2024-04-23T18:25:25.909738+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:25:25.909738+0000","last_undegraded":"2024-04-23T18:25:25.909738+0000","last_fullsized":"2024-04-23T18:25:25.909738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.932040+0000","last_change":"2024-04-23T18:25:25.931554+0000","last_active":"2024-04-23T18:25:25.932040+0000","last_peered":"2024-04-23T18:25:25.932040+0000","last_clean":"2024-04-23T18:25:25.932040+0000","last_became_active":"2024-04-23T18:25:25.919193+0000","last_became_peered":"2024-04-23T18:25:25.919193+0000","last_unstale":"2024-04-23T18:25:25.932040+0000","last_undegraded":"2024-04-23T18:25:25.932040+0000","last_fullsized":"2024-04-23T18:25:25.932040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:34.700255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090312+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:25:26.090312+0000","last_peered":"2024-04-23T18:25:26.090312+0000","last_clean":"2024-04-23T18:25:26.090312+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:25:26.090312+0000","last_undegraded":"2024-04-23T18:25:26.090312+0000","last_fullsized":"2024-04-23T18:25:26.090312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910607+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:25:25.910607+0000","last_peered":"2024-04-23T18:25:25.910607+0000","last_clean":"2024-04-23T18:25:25.910607+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:25:25.910607+0000","last_undegraded":"2024-04-23T18:25:25.910607+0000","last_fullsized":"2024-04-23T18:25:25.910607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914878+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:25:25.914878+0000","last_peered":"2024-04-23T18:25:25.914878+0000","last_clean":"2024-04-23T18:25:25.914878+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:25:25.914878+0000","last_undegraded":"2024-04-23T18:25:25.914878+0000","last_fullsized":"2024-04-23T18:25:25.914878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:40.455737+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:25:40.455737+0000","last_peered":"2024-04-23T18:25:40.455737+0000","last_clean":"2024-04-23T18:25:40.455737+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:25:40.455737+0000","last_undegraded":"2024-04-23T18:25:40.455737+0000","last_fullsized":"2024-04-23T18:25:40.455737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910119+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:25:25.910119+0000","last_peered":"2024-04-23T18:25:25.910119+0000","last_clean":"2024-04-23T18:25:25.910119+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:25:25.910119+0000","last_undegraded":"2024-04-23T18:25:25.910119+0000","last_fullsized":"2024-04-23T18:25:25.910119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152575+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:25:26.152575+0000","last_peered":"2024-04-23T18:25:26.152575+0000","last_clean":"2024-04-23T18:25:26.152575+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:25:26.152575+0000","last_undegraded":"2024-04-23T18:25:26.152575+0000","last_fullsized":"2024-04-23T18:25:26.152575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.919173+0000","last_change":"2024-04-23T18:25:25.919173+0000","last_active":"2024-04-23T18:25:25.919173+0000","last_peered":"2024-04-23T18:25:25.919173+0000","last_clean":"2024-04-23T18:25:25.919173+0000","last_became_active":"2024-04-23T18:25:25.918830+0000","last_became_peered":"2024-04-23T18:25:25.918830+0000","last_unstale":"2024-04-23T18:25:25.919173+0000","last_undegraded":"2024-04-23T18:25:25.919173+0000","last_fullsized":"2024-04-23T18:25:25.919173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:51.415326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912559+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:25:25.912559+0000","last_peered":"2024-04-23T18:25:25.912559+0000","last_clean":"2024-04-23T18:25:25.912559+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:25:25.912559+0000","last_undegraded":"2024-04-23T18:25:25.912559+0000","last_fullsized":"2024-04-23T18:25:25.912559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915357+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:25:25.915357+0000","last_peered":"2024-04-23T18:25:25.915357+0000","last_clean":"2024-04-23T18:25:25.915357+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:25:25.915357+0000","last_undegraded":"2024-04-23T18:25:25.915357+0000","last_fullsized":"2024-04-23T18:25:25.915357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914369+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:25:25.914369+0000","last_peered":"2024-04-23T18:25:25.914369+0000","last_clean":"2024-04-23T18:25:25.914369+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:25:25.914369+0000","last_undegraded":"2024-04-23T18:25:25.914369+0000","last_fullsized":"2024-04-23T18:25:25.914369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.153727+0000","last_change":"2024-04-23T18:25:26.153727+0000","last_active":"2024-04-23T18:25:26.153727+0000","last_peered":"2024-04-23T18:25:26.153727+0000","last_clean":"2024-04-23T18:25:26.153727+0000","last_became_active":"2024-04-23T18:25:26.153420+0000","last_became_peered":"2024-04-23T18:25:26.153420+0000","last_unstale":"2024-04-23T18:25:26.153727+0000","last_undegraded":"2024-04-23T18:25:26.153727+0000","last_fullsized":"2024-04-23T18:25:26.153727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:20.614833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":155,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:39.453729+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:25:39.453729+0000","last_peered":"2024-04-23T18:25:39.453729+0000","last_clean":"2024-04-23T18:25:39.453729+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:25:39.453729+0000","last_undegraded":"2024-04-23T18:25:39.453729+0000","last_fullsized":"2024-04-23T18:25:39.453729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":132,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090540+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:25:26.090540+0000","last_peered":"2024-04-23T18:25:26.090540+0000","last_clean":"2024-04-23T18:25:26.090540+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:25:26.090540+0000","last_undegraded":"2024-04-23T18:25:26.090540+0000","last_fullsized":"2024-04-23T18:25:26.090540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910534+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:25:25.910534+0000","last_peered":"2024-04-23T18:25:25.910534+0000","last_clean":"2024-04-23T18:25:25.910534+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:25:25.910534+0000","last_undegraded":"2024-04-23T18:25:25.910534+0000","last_fullsized":"2024-04-23T18:25:25.910534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916633+0000","last_change":"2024-04-23T18:25:25.916633+0000","last_active":"2024-04-23T18:25:25.916633+0000","last_peered":"2024-04-23T18:25:25.916633+0000","last_clean":"2024-04-23T18:25:25.916633+0000","last_became_active":"2024-04-23T18:25:25.916224+0000","last_became_peered":"2024-04-23T18:25:25.916224+0000","last_unstale":"2024-04-23T18:25:25.916633+0000","last_undegraded":"2024-04-23T18:25:25.916633+0000","last_fullsized":"2024-04-23T18:25:25.916633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:10.172049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915007+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:25:25.915007+0000","last_peered":"2024-04-23T18:25:25.915007+0000","last_clean":"2024-04-23T18:25:25.915007+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:25:25.915007+0000","last_undegraded":"2024-04-23T18:25:25.915007+0000","last_fullsized":"2024-04-23T18:25:25.915007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090488+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:25:26.090488+0000","last_peered":"2024-04-23T18:25:26.090488+0000","last_clean":"2024-04-23T18:25:26.090488+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:25:26.090488+0000","last_undegraded":"2024-04-23T18:25:26.090488+0000","last_fullsized":"2024-04-23T18:25:26.090488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'32","reported_seq":189,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:39.470394+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:25:39.470394+0000","last_peered":"2024-04-23T18:25:39.470394+0000","last_clean":"2024-04-23T18:25:39.470394+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:25:39.470394+0000","last_undegraded":"2024-04-23T18:25:39.470394+0000","last_fullsized":"2024-04-23T18:25:39.470394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":138,"num_read_kb":127,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910391+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:25:25.910391+0000","last_peered":"2024-04-23T18:25:25.910391+0000","last_clean":"2024-04-23T18:25:25.910391+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:25:25.910391+0000","last_undegraded":"2024-04-23T18:25:25.910391+0000","last_fullsized":"2024-04-23T18:25:25.910391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916548+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:25:25.916548+0000","last_peered":"2024-04-23T18:25:25.916548+0000","last_clean":"2024-04-23T18:25:25.916548+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:25:25.916548+0000","last_undegraded":"2024-04-23T18:25:25.916548+0000","last_fullsized":"2024-04-23T18:25:25.916548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912594+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:25:25.912594+0000","last_peered":"2024-04-23T18:25:25.912594+0000","last_clean":"2024-04-23T18:25:25.912594+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:25:25.912594+0000","last_undegraded":"2024-04-23T18:25:25.912594+0000","last_fullsized":"2024-04-23T18:25:25.912594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914981+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:25:25.914981+0000","last_peered":"2024-04-23T18:25:25.914981+0000","last_clean":"2024-04-23T18:25:25.914981+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:25:25.914981+0000","last_undegraded":"2024-04-23T18:25:25.914981+0000","last_fullsized":"2024-04-23T18:25:25.914981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910519+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:25:25.910519+0000","last_peered":"2024-04-23T18:25:25.910519+0000","last_clean":"2024-04-23T18:25:25.910519+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:25:25.910519+0000","last_undegraded":"2024-04-23T18:25:25.910519+0000","last_fullsized":"2024-04-23T18:25:25.910519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.457585+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:25:41.457585+0000","last_peered":"2024-04-23T18:25:41.457585+0000","last_clean":"2024-04-23T18:25:41.457585+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:25:41.457585+0000","last_undegraded":"2024-04-23T18:25:41.457585+0000","last_fullsized":"2024-04-23T18:25:41.457585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910049+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:25:25.910049+0000","last_peered":"2024-04-23T18:25:25.910049+0000","last_clean":"2024-04-23T18:25:25.910049+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:25:25.910049+0000","last_undegraded":"2024-04-23T18:25:25.910049+0000","last_fullsized":"2024-04-23T18:25:25.910049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909951+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:25:25.909951+0000","last_peered":"2024-04-23T18:25:25.909951+0000","last_clean":"2024-04-23T18:25:25.909951+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:25:25.909951+0000","last_undegraded":"2024-04-23T18:25:25.909951+0000","last_fullsized":"2024-04-23T18:25:25.909951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915697+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:25:25.915697+0000","last_peered":"2024-04-23T18:25:25.915697+0000","last_clean":"2024-04-23T18:25:25.915697+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:25:25.915697+0000","last_undegraded":"2024-04-23T18:25:25.915697+0000","last_fullsized":"2024-04-23T18:25:25.915697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910447+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:25:25.910447+0000","last_peered":"2024-04-23T18:25:25.910447+0000","last_clean":"2024-04-23T18:25:25.910447+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:25:25.910447+0000","last_undegraded":"2024-04-23T18:25:25.910447+0000","last_fullsized":"2024-04-23T18:25:25.910447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912181+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:25:25.912181+0000","last_peered":"2024-04-23T18:25:25.912181+0000","last_clean":"2024-04-23T18:25:25.912181+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:25:25.912181+0000","last_undegraded":"2024-04-23T18:25:25.912181+0000","last_fullsized":"2024-04-23T18:25:25.912181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913636+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:25:25.913636+0000","last_peered":"2024-04-23T18:25:25.913636+0000","last_clean":"2024-04-23T18:25:25.913636+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:25:25.913636+0000","last_undegraded":"2024-04-23T18:25:25.913636+0000","last_fullsized":"2024-04-23T18:25:25.913636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152950+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:25:26.152950+0000","last_peered":"2024-04-23T18:25:26.152950+0000","last_clean":"2024-04-23T18:25:26.152950+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:25:26.152950+0000","last_undegraded":"2024-04-23T18:25:26.152950+0000","last_fullsized":"2024-04-23T18:25:26.152950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:40.457649+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:25:40.457649+0000","last_peered":"2024-04-23T18:25:40.457649+0000","last_clean":"2024-04-23T18:25:40.457649+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:25:40.457649+0000","last_undegraded":"2024-04-23T18:25:40.457649+0000","last_fullsized":"2024-04-23T18:25:40.457649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152792+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:25:26.152792+0000","last_peered":"2024-04-23T18:25:26.152792+0000","last_clean":"2024-04-23T18:25:26.152792+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:25:26.152792+0000","last_undegraded":"2024-04-23T18:25:26.152792+0000","last_fullsized":"2024-04-23T18:25:26.152792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914583+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:25:25.914583+0000","last_peered":"2024-04-23T18:25:25.914583+0000","last_clean":"2024-04-23T18:25:25.914583+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:25:25.914583+0000","last_undegraded":"2024-04-23T18:25:25.914583+0000","last_fullsized":"2024-04-23T18:25:25.914583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.456892+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:25:41.456892+0000","last_peered":"2024-04-23T18:25:41.456892+0000","last_clean":"2024-04-23T18:25:41.456892+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:25:41.456892+0000","last_undegraded":"2024-04-23T18:25:41.456892+0000","last_fullsized":"2024-04-23T18:25:41.456892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.092051+0000","last_change":"2024-04-23T18:25:26.091646+0000","last_active":"2024-04-23T18:25:26.092051+0000","last_peered":"2024-04-23T18:25:26.092051+0000","last_clean":"2024-04-23T18:25:26.092051+0000","last_became_active":"2024-04-23T18:25:26.091303+0000","last_became_peered":"2024-04-23T18:25:26.091303+0000","last_unstale":"2024-04-23T18:25:26.092051+0000","last_undegraded":"2024-04-23T18:25:26.092051+0000","last_fullsized":"2024-04-23T18:25:26.092051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:27.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101116+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:25:26.101116+0000","last_peered":"2024-04-23T18:25:26.101116+0000","last_clean":"2024-04-23T18:25:26.101116+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:25:26.101116+0000","last_undegraded":"2024-04-23T18:25:26.101116+0000","last_fullsized":"2024-04-23T18:25:26.101116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910096+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:25:25.910096+0000","last_peered":"2024-04-23T18:25:25.910096+0000","last_clean":"2024-04-23T18:25:25.910096+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:25:25.910096+0000","last_undegraded":"2024-04-23T18:25:25.910096+0000","last_fullsized":"2024-04-23T18:25:25.910096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912407+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:25:25.912407+0000","last_peered":"2024-04-23T18:25:25.912407+0000","last_clean":"2024-04-23T18:25:25.912407+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:25:25.912407+0000","last_undegraded":"2024-04-23T18:25:25.912407+0000","last_fullsized":"2024-04-23T18:25:25.912407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910215+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:25:25.910215+0000","last_peered":"2024-04-23T18:25:25.910215+0000","last_clean":"2024-04-23T18:25:25.910215+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:25:25.910215+0000","last_undegraded":"2024-04-23T18:25:25.910215+0000","last_fullsized":"2024-04-23T18:25:25.910215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910523+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:25:25.910523+0000","last_peered":"2024-04-23T18:25:25.910523+0000","last_clean":"2024-04-23T18:25:25.910523+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:25:25.910523+0000","last_undegraded":"2024-04-23T18:25:25.910523+0000","last_fullsized":"2024-04-23T18:25:25.910523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912868+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:25:25.912868+0000","last_peered":"2024-04-23T18:25:25.912868+0000","last_clean":"2024-04-23T18:25:25.912868+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:25:25.912868+0000","last_undegraded":"2024-04-23T18:25:25.912868+0000","last_fullsized":"2024-04-23T18:25:25.912868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913016+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:25:25.913016+0000","last_peered":"2024-04-23T18:25:25.913016+0000","last_clean":"2024-04-23T18:25:25.913016+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:25:25.913016+0000","last_undegraded":"2024-04-23T18:25:25.913016+0000","last_fullsized":"2024-04-23T18:25:25.913016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916403+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:25:25.916403+0000","last_peered":"2024-04-23T18:25:25.916403+0000","last_clean":"2024-04-23T18:25:25.916403+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:25:25.916403+0000","last_undegraded":"2024-04-23T18:25:25.916403+0000","last_fullsized":"2024-04-23T18:25:25.916403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101304+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:25:26.101304+0000","last_peered":"2024-04-23T18:25:26.101304+0000","last_clean":"2024-04-23T18:25:26.101304+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:25:26.101304+0000","last_undegraded":"2024-04-23T18:25:26.101304+0000","last_fullsized":"2024-04-23T18:25:26.101304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909646+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:25:25.909646+0000","last_peered":"2024-04-23T18:25:25.909646+0000","last_clean":"2024-04-23T18:25:25.909646+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:25:25.909646+0000","last_undegraded":"2024-04-23T18:25:25.909646+0000","last_fullsized":"2024-04-23T18:25:25.909646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090267+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:25:26.090267+0000","last_peered":"2024-04-23T18:25:26.090267+0000","last_clean":"2024-04-23T18:25:26.090267+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:25:26.090267+0000","last_undegraded":"2024-04-23T18:25:26.090267+0000","last_fullsized":"2024-04-23T18:25:26.090267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909943+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:25:25.909943+0000","last_peered":"2024-04-23T18:25:25.909943+0000","last_clean":"2024-04-23T18:25:25.909943+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:25:25.909943+0000","last_undegraded":"2024-04-23T18:25:25.909943+0000","last_fullsized":"2024-04-23T18:25:25.909943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915935+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:25:25.915935+0000","last_peered":"2024-04-23T18:25:25.915935+0000","last_clean":"2024-04-23T18:25:25.915935+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:25:25.915935+0000","last_undegraded":"2024-04-23T18:25:25.915935+0000","last_fullsized":"2024-04-23T18:25:25.915935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912218+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:25:25.912218+0000","last_peered":"2024-04-23T18:25:25.912218+0000","last_clean":"2024-04-23T18:25:25.912218+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:25:25.912218+0000","last_undegraded":"2024-04-23T18:25:25.912218+0000","last_fullsized":"2024-04-23T18:25:25.912218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.935896+0000","last_change":"2024-04-23T18:25:25.935443+0000","last_active":"2024-04-23T18:25:25.935896+0000","last_peered":"2024-04-23T18:25:25.935896+0000","last_clean":"2024-04-23T18:25:25.935896+0000","last_became_active":"2024-04-23T18:25:25.919406+0000","last_became_peered":"2024-04-23T18:25:25.919406+0000","last_unstale":"2024-04-23T18:25:25.935896+0000","last_undegraded":"2024-04-23T18:25:25.935896+0000","last_fullsized":"2024-04-23T18:25:25.935896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:10:03.007617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915576+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:25:25.915576+0000","last_peered":"2024-04-23T18:25:25.915576+0000","last_clean":"2024-04-23T18:25:25.915576+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:25:25.915576+0000","last_undegraded":"2024-04-23T18:25:25.915576+0000","last_fullsized":"2024-04-23T18:25:25.915576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910051+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:25:25.910051+0000","last_peered":"2024-04-23T18:25:25.910051+0000","last_clean":"2024-04-23T18:25:25.910051+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:25:25.910051+0000","last_undegraded":"2024-04-23T18:25:25.910051+0000","last_fullsized":"2024-04-23T18:25:25.910051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910474+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:25:25.910474+0000","last_peered":"2024-04-23T18:25:25.910474+0000","last_clean":"2024-04-23T18:25:25.910474+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:25:25.910474+0000","last_undegraded":"2024-04-23T18:25:25.910474+0000","last_fullsized":"2024-04-23T18:25:25.910474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.467618+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:25:41.467618+0000","last_peered":"2024-04-23T18:25:41.467618+0000","last_clean":"2024-04-23T18:25:41.467618+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:25:41.467618+0000","last_undegraded":"2024-04-23T18:25:41.467618+0000","last_fullsized":"2024-04-23T18:25:41.467618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910665+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:25:25.910665+0000","last_peered":"2024-04-23T18:25:25.910665+0000","last_clean":"2024-04-23T18:25:25.910665+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:25:25.910665+0000","last_undegraded":"2024-04-23T18:25:25.910665+0000","last_fullsized":"2024-04-23T18:25:25.910665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915000+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:25:25.915000+0000","last_peered":"2024-04-23T18:25:25.915000+0000","last_clean":"2024-04-23T18:25:25.915000+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:25:25.915000+0000","last_undegraded":"2024-04-23T18:25:25.915000+0000","last_fullsized":"2024-04-23T18:25:25.915000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910446+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:25:25.910446+0000","last_peered":"2024-04-23T18:25:25.910446+0000","last_clean":"2024-04-23T18:25:25.910446+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:25:25.910446+0000","last_undegraded":"2024-04-23T18:25:25.910446+0000","last_fullsized":"2024-04-23T18:25:25.910446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.106394+0000","last_change":"2024-04-23T18:25:26.106394+0000","last_active":"2024-04-23T18:25:26.106394+0000","last_peered":"2024-04-23T18:25:26.106394+0000","last_clean":"2024-04-23T18:25:26.106394+0000","last_became_active":"2024-04-23T18:25:26.102178+0000","last_became_peered":"2024-04-23T18:25:26.102178+0000","last_unstale":"2024-04-23T18:25:26.106394+0000","last_undegraded":"2024-04-23T18:25:26.106394+0000","last_fullsized":"2024-04-23T18:25:26.106394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:20.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910717+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:25:25.910717+0000","last_peered":"2024-04-23T18:25:25.910717+0000","last_clean":"2024-04-23T18:25:25.910717+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:25:25.910717+0000","last_undegraded":"2024-04-23T18:25:25.910717+0000","last_fullsized":"2024-04-23T18:25:25.910717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912294+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:25:25.912294+0000","last_peered":"2024-04-23T18:25:25.912294+0000","last_clean":"2024-04-23T18:25:25.912294+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:25:25.912294+0000","last_undegraded":"2024-04-23T18:25:25.912294+0000","last_fullsized":"2024-04-23T18:25:25.912294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916821+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:25:25.916821+0000","last_peered":"2024-04-23T18:25:25.916821+0000","last_clean":"2024-04-23T18:25:25.916821+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:25:25.916821+0000","last_undegraded":"2024-04-23T18:25:25.916821+0000","last_fullsized":"2024-04-23T18:25:25.916821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.918437+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:25:25.918437+0000","last_peered":"2024-04-23T18:25:25.918437+0000","last_clean":"2024-04-23T18:25:25.918437+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:25:25.918437+0000","last_undegraded":"2024-04-23T18:25:25.918437+0000","last_fullsized":"2024-04-23T18:25:25.918437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909391+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:25:25.909391+0000","last_peered":"2024-04-23T18:25:25.909391+0000","last_clean":"2024-04-23T18:25:25.909391+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:25:25.909391+0000","last_undegraded":"2024-04-23T18:25:25.909391+0000","last_fullsized":"2024-04-23T18:25:25.909391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912810+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:25:25.912810+0000","last_peered":"2024-04-23T18:25:25.912810+0000","last_clean":"2024-04-23T18:25:25.912810+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:25:25.912810+0000","last_undegraded":"2024-04-23T18:25:25.912810+0000","last_fullsized":"2024-04-23T18:25:25.912810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915473+0000","last_change":"2024-04-23T18:25:25.915473+0000","last_active":"2024-04-23T18:25:25.915473+0000","last_peered":"2024-04-23T18:25:25.915473+0000","last_clean":"2024-04-23T18:25:25.915473+0000","last_became_active":"2024-04-23T18:25:25.914874+0000","last_became_peered":"2024-04-23T18:25:25.914874+0000","last_unstale":"2024-04-23T18:25:25.915473+0000","last_undegraded":"2024-04-23T18:25:25.915473+0000","last_fullsized":"2024-04-23T18:25:25.915473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:14.348994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.918150+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:25:25.918150+0000","last_peered":"2024-04-23T18:25:25.918150+0000","last_clean":"2024-04-23T18:25:25.918150+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:25:25.918150+0000","last_undegraded":"2024-04-23T18:25:25.918150+0000","last_fullsized":"2024-04-23T18:25:25.918150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916300+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:25:25.916300+0000","last_peered":"2024-04-23T18:25:25.916300+0000","last_clean":"2024-04-23T18:25:25.916300+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:25:25.916300+0000","last_undegraded":"2024-04-23T18:25:25.916300+0000","last_fullsized":"2024-04-23T18:25:25.916300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913657+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:25:25.913657+0000","last_peered":"2024-04-23T18:25:25.913657+0000","last_clean":"2024-04-23T18:25:25.913657+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:25:25.913657+0000","last_undegraded":"2024-04-23T18:25:25.913657+0000","last_fullsized":"2024-04-23T18:25:25.913657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915800+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:25:25.915800+0000","last_peered":"2024-04-23T18:25:25.915800+0000","last_clean":"2024-04-23T18:25:25.915800+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:25:25.915800+0000","last_undegraded":"2024-04-23T18:25:25.915800+0000","last_fullsized":"2024-04-23T18:25:25.915800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090743+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:25:26.090743+0000","last_peered":"2024-04-23T18:25:26.090743+0000","last_clean":"2024-04-23T18:25:26.090743+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:25:26.090743+0000","last_undegraded":"2024-04-23T18:25:26.090743+0000","last_fullsized":"2024-04-23T18:25:26.090743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912808+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:25:25.912808+0000","last_peered":"2024-04-23T18:25:25.912808+0000","last_clean":"2024-04-23T18:25:25.912808+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:25:25.912808+0000","last_undegraded":"2024-04-23T18:25:25.912808+0000","last_fullsized":"2024-04-23T18:25:25.912808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.917314+0000","last_change":"2024-04-23T18:25:25.917314+0000","last_active":"2024-04-23T18:25:25.917314+0000","last_peered":"2024-04-23T18:25:25.917314+0000","last_clean":"2024-04-23T18:25:25.917314+0000","last_became_active":"2024-04-23T18:25:25.916841+0000","last_became_peered":"2024-04-23T18:25:25.916841+0000","last_unstale":"2024-04-23T18:25:25.917314+0000","last_undegraded":"2024-04-23T18:25:25.917314+0000","last_fullsized":"2024-04-23T18:25:25.917314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:03.413973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280434,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67227,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":711,"num_read_kb":426,"num_write":239,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":456894,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1108337,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456824,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":457202,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1050111,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456541,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":517477,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":457880,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":518088,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456366,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456793,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1049398,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:25:45.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-23T18:25:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:46.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:46.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:46.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:46.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:45 smithi033 ceph-mon[29274]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:46.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:46.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:45 smithi122 ceph-mon[32546]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:25:46.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:25:46.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:45 smithi106 ceph-mon[32716]: from='mgr.14152 172.21.15.33:0/3221768340' entity='mgr.x' 2024-04-23T18:25:47.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:46 smithi033 ceph-mon[29274]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:47.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:46 smithi106 ceph-mon[32716]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:47.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:46 smithi122 ceph-mon[32546]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 -- 172.21.15.33:0/2170755332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0102ac0 msgr2=0x7fedc0102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 --2- 172.21.15.33:0/2170755332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0102ac0 0x7fedc0102f40 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fedb4009990 tx=0x7fedb4009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 -- 172.21.15.33:0/2170755332 shutdown_connections 2024-04-23T18:25:47.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 --2- 172.21.15.33:0/2170755332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0103480 0x7fedc010f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 --2- 172.21.15.33:0/2170755332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0108ab0 0x7fedc0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 --2- 172.21.15.33:0/2170755332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0102ac0 0x7fedc0102f40 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 -- 172.21.15.33:0/2170755332 >> 172.21.15.33:0/2170755332 conn(0x7fedc0079bf0 msgr2=0x7fedc007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:47.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 -- 172.21.15.33:0/2170755332 shutdown_connections 2024-04-23T18:25:47.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.692+0000 7fedc75f2700 1 -- 172.21.15.33:0/2170755332 wait complete. 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 Processor -- start 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 -- start start 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0102ac0 0x7fedc01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0103480 0x7fedc0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0112d40 con 0x7fedc0108ab0 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0112bc0 con 0x7fedc0102ac0 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc75f2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedc0112ec0 con 0x7fedc0103480 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc5b8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc5b8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32820/0 (socket says 172.21.15.33:32820) 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc5b8f700 1 -- 172.21.15.33:0/3240154237 learned_addr learned my addr 172.21.15.33:0/3240154237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:47.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc4b8d700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0103480 0x7fedc0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.693+0000 7fedc538e700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0102ac0 0x7fedc01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 -- 172.21.15.33:0/3240154237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0103480 msgr2=0x7fedc0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0103480 0x7fedc0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 -- 172.21.15.33:0/3240154237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0102ac0 msgr2=0x7fedc01992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0102ac0 0x7fedc01992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 -- 172.21.15.33:0/3240154237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc019e2e0 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc5b8f700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedb000b970 tx=0x7fedb000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0019070 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb000cde0 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc019e5d0 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0007370 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.694+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc01a5e50 con 0x7fedc0108ab0 2024-04-23T18:25:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.695+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc0069730 con 0x7fedc0108ab0 2024-04-23T18:25:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.696+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fedb001c070 con 0x7fedc0108ab0 2024-04-23T18:25:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.697+0000 7fedbe7fc700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 0x7feda80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.697+0000 7fedc538e700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 0x7feda80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.697+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fedb00d0100 con 0x7fedc0108ab0 2024-04-23T18:25:47.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.698+0000 7fedc538e700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 0x7feda80a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005950 tx=0x7fedb80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:47.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.701+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedb0023070 con 0x7fedc0108ab0 2024-04-23T18:25:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.902+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedc019a630 con 0x7feda80a3cd0 2024-04-23T18:25:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.909+0000 7fedbe7fc700 1 -- 172.21.15.33:0/3240154237 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259022 (secure 0 0 0) 0x7fedc019a630 con 0x7feda80a3cd0 2024-04-23T18:25:47.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 msgr2=0x7feda80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 0x7feda80a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005950 tx=0x7fedb80058e0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 msgr2=0x7fedc019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fedb000b970 tx=0x7fedb000bc80 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 shutdown_connections 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7feda80a3cd0 0x7feda80a6190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedc0108ab0 0x7fedc019dba0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedc0102ac0 0x7fedc01992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 --2- 172.21.15.33:0/3240154237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedc0103480 0x7fedc0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 >> 172.21.15.33:0/3240154237 conn(0x7fedc0079bf0 msgr2=0x7fedc0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 shutdown_connections 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:47.912+0000 7fedc75f2700 1 -- 172.21.15.33:0/3240154237 wait complete. 2024-04-23T18:25:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:25:48.215 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:47 smithi033 ceph-mon[29274]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:48.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:47 smithi106 ceph-mon[32716]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:48.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:47 smithi122 ceph-mon[32546]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:25:48.564 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-04-23T18:25:46.174508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657829,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":867,"num_read_kb":560,"num_write":422,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381748,"kb_used_data":11712,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560076,"statfs":{"total":1151940427776,"available":1151549517824,"internally_reserved":0,"allocated":11993088,"data_stored":7441911,"data_compressed":176862,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":65,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003028"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912449+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:25:25.912449+0000","last_peered":"2024-04-23T18:25:25.912449+0000","last_clean":"2024-04-23T18:25:25.912449+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:25:25.912449+0000","last_undegraded":"2024-04-23T18:25:25.912449+0000","last_fullsized":"2024-04-23T18:25:25.912449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909607+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:25:25.909607+0000","last_peered":"2024-04-23T18:25:25.909607+0000","last_clean":"2024-04-23T18:25:25.909607+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:25:25.909607+0000","last_undegraded":"2024-04-23T18:25:25.909607+0000","last_fullsized":"2024-04-23T18:25:25.909607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909532+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:25:25.909532+0000","last_peered":"2024-04-23T18:25:25.909532+0000","last_clean":"2024-04-23T18:25:25.909532+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:25:25.909532+0000","last_undegraded":"2024-04-23T18:25:25.909532+0000","last_fullsized":"2024-04-23T18:25:25.909532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101564+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:25:26.101564+0000","last_peered":"2024-04-23T18:25:26.101564+0000","last_clean":"2024-04-23T18:25:26.101564+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:25:26.101564+0000","last_undegraded":"2024-04-23T18:25:26.101564+0000","last_fullsized":"2024-04-23T18:25:26.101564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910158+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:25:25.910158+0000","last_peered":"2024-04-23T18:25:25.910158+0000","last_clean":"2024-04-23T18:25:25.910158+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:25:25.910158+0000","last_undegraded":"2024-04-23T18:25:25.910158+0000","last_fullsized":"2024-04-23T18:25:25.910158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152686+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:25:26.152686+0000","last_peered":"2024-04-23T18:25:26.152686+0000","last_clean":"2024-04-23T18:25:26.152686+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:25:26.152686+0000","last_undegraded":"2024-04-23T18:25:26.152686+0000","last_fullsized":"2024-04-23T18:25:26.152686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910122+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:25:25.910122+0000","last_peered":"2024-04-23T18:25:25.910122+0000","last_clean":"2024-04-23T18:25:25.910122+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:25:25.910122+0000","last_undegraded":"2024-04-23T18:25:25.910122+0000","last_fullsized":"2024-04-23T18:25:25.910122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915474+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:25:25.915474+0000","last_peered":"2024-04-23T18:25:25.915474+0000","last_clean":"2024-04-23T18:25:25.915474+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:25:25.915474+0000","last_undegraded":"2024-04-23T18:25:25.915474+0000","last_fullsized":"2024-04-23T18:25:25.915474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090668+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:25:26.090668+0000","last_peered":"2024-04-23T18:25:26.090668+0000","last_clean":"2024-04-23T18:25:26.090668+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:25:26.090668+0000","last_undegraded":"2024-04-23T18:25:26.090668+0000","last_fullsized":"2024-04-23T18:25:26.090668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.153093+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:25:26.153093+0000","last_peered":"2024-04-23T18:25:26.153093+0000","last_clean":"2024-04-23T18:25:26.153093+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:25:26.153093+0000","last_undegraded":"2024-04-23T18:25:26.153093+0000","last_fullsized":"2024-04-23T18:25:26.153093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910649+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:25:25.910649+0000","last_peered":"2024-04-23T18:25:25.910649+0000","last_clean":"2024-04-23T18:25:25.910649+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:25:25.910649+0000","last_undegraded":"2024-04-23T18:25:25.910649+0000","last_fullsized":"2024-04-23T18:25:25.910649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101498+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:25:26.101498+0000","last_peered":"2024-04-23T18:25:26.101498+0000","last_clean":"2024-04-23T18:25:26.101498+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:25:26.101498+0000","last_undegraded":"2024-04-23T18:25:26.101498+0000","last_fullsized":"2024-04-23T18:25:26.101498+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912688+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:25:25.912688+0000","last_peered":"2024-04-23T18:25:25.912688+0000","last_clean":"2024-04-23T18:25:25.912688+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:25:25.912688+0000","last_undegraded":"2024-04-23T18:25:25.912688+0000","last_fullsized":"2024-04-23T18:25:25.912688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909244+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:25:25.909244+0000","last_peered":"2024-04-23T18:25:25.909244+0000","last_clean":"2024-04-23T18:25:25.909244+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:25:25.909244+0000","last_undegraded":"2024-04-23T18:25:25.909244+0000","last_fullsized":"2024-04-23T18:25:25.909244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909207+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:25:25.909207+0000","last_peered":"2024-04-23T18:25:25.909207+0000","last_clean":"2024-04-23T18:25:25.909207+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:25:25.909207+0000","last_undegraded":"2024-04-23T18:25:25.909207+0000","last_fullsized":"2024-04-23T18:25:25.909207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090180+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:25:26.090180+0000","last_peered":"2024-04-23T18:25:26.090180+0000","last_clean":"2024-04-23T18:25:26.090180+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:25:26.090180+0000","last_undegraded":"2024-04-23T18:25:26.090180+0000","last_fullsized":"2024-04-23T18:25:26.090180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912562+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:25:25.912562+0000","last_peered":"2024-04-23T18:25:25.912562+0000","last_clean":"2024-04-23T18:25:25.912562+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:25:25.912562+0000","last_undegraded":"2024-04-23T18:25:25.912562+0000","last_fullsized":"2024-04-23T18:25:25.912562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912960+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:25:25.912960+0000","last_peered":"2024-04-23T18:25:25.912960+0000","last_clean":"2024-04-23T18:25:25.912960+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:25:25.912960+0000","last_undegraded":"2024-04-23T18:25:25.912960+0000","last_fullsized":"2024-04-23T18:25:25.912960+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.917216+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:25:25.917216+0000","last_peered":"2024-04-23T18:25:25.917216+0000","last_clean":"2024-04-23T18:25:25.917216+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:25:25.917216+0000","last_undegraded":"2024-04-23T18:25:25.917216+0000","last_fullsized":"2024-04-23T18:25:25.917216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'13","reported_seq":103,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:42.462471+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:25:42.462471+0000","last_peered":"2024-04-23T18:25:42.462471+0000","last_clean":"2024-04-23T18:25:42.462471+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:25:42.462471+0000","last_undegraded":"2024-04-23T18:25:42.462471+0000","last_fullsized":"2024-04-23T18:25:42.462471+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915556+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:25:25.915556+0000","last_peered":"2024-04-23T18:25:25.915556+0000","last_clean":"2024-04-23T18:25:25.915556+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:25:25.915556+0000","last_undegraded":"2024-04-23T18:25:25.915556+0000","last_fullsized":"2024-04-23T18:25:25.915556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915769+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:25:25.915769+0000","last_peered":"2024-04-23T18:25:25.915769+0000","last_clean":"2024-04-23T18:25:25.915769+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:25:25.915769+0000","last_undegraded":"2024-04-23T18:25:25.915769+0000","last_fullsized":"2024-04-23T18:25:25.915769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909738+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:25:25.909738+0000","last_peered":"2024-04-23T18:25:25.909738+0000","last_clean":"2024-04-23T18:25:25.909738+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:25:25.909738+0000","last_undegraded":"2024-04-23T18:25:25.909738+0000","last_fullsized":"2024-04-23T18:25:25.909738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.932040+0000","last_change":"2024-04-23T18:25:25.931554+0000","last_active":"2024-04-23T18:25:25.932040+0000","last_peered":"2024-04-23T18:25:25.932040+0000","last_clean":"2024-04-23T18:25:25.932040+0000","last_became_active":"2024-04-23T18:25:25.919193+0000","last_became_peered":"2024-04-23T18:25:25.919193+0000","last_unstale":"2024-04-23T18:25:25.932040+0000","last_undegraded":"2024-04-23T18:25:25.932040+0000","last_fullsized":"2024-04-23T18:25:25.932040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:34.700255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090312+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:25:26.090312+0000","last_peered":"2024-04-23T18:25:26.090312+0000","last_clean":"2024-04-23T18:25:26.090312+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:25:26.090312+0000","last_undegraded":"2024-04-23T18:25:26.090312+0000","last_fullsized":"2024-04-23T18:25:26.090312+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910607+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:25:25.910607+0000","last_peered":"2024-04-23T18:25:25.910607+0000","last_clean":"2024-04-23T18:25:25.910607+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:25:25.910607+0000","last_undegraded":"2024-04-23T18:25:25.910607+0000","last_fullsized":"2024-04-23T18:25:25.910607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914878+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:25:25.914878+0000","last_peered":"2024-04-23T18:25:25.914878+0000","last_clean":"2024-04-23T18:25:25.914878+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:25:25.914878+0000","last_undegraded":"2024-04-23T18:25:25.914878+0000","last_fullsized":"2024-04-23T18:25:25.914878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:40.455737+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:25:40.455737+0000","last_peered":"2024-04-23T18:25:40.455737+0000","last_clean":"2024-04-23T18:25:40.455737+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:25:40.455737+0000","last_undegraded":"2024-04-23T18:25:40.455737+0000","last_fullsized":"2024-04-23T18:25:40.455737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910119+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:25:25.910119+0000","last_peered":"2024-04-23T18:25:25.910119+0000","last_clean":"2024-04-23T18:25:25.910119+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:25:25.910119+0000","last_undegraded":"2024-04-23T18:25:25.910119+0000","last_fullsized":"2024-04-23T18:25:25.910119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152575+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:25:26.152575+0000","last_peered":"2024-04-23T18:25:26.152575+0000","last_clean":"2024-04-23T18:25:26.152575+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:25:26.152575+0000","last_undegraded":"2024-04-23T18:25:26.152575+0000","last_fullsized":"2024-04-23T18:25:26.152575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.919173+0000","last_change":"2024-04-23T18:25:25.919173+0000","last_active":"2024-04-23T18:25:25.919173+0000","last_peered":"2024-04-23T18:25:25.919173+0000","last_clean":"2024-04-23T18:25:25.919173+0000","last_became_active":"2024-04-23T18:25:25.918830+0000","last_became_peered":"2024-04-23T18:25:25.918830+0000","last_unstale":"2024-04-23T18:25:25.919173+0000","last_undegraded":"2024-04-23T18:25:25.919173+0000","last_fullsized":"2024-04-23T18:25:25.919173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:51.415326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912559+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:25:25.912559+0000","last_peered":"2024-04-23T18:25:25.912559+0000","last_clean":"2024-04-23T18:25:25.912559+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:25:25.912559+0000","last_undegraded":"2024-04-23T18:25:25.912559+0000","last_fullsized":"2024-04-23T18:25:25.912559+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915357+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:25:25.915357+0000","last_peered":"2024-04-23T18:25:25.915357+0000","last_clean":"2024-04-23T18:25:25.915357+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:25:25.915357+0000","last_undegraded":"2024-04-23T18:25:25.915357+0000","last_fullsized":"2024-04-23T18:25:25.915357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914369+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:25:25.914369+0000","last_peered":"2024-04-23T18:25:25.914369+0000","last_clean":"2024-04-23T18:25:25.914369+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:25:25.914369+0000","last_undegraded":"2024-04-23T18:25:25.914369+0000","last_fullsized":"2024-04-23T18:25:25.914369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.153727+0000","last_change":"2024-04-23T18:25:26.153727+0000","last_active":"2024-04-23T18:25:26.153727+0000","last_peered":"2024-04-23T18:25:26.153727+0000","last_clean":"2024-04-23T18:25:26.153727+0000","last_became_active":"2024-04-23T18:25:26.153420+0000","last_became_peered":"2024-04-23T18:25:26.153420+0000","last_unstale":"2024-04-23T18:25:26.153727+0000","last_undegraded":"2024-04-23T18:25:26.153727+0000","last_fullsized":"2024-04-23T18:25:26.153727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:20.614833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'4","reported_seq":165,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:44.460788+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:25:44.460788+0000","last_peered":"2024-04-23T18:25:44.460788+0000","last_clean":"2024-04-23T18:25:44.460788+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:25:44.460788+0000","last_undegraded":"2024-04-23T18:25:44.460788+0000","last_fullsized":"2024-04-23T18:25:44.460788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090540+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:25:26.090540+0000","last_peered":"2024-04-23T18:25:26.090540+0000","last_clean":"2024-04-23T18:25:26.090540+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:25:26.090540+0000","last_undegraded":"2024-04-23T18:25:26.090540+0000","last_fullsized":"2024-04-23T18:25:26.090540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910534+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:25:25.910534+0000","last_peered":"2024-04-23T18:25:25.910534+0000","last_clean":"2024-04-23T18:25:25.910534+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:25:25.910534+0000","last_undegraded":"2024-04-23T18:25:25.910534+0000","last_fullsized":"2024-04-23T18:25:25.910534+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916633+0000","last_change":"2024-04-23T18:25:25.916633+0000","last_active":"2024-04-23T18:25:25.916633+0000","last_peered":"2024-04-23T18:25:25.916633+0000","last_clean":"2024-04-23T18:25:25.916633+0000","last_became_active":"2024-04-23T18:25:25.916224+0000","last_became_peered":"2024-04-23T18:25:25.916224+0000","last_unstale":"2024-04-23T18:25:25.916633+0000","last_undegraded":"2024-04-23T18:25:25.916633+0000","last_fullsized":"2024-04-23T18:25:25.916633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:10.172049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915007+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:25:25.915007+0000","last_peered":"2024-04-23T18:25:25.915007+0000","last_clean":"2024-04-23T18:25:25.915007+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:25:25.915007+0000","last_undegraded":"2024-04-23T18:25:25.915007+0000","last_fullsized":"2024-04-23T18:25:25.915007+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090488+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:25:26.090488+0000","last_peered":"2024-04-23T18:25:26.090488+0000","last_clean":"2024-04-23T18:25:26.090488+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:25:26.090488+0000","last_undegraded":"2024-04-23T18:25:26.090488+0000","last_fullsized":"2024-04-23T18:25:26.090488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'32","reported_seq":199,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:44.477765+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:25:44.477765+0000","last_peered":"2024-04-23T18:25:44.477765+0000","last_clean":"2024-04-23T18:25:44.477765+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:25:44.477765+0000","last_undegraded":"2024-04-23T18:25:44.477765+0000","last_fullsized":"2024-04-23T18:25:44.477765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60023,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":148,"num_read_kb":132,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910391+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:25:25.910391+0000","last_peered":"2024-04-23T18:25:25.910391+0000","last_clean":"2024-04-23T18:25:25.910391+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:25:25.910391+0000","last_undegraded":"2024-04-23T18:25:25.910391+0000","last_fullsized":"2024-04-23T18:25:25.910391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916548+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:25:25.916548+0000","last_peered":"2024-04-23T18:25:25.916548+0000","last_clean":"2024-04-23T18:25:25.916548+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:25:25.916548+0000","last_undegraded":"2024-04-23T18:25:25.916548+0000","last_fullsized":"2024-04-23T18:25:25.916548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912594+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:25:25.912594+0000","last_peered":"2024-04-23T18:25:25.912594+0000","last_clean":"2024-04-23T18:25:25.912594+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:25:25.912594+0000","last_undegraded":"2024-04-23T18:25:25.912594+0000","last_fullsized":"2024-04-23T18:25:25.912594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914981+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:25:25.914981+0000","last_peered":"2024-04-23T18:25:25.914981+0000","last_clean":"2024-04-23T18:25:25.914981+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:25:25.914981+0000","last_undegraded":"2024-04-23T18:25:25.914981+0000","last_fullsized":"2024-04-23T18:25:25.914981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910519+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:25:25.910519+0000","last_peered":"2024-04-23T18:25:25.910519+0000","last_clean":"2024-04-23T18:25:25.910519+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:25:25.910519+0000","last_undegraded":"2024-04-23T18:25:25.910519+0000","last_fullsized":"2024-04-23T18:25:25.910519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.457585+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:25:41.457585+0000","last_peered":"2024-04-23T18:25:41.457585+0000","last_clean":"2024-04-23T18:25:41.457585+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:25:41.457585+0000","last_undegraded":"2024-04-23T18:25:41.457585+0000","last_fullsized":"2024-04-23T18:25:41.457585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910049+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:25:25.910049+0000","last_peered":"2024-04-23T18:25:25.910049+0000","last_clean":"2024-04-23T18:25:25.910049+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:25:25.910049+0000","last_undegraded":"2024-04-23T18:25:25.910049+0000","last_fullsized":"2024-04-23T18:25:25.910049+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909951+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:25:25.909951+0000","last_peered":"2024-04-23T18:25:25.909951+0000","last_clean":"2024-04-23T18:25:25.909951+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:25:25.909951+0000","last_undegraded":"2024-04-23T18:25:25.909951+0000","last_fullsized":"2024-04-23T18:25:25.909951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915697+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:25:25.915697+0000","last_peered":"2024-04-23T18:25:25.915697+0000","last_clean":"2024-04-23T18:25:25.915697+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:25:25.915697+0000","last_undegraded":"2024-04-23T18:25:25.915697+0000","last_fullsized":"2024-04-23T18:25:25.915697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910447+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:25:25.910447+0000","last_peered":"2024-04-23T18:25:25.910447+0000","last_clean":"2024-04-23T18:25:25.910447+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:25:25.910447+0000","last_undegraded":"2024-04-23T18:25:25.910447+0000","last_fullsized":"2024-04-23T18:25:25.910447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912181+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:25:25.912181+0000","last_peered":"2024-04-23T18:25:25.912181+0000","last_clean":"2024-04-23T18:25:25.912181+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:25:25.912181+0000","last_undegraded":"2024-04-23T18:25:25.912181+0000","last_fullsized":"2024-04-23T18:25:25.912181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913636+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:25:25.913636+0000","last_peered":"2024-04-23T18:25:25.913636+0000","last_clean":"2024-04-23T18:25:25.913636+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:25:25.913636+0000","last_undegraded":"2024-04-23T18:25:25.913636+0000","last_fullsized":"2024-04-23T18:25:25.913636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152950+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:25:26.152950+0000","last_peered":"2024-04-23T18:25:26.152950+0000","last_clean":"2024-04-23T18:25:26.152950+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:25:26.152950+0000","last_undegraded":"2024-04-23T18:25:26.152950+0000","last_fullsized":"2024-04-23T18:25:26.152950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'5","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:40.457649+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:25:40.457649+0000","last_peered":"2024-04-23T18:25:40.457649+0000","last_clean":"2024-04-23T18:25:40.457649+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:25:40.457649+0000","last_undegraded":"2024-04-23T18:25:40.457649+0000","last_fullsized":"2024-04-23T18:25:40.457649+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.152792+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:25:26.152792+0000","last_peered":"2024-04-23T18:25:26.152792+0000","last_clean":"2024-04-23T18:25:26.152792+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:25:26.152792+0000","last_undegraded":"2024-04-23T18:25:26.152792+0000","last_fullsized":"2024-04-23T18:25:26.152792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.914583+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:25:25.914583+0000","last_peered":"2024-04-23T18:25:25.914583+0000","last_clean":"2024-04-23T18:25:25.914583+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:25:25.914583+0000","last_undegraded":"2024-04-23T18:25:25.914583+0000","last_fullsized":"2024-04-23T18:25:25.914583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'5","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.456892+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:25:41.456892+0000","last_peered":"2024-04-23T18:25:41.456892+0000","last_clean":"2024-04-23T18:25:41.456892+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:25:41.456892+0000","last_undegraded":"2024-04-23T18:25:41.456892+0000","last_fullsized":"2024-04-23T18:25:41.456892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.092051+0000","last_change":"2024-04-23T18:25:26.091646+0000","last_active":"2024-04-23T18:25:26.092051+0000","last_peered":"2024-04-23T18:25:26.092051+0000","last_clean":"2024-04-23T18:25:26.092051+0000","last_became_active":"2024-04-23T18:25:26.091303+0000","last_became_peered":"2024-04-23T18:25:26.091303+0000","last_unstale":"2024-04-23T18:25:26.092051+0000","last_undegraded":"2024-04-23T18:25:26.092051+0000","last_fullsized":"2024-04-23T18:25:26.092051+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:27.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101116+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:25:26.101116+0000","last_peered":"2024-04-23T18:25:26.101116+0000","last_clean":"2024-04-23T18:25:26.101116+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:25:26.101116+0000","last_undegraded":"2024-04-23T18:25:26.101116+0000","last_fullsized":"2024-04-23T18:25:26.101116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910096+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:25:25.910096+0000","last_peered":"2024-04-23T18:25:25.910096+0000","last_clean":"2024-04-23T18:25:25.910096+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:25:25.910096+0000","last_undegraded":"2024-04-23T18:25:25.910096+0000","last_fullsized":"2024-04-23T18:25:25.910096+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912407+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:25:25.912407+0000","last_peered":"2024-04-23T18:25:25.912407+0000","last_clean":"2024-04-23T18:25:25.912407+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:25:25.912407+0000","last_undegraded":"2024-04-23T18:25:25.912407+0000","last_fullsized":"2024-04-23T18:25:25.912407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910215+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:25:25.910215+0000","last_peered":"2024-04-23T18:25:25.910215+0000","last_clean":"2024-04-23T18:25:25.910215+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:25:25.910215+0000","last_undegraded":"2024-04-23T18:25:25.910215+0000","last_fullsized":"2024-04-23T18:25:25.910215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910523+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:25:25.910523+0000","last_peered":"2024-04-23T18:25:25.910523+0000","last_clean":"2024-04-23T18:25:25.910523+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:25:25.910523+0000","last_undegraded":"2024-04-23T18:25:25.910523+0000","last_fullsized":"2024-04-23T18:25:25.910523+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912868+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:25:25.912868+0000","last_peered":"2024-04-23T18:25:25.912868+0000","last_clean":"2024-04-23T18:25:25.912868+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:25:25.912868+0000","last_undegraded":"2024-04-23T18:25:25.912868+0000","last_fullsized":"2024-04-23T18:25:25.912868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913016+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:25:25.913016+0000","last_peered":"2024-04-23T18:25:25.913016+0000","last_clean":"2024-04-23T18:25:25.913016+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:25:25.913016+0000","last_undegraded":"2024-04-23T18:25:25.913016+0000","last_fullsized":"2024-04-23T18:25:25.913016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916403+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:25:25.916403+0000","last_peered":"2024-04-23T18:25:25.916403+0000","last_clean":"2024-04-23T18:25:25.916403+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:25:25.916403+0000","last_undegraded":"2024-04-23T18:25:25.916403+0000","last_fullsized":"2024-04-23T18:25:25.916403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.101304+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:25:26.101304+0000","last_peered":"2024-04-23T18:25:26.101304+0000","last_clean":"2024-04-23T18:25:26.101304+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:25:26.101304+0000","last_undegraded":"2024-04-23T18:25:26.101304+0000","last_fullsized":"2024-04-23T18:25:26.101304+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909646+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:25:25.909646+0000","last_peered":"2024-04-23T18:25:25.909646+0000","last_clean":"2024-04-23T18:25:25.909646+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:25:25.909646+0000","last_undegraded":"2024-04-23T18:25:25.909646+0000","last_fullsized":"2024-04-23T18:25:25.909646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090267+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:25:26.090267+0000","last_peered":"2024-04-23T18:25:26.090267+0000","last_clean":"2024-04-23T18:25:26.090267+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:25:26.090267+0000","last_undegraded":"2024-04-23T18:25:26.090267+0000","last_fullsized":"2024-04-23T18:25:26.090267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909943+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:25:25.909943+0000","last_peered":"2024-04-23T18:25:25.909943+0000","last_clean":"2024-04-23T18:25:25.909943+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:25:25.909943+0000","last_undegraded":"2024-04-23T18:25:25.909943+0000","last_fullsized":"2024-04-23T18:25:25.909943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915935+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:25:25.915935+0000","last_peered":"2024-04-23T18:25:25.915935+0000","last_clean":"2024-04-23T18:25:25.915935+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:25:25.915935+0000","last_undegraded":"2024-04-23T18:25:25.915935+0000","last_fullsized":"2024-04-23T18:25:25.915935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912218+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:25:25.912218+0000","last_peered":"2024-04-23T18:25:25.912218+0000","last_clean":"2024-04-23T18:25:25.912218+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:25:25.912218+0000","last_undegraded":"2024-04-23T18:25:25.912218+0000","last_fullsized":"2024-04-23T18:25:25.912218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'3","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.935896+0000","last_change":"2024-04-23T18:25:25.935443+0000","last_active":"2024-04-23T18:25:25.935896+0000","last_peered":"2024-04-23T18:25:25.935896+0000","last_clean":"2024-04-23T18:25:25.935896+0000","last_became_active":"2024-04-23T18:25:25.919406+0000","last_became_peered":"2024-04-23T18:25:25.919406+0000","last_unstale":"2024-04-23T18:25:25.935896+0000","last_undegraded":"2024-04-23T18:25:25.935896+0000","last_fullsized":"2024-04-23T18:25:25.935896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:10:03.007617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915576+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:25:25.915576+0000","last_peered":"2024-04-23T18:25:25.915576+0000","last_clean":"2024-04-23T18:25:25.915576+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:25:25.915576+0000","last_undegraded":"2024-04-23T18:25:25.915576+0000","last_fullsized":"2024-04-23T18:25:25.915576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910051+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:25:25.910051+0000","last_peered":"2024-04-23T18:25:25.910051+0000","last_clean":"2024-04-23T18:25:25.910051+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:25:25.910051+0000","last_undegraded":"2024-04-23T18:25:25.910051+0000","last_fullsized":"2024-04-23T18:25:25.910051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910474+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:25:25.910474+0000","last_peered":"2024-04-23T18:25:25.910474+0000","last_clean":"2024-04-23T18:25:25.910474+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:25:25.910474+0000","last_undegraded":"2024-04-23T18:25:25.910474+0000","last_fullsized":"2024-04-23T18:25:25.910474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'5","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:41.467618+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:25:41.467618+0000","last_peered":"2024-04-23T18:25:41.467618+0000","last_clean":"2024-04-23T18:25:41.467618+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:25:41.467618+0000","last_undegraded":"2024-04-23T18:25:41.467618+0000","last_fullsized":"2024-04-23T18:25:41.467618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910665+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:25:25.910665+0000","last_peered":"2024-04-23T18:25:25.910665+0000","last_clean":"2024-04-23T18:25:25.910665+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:25:25.910665+0000","last_undegraded":"2024-04-23T18:25:25.910665+0000","last_fullsized":"2024-04-23T18:25:25.910665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915000+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:25:25.915000+0000","last_peered":"2024-04-23T18:25:25.915000+0000","last_clean":"2024-04-23T18:25:25.915000+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:25:25.915000+0000","last_undegraded":"2024-04-23T18:25:25.915000+0000","last_fullsized":"2024-04-23T18:25:25.915000+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910446+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:25:25.910446+0000","last_peered":"2024-04-23T18:25:25.910446+0000","last_clean":"2024-04-23T18:25:25.910446+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:25:25.910446+0000","last_undegraded":"2024-04-23T18:25:25.910446+0000","last_fullsized":"2024-04-23T18:25:25.910446+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.106394+0000","last_change":"2024-04-23T18:25:26.106394+0000","last_active":"2024-04-23T18:25:26.106394+0000","last_peered":"2024-04-23T18:25:26.106394+0000","last_clean":"2024-04-23T18:25:26.106394+0000","last_became_active":"2024-04-23T18:25:26.102178+0000","last_became_peered":"2024-04-23T18:25:26.102178+0000","last_unstale":"2024-04-23T18:25:26.106394+0000","last_undegraded":"2024-04-23T18:25:26.106394+0000","last_fullsized":"2024-04-23T18:25:26.106394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:20.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.910717+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:25:25.910717+0000","last_peered":"2024-04-23T18:25:25.910717+0000","last_clean":"2024-04-23T18:25:25.910717+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:25:25.910717+0000","last_undegraded":"2024-04-23T18:25:25.910717+0000","last_fullsized":"2024-04-23T18:25:25.910717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912294+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:25:25.912294+0000","last_peered":"2024-04-23T18:25:25.912294+0000","last_clean":"2024-04-23T18:25:25.912294+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:25:25.912294+0000","last_undegraded":"2024-04-23T18:25:25.912294+0000","last_fullsized":"2024-04-23T18:25:25.912294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916821+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:25:25.916821+0000","last_peered":"2024-04-23T18:25:25.916821+0000","last_clean":"2024-04-23T18:25:25.916821+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:25:25.916821+0000","last_undegraded":"2024-04-23T18:25:25.916821+0000","last_fullsized":"2024-04-23T18:25:25.916821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.918437+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:25:25.918437+0000","last_peered":"2024-04-23T18:25:25.918437+0000","last_clean":"2024-04-23T18:25:25.918437+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:25:25.918437+0000","last_undegraded":"2024-04-23T18:25:25.918437+0000","last_fullsized":"2024-04-23T18:25:25.918437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.909391+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:25:25.909391+0000","last_peered":"2024-04-23T18:25:25.909391+0000","last_clean":"2024-04-23T18:25:25.909391+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:25:25.909391+0000","last_undegraded":"2024-04-23T18:25:25.909391+0000","last_fullsized":"2024-04-23T18:25:25.909391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912810+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:25:25.912810+0000","last_peered":"2024-04-23T18:25:25.912810+0000","last_clean":"2024-04-23T18:25:25.912810+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:25:25.912810+0000","last_undegraded":"2024-04-23T18:25:25.912810+0000","last_fullsized":"2024-04-23T18:25:25.912810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915473+0000","last_change":"2024-04-23T18:25:25.915473+0000","last_active":"2024-04-23T18:25:25.915473+0000","last_peered":"2024-04-23T18:25:25.915473+0000","last_clean":"2024-04-23T18:25:25.915473+0000","last_became_active":"2024-04-23T18:25:25.914874+0000","last_became_peered":"2024-04-23T18:25:25.914874+0000","last_unstale":"2024-04-23T18:25:25.915473+0000","last_undegraded":"2024-04-23T18:25:25.915473+0000","last_fullsized":"2024-04-23T18:25:25.915473+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:14.348994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.918150+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:25:25.918150+0000","last_peered":"2024-04-23T18:25:25.918150+0000","last_clean":"2024-04-23T18:25:25.918150+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:25:25.918150+0000","last_undegraded":"2024-04-23T18:25:25.918150+0000","last_fullsized":"2024-04-23T18:25:25.918150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.916300+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:25:25.916300+0000","last_peered":"2024-04-23T18:25:25.916300+0000","last_clean":"2024-04-23T18:25:25.916300+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:25:25.916300+0000","last_undegraded":"2024-04-23T18:25:25.916300+0000","last_fullsized":"2024-04-23T18:25:25.916300+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.913657+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:25:25.913657+0000","last_peered":"2024-04-23T18:25:25.913657+0000","last_clean":"2024-04-23T18:25:25.913657+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:25:25.913657+0000","last_undegraded":"2024-04-23T18:25:25.913657+0000","last_fullsized":"2024-04-23T18:25:25.913657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.915800+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:25:25.915800+0000","last_peered":"2024-04-23T18:25:25.915800+0000","last_clean":"2024-04-23T18:25:25.915800+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:25:25.915800+0000","last_undegraded":"2024-04-23T18:25:25.915800+0000","last_fullsized":"2024-04-23T18:25:25.915800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:26.090743+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:25:26.090743+0000","last_peered":"2024-04-23T18:25:26.090743+0000","last_clean":"2024-04-23T18:25:26.090743+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:25:26.090743+0000","last_undegraded":"2024-04-23T18:25:26.090743+0000","last_fullsized":"2024-04-23T18:25:26.090743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.912808+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:25:25.912808+0000","last_peered":"2024-04-23T18:25:25.912808+0000","last_clean":"2024-04-23T18:25:25.912808+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:25:25.912808+0000","last_undegraded":"2024-04-23T18:25:25.912808+0000","last_fullsized":"2024-04-23T18:25:25.912808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-04-23T18:25:25.917314+0000","last_change":"2024-04-23T18:25:25.917314+0000","last_active":"2024-04-23T18:25:25.917314+0000","last_peered":"2024-04-23T18:25:25.917314+0000","last_clean":"2024-04-23T18:25:25.917314+0000","last_became_active":"2024-04-23T18:25:25.916841+0000","last_became_peered":"2024-04-23T18:25:25.916841+0000","last_unstale":"2024-04-23T18:25:25.917314+0000","last_undegraded":"2024-04-23T18:25:25.917314+0000","last_fullsized":"2024-04-23T18:25:25.917314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:03.413973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280434,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67227,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":731,"num_read_kb":441,"num_write":239,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":456894,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645347,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1108337,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":974848,"data_stored":456824,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":457202,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":1050111,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":456541,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1019904,"data_stored":517477,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":457880,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":518088,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":456366,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456793,"data_compressed":5821,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":1049398,"data_compressed":41491,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34028,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:25:48.567 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-23T18:25:48.567 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-23T18:25:48.567 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-23T18:25:48.568 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-23T18:25:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:49.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:48 smithi033 ceph-mon[29274]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:49.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:48 smithi106 ceph-mon[32716]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:49.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:48 smithi122 ceph-mon[32546]: from='client.14967 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:25:50.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:49 smithi033 ceph-mon[29274]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:50.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:49 smithi106 ceph-mon[32716]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:50.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:49 smithi122 ceph-mon[32546]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- 172.21.15.33:0/2495777983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06a750 msgr2=0x7fbccc06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 --2- 172.21.15.33:0/2495777983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06a750 0x7fbccc06abb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009990 tx=0x7fbcbc009ca0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- 172.21.15.33:0/2495777983 shutdown_connections 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 --2- 172.21.15.33:0/2495777983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06b0f0 0x7fbccc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 --2- 172.21.15.33:0/2495777983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 0x7fbccc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 --2- 172.21.15.33:0/2495777983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06a750 0x7fbccc06abb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- 172.21.15.33:0/2495777983 >> 172.21.15.33:0/2495777983 conn(0x7fbccc0788f0 msgr2=0x7fbccc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- 172.21.15.33:0/2495777983 shutdown_connections 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- 172.21.15.33:0/2495777983 wait complete. 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 Processor -- start 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.489+0000 7fbcd4439700 1 -- start start 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06a750 0x7fbccc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:50.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 0x7fbccc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112bd0 con 0x7fbccc06b0f0 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112a50 con 0x7fbccc108a30 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd4439700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc112d50 con 0x7fbccc06a750 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32832/0 (socket says 172.21.15.33:32832) 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 -- 172.21.15.33:0/3613673619 learned_addr learned my addr 172.21.15.33:0/3613673619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd29d6700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 0x7fbccc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd21d5700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06a750 0x7fbccc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 -- 172.21.15.33:0/3613673619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06a750 msgr2=0x7fbccc199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06a750 0x7fbccc199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 -- 172.21.15.33:0/3613673619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 msgr2=0x7fbccc19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 0x7fbccc19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 -- 172.21.15.33:0/3613673619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc19e110 con 0x7fbccc06b0f0 2024-04-23T18:25:50.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcd19d4700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc000c00 tx=0x7fbcbc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.490+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc027070 con 0x7fbccc06b0f0 2024-04-23T18:25:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.491+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcbc003f20 con 0x7fbccc06b0f0 2024-04-23T18:25:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.491+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc19e3a0 con 0x7fbccc06b0f0 2024-04-23T18:25:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.491+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc022ca0 con 0x7fbccc06b0f0 2024-04-23T18:25:50.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.491+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1a5c80 con 0x7fbccc06b0f0 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.492+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc069730 con 0x7fbccc06b0f0 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.493+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbcbc02a070 con 0x7fbccc06b0f0 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.493+0000 7fbcc37fe700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 0x7fbcb80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.494+0000 7fbcd21d5700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 0x7fbcb80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.494+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fbcbc0d1750 con 0x7fbccc06b0f0 2024-04-23T18:25:50.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.494+0000 7fbcd21d5700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 0x7fbcb80a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007b60 tx=0x7fbcc80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.497+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcbc031070 con 0x7fbccc06b0f0 2024-04-23T18:25:50.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.755+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fbccc02df80 con 0x7fbccc06b0f0 2024-04-23T18:25:50.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.755+0000 7fbcc37fe700 1 -- 172.21.15.33:0/3613673619 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fbcbc013070 con 0x7fbccc06b0f0 2024-04-23T18:25:50.748 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:50.748 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-23T18:25:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 msgr2=0x7fbcb80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 0x7fbcb80a6210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8007b60 tx=0x7fbcc80073d0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 msgr2=0x7fbccc199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc000c00 tx=0x7fbcbc0047b0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 shutdown_connections 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fbcb80a3d50 0x7fbcb80a6210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbccc108a30 0x7fbccc19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc06a750 0x7fbccc199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 --2- 172.21.15.33:0/3613673619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbccc06b0f0 0x7fbccc199640 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 >> 172.21.15.33:0/3613673619 conn(0x7fbccc0788f0 msgr2=0x7fbccc0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 shutdown_connections 2024-04-23T18:25:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:50.757+0000 7fbcd4439700 1 -- 172.21.15.33:0/3613673619 wait complete. 2024-04-23T18:25:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:50 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3613673619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:51.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:50 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3613673619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:51.417 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-23T18:25:51.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:51.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:50 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3613673619' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- 172.21.15.33:0/499491362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d81031b0 msgr2=0x7f78d8103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 --2- 172.21.15.33:0/499491362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d81031b0 0x7f78d8103630 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009900 tx=0x7f78c002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- 172.21.15.33:0/499491362 shutdown_connections 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 --2- 172.21.15.33:0/499491362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8103b70 0x7f78d810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 --2- 172.21.15.33:0/499491362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d81031b0 0x7f78d8103630 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 --2- 172.21.15.33:0/499491362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d8101fc0 0x7f78d81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- 172.21.15.33:0/499491362 >> 172.21.15.33:0/499491362 conn(0x7f78d80fd550 msgr2=0x7f78d80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- 172.21.15.33:0/499491362 shutdown_connections 2024-04-23T18:25:51.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- 172.21.15.33:0/499491362 wait complete. 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 Processor -- start 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.534+0000 7f78dc88a640 1 -- start start 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8101fc0 0x7f78d8075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d81031b0 0x7f78d8074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810db00 con 0x7f78d8103b70 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810d980 con 0x7f78d81031b0 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f78d810dc80 con 0x7f78d8101fc0 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 -- 172.21.15.33:0/1636469856 learned_addr learned my addr 172.21.15.33:0/1636469856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d659c640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8101fc0 0x7f78d8075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78cffff640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d81031b0 0x7f78d8074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 -- 172.21.15.33:0/1636469856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8101fc0 msgr2=0x7f78d8075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8101fc0 0x7f78d8075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 -- 172.21.15.33:0/1636469856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d81031b0 msgr2=0x7f78d8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d81031b0 0x7f78d8074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 -- 172.21.15.33:0/1636469856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d8075110 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78d6d9d640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78d000cce0 tx=0x7f78d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d000f450 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d81a2290 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d000faa0 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.535+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0018b70 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.536+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d81a2790 con 0x7f78d8103b70 2024-04-23T18:25:51.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.537+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d81092e0 con 0x7f78d8103b70 2024-04-23T18:25:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.538+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f78d0018d10 con 0x7f78d8103b70 2024-04-23T18:25:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.538+0000 7f78cf7fe640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 0x7f78c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.538+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f78d00d0b10 con 0x7f78d8103b70 2024-04-23T18:25:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.538+0000 7f78d659c640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 0x7f78c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.539+0000 7f78d659c640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 0x7f78c40a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f78d8103010 tx=0x7f78c800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.542+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78d0014030 con 0x7f78d8103b70 2024-04-23T18:25:51.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.730+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78d8062b50 con 0x7f78d8103b70 2024-04-23T18:25:51.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.731+0000 7f78cf7fe640 1 -- 172.21.15.33:0/1636469856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f78d009d420 con 0x7f78d8103b70 2024-04-23T18:25:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:51.724 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:51.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 msgr2=0x7f78c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 0x7f78c40a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f78d8103010 tx=0x7f78c800b390 comp rx=0 tx=0).stop 2024-04-23T18:25:51.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 msgr2=0x7f78d8074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f78d000cce0 tx=0x7f78d000a620 comp rx=0 tx=0).stop 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 shutdown_connections 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f78c40a3c40 0x7f78c40a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78d8103b70 0x7f78d8074880 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78d81031b0 0x7f78d8074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 --2- 172.21.15.33:0/1636469856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78d8101fc0 0x7f78d8075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 >> 172.21.15.33:0/1636469856 conn(0x7f78d80fd550 msgr2=0x7f78d81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 shutdown_connections 2024-04-23T18:25:51.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.733+0000 7f78dc88a640 1 -- 172.21.15.33:0/1636469856 wait complete. 2024-04-23T18:25:51.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:51.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- 172.21.15.33:0/3775324534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 msgr2=0x7f0410101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 --2- 172.21.15.33:0/3775324534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f0410101280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f040c009900 tx=0x7f040c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- 172.21.15.33:0/3775324534 shutdown_connections 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 --2- 172.21.15.33:0/3775324534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102a00 0x7f0410109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 --2- 172.21.15.33:0/3775324534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102040 0x7f04101024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 --2- 172.21.15.33:0/3775324534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f0410101280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- 172.21.15.33:0/3775324534 >> 172.21.15.33:0/3775324534 conn(0x7f04100fc420 msgr2=0x7f04100fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- 172.21.15.33:0/3775324534 shutdown_connections 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- 172.21.15.33:0/3775324534 wait complete. 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 Processor -- start 2024-04-23T18:25:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.850+0000 7f0417968640 1 -- start start 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102040 0x7f04101986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102a00 0x7f041019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f041010db50 con 0x7f0410100e60 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f041010d9d0 con 0x7f0410102040 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f041010dcd0 con 0x7f0410102a00 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32864/0 (socket says 172.21.15.33:32864) 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 -- 172.21.15.33:0/2942593224 learned_addr learned my addr 172.21.15.33:0/2942593224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0414f03640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102040 0x7f04101986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415f05640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102a00 0x7f041019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 -- 172.21.15.33:0/2942593224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102a00 msgr2=0x7f041019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102a00 0x7f041019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 -- 172.21.15.33:0/2942593224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102040 msgr2=0x7f04101986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102040 0x7f04101986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 -- 172.21.15.33:0/2942593224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041019fdc0 con 0x7f0410100e60 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0415704640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f040c031500 tx=0x7f040c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c03f070 con 0x7f0410100e60 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040c031c50 con 0x7f0410100e60 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.851+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04101a0050 con 0x7f0410100e60 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.852+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c035030 con 0x7f0410100e60 2024-04-23T18:25:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.852+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04101a0540 con 0x7f0410100e60 2024-04-23T18:25:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.853+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0410108250 con 0x7f0410100e60 2024-04-23T18:25:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.854+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f040c042070 con 0x7f0410100e60 2024-04-23T18:25:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.855+0000 7f04067fc640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 0x7f03fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.855+0000 7f0414f03640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 0x7f03fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.855+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f040c0f0d40 con 0x7f0410100e60 2024-04-23T18:25:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.855+0000 7f0414f03640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 0x7f03fc0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04101996b0 tx=0x7f040000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:51.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:51.859+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f040c0bd5c0 con 0x7f0410100e60 2024-04-23T18:25:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.045+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0410062b50 con 0x7f0410100e60 2024-04-23T18:25:52.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.045+0000 7f04067fc640 1 -- 172.21.15.33:0/2942593224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f040c0bcec0 con 0x7f0410100e60 2024-04-23T18:25:52.038 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:25:52.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 msgr2=0x7f03fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 0x7f03fc0a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04101996b0 tx=0x7f040000a2d0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 msgr2=0x7f04101981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f040c031500 tx=0x7f040c003c10 comp rx=0 tx=0).stop 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 shutdown_connections 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f03fc0a3c40 0x7f03fc0a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0410102a00 0x7f041019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0410102040 0x7f04101986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 --2- 172.21.15.33:0/2942593224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0410100e60 0x7f04101981a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.047+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 >> 172.21.15.33:0/2942593224 conn(0x7f04100fc420 msgr2=0x7f0410105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.048+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 shutdown_connections 2024-04-23T18:25:52.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.048+0000 7f0417968640 1 -- 172.21.15.33:0/2942593224 wait complete. 2024-04-23T18:25:52.050 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:51 smithi033 ceph-mon[29274]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:52.051 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:51 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1636469856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:52.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 -- 172.21.15.33:0/1984565790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094109e60 msgr2=0x7f209410c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 --2- 172.21.15.33:0/1984565790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094109e60 0x7f209410c2f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f208c009ca0 tx=0x7f208c02a950 comp rx=0 tx=0).stop 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 -- 172.21.15.33:0/1984565790 shutdown_connections 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 --2- 172.21.15.33:0/1984565790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094109e60 0x7f209410c2f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 --2- 172.21.15.33:0/1984565790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094074930 0x7f2094074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 --2- 172.21.15.33:0/1984565790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2094075f10 0x7f2094074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 -- 172.21.15.33:0/1984565790 >> 172.21.15.33:0/1984565790 conn(0x7f20940fd540 msgr2=0x7f20940ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 -- 172.21.15.33:0/1984565790 shutdown_connections 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.163+0000 7f209bce1640 1 -- 172.21.15.33:0/1984565790 wait complete. 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 Processor -- start 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 -- start start 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 0x7f209419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2094109e60 0x7f20941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f209410fd40 con 0x7f2094074930 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f209410fbc0 con 0x7f2094109e60 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209bce1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f209410fec0 con 0x7f2094075f10 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f2099a7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f2099a7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32870/0 (socket says 172.21.15.33:32870) 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f2099a7d640 1 -- 172.21.15.33:0/1415148789 learned_addr learned my addr 172.21.15.33:0/1415148789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.164+0000 7f209927c640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 0x7f209419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 -- 172.21.15.33:0/1415148789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 msgr2=0x7f209419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 0x7f209419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 -- 172.21.15.33:0/1415148789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2094109e60 msgr2=0x7f20941a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2094109e60 0x7f20941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 -- 172.21.15.33:0/1415148789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20941a1f50 con 0x7f2094074930 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f209927c640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 0x7f209419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f2099a7d640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f209000c920 tx=0x7f209000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090007c70 con 0x7f2094074930 2024-04-23T18:25:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090007e10 con 0x7f2094074930 2024-04-23T18:25:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20941a2240 con 0x7f2094074930 2024-04-23T18:25:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090018820 con 0x7f2094074930 2024-04-23T18:25:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.165+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941a2810 con 0x7f2094074930 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.166+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2094108c20 con 0x7f2094074930 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.167+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f209000f450 con 0x7f2094074930 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.168+0000 7f208affd640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 0x7f20800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.168+0000 7f209927c640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 0x7f20800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.168+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f20900d09f0 con 0x7f2094074930 2024-04-23T18:25:52.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.169+0000 7f209927c640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 0x7f20800a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2084006fd0 tx=0x7f2084008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.172+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2090014030 con 0x7f2094074930 2024-04-23T18:25:52.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.357+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2094062bf0 con 0x7f2094074930 2024-04-23T18:25:52.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.358+0000 7f208affd640 1 -- 172.21.15.33:0/1415148789 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f209009d300 con 0x7f2094074930 2024-04-23T18:25:52.351 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:25:52.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 msgr2=0x7f20800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 0x7f20800a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2084006fd0 tx=0x7f2084008040 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 msgr2=0x7f209419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f209000c920 tx=0x7f209000cde0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 shutdown_connections 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f20800a3c40 0x7f20800a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2094109e60 0x7f20941a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2094075f10 0x7f209419a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 --2- 172.21.15.33:0/1415148789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2094074930 0x7f209419a330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 >> 172.21.15.33:0/1415148789 conn(0x7f20940fd540 msgr2=0x7f2094105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 shutdown_connections 2024-04-23T18:25:52.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.360+0000 7f209bce1640 1 -- 172.21.15.33:0/1415148789 wait complete. 2024-04-23T18:25:52.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:52.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:51 smithi106 ceph-mon[32716]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:52.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:51 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1636469856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:52.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:51 smithi122 ceph-mon[32546]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:25:52.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:51 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1636469856' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- 172.21.15.33:0/1159551147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 msgr2=0x7ff1d410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/1159551147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d410a0c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009900 tx=0x7ff1cc02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- 172.21.15.33:0/1159551147 shutdown_connections 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/1159551147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d410a0c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/1159551147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 0x7ff1d4107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/1159551147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 0x7ff1d4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- 172.21.15.33:0/1159551147 >> 172.21.15.33:0/1159551147 conn(0x7ff1d40fc400 msgr2=0x7ff1d40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- 172.21.15.33:0/1159551147 shutdown_connections 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- 172.21.15.33:0/1159551147 wait complete. 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 Processor -- start 2024-04-23T18:25:52.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.476+0000 7ff1dbb95640 1 -- start start 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 0x7ff1d41981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 0x7ff1d4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410dac0 con 0x7ff1d4107c70 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410d940 con 0x7ff1d4105340 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d410dc40 con 0x7ff1d4102a10 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32900/0 (socket says 172.21.15.33:32900) 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 -- 172.21.15.33:0/3118319842 learned_addr learned my addr 172.21.15.33:0/3118319842 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1d9130640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 0x7ff1d4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1d9931640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 0x7ff1d41981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 -- 172.21.15.33:0/3118319842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 msgr2=0x7ff1d41981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 0x7ff1d41981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 -- 172.21.15.33:0/3118319842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 msgr2=0x7ff1d4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 0x7ff1d4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 -- 172.21.15.33:0/3118319842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d419fde0 con 0x7ff1d4107c70 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1da132640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc031500 tx=0x7ff1cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc03f030 con 0x7ff1d4107c70 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1cc003cb0 con 0x7ff1d4107c70 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d41a0070 con 0x7ff1d4107c70 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.477+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc035070 con 0x7ff1d4107c70 2024-04-23T18:25:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.478+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d41a0560 con 0x7ff1d4107c70 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.478+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d40fdef0 con 0x7ff1d4107c70 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.480+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff1cc042030 con 0x7ff1d4107c70 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.480+0000 7ff1caffd640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 0x7ff1c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.480+0000 7ff1d9931640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 0x7ff1c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.480+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7ff1cc0f1140 con 0x7ff1d4107c70 2024-04-23T18:25:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.481+0000 7ff1d9931640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 0x7ff1c00a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005fd0 tx=0x7ff1d0009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.484+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1cc0bd9c0 con 0x7ff1d4107c70 2024-04-23T18:25:52.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.669+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff1d402d5c0 con 0x7ff1d4107c70 2024-04-23T18:25:52.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.670+0000 7ff1caffd640 1 -- 172.21.15.33:0/3118319842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7ff1cc0bd2c0 con 0x7ff1d4107c70 2024-04-23T18:25:52.663 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 msgr2=0x7ff1c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 0x7ff1c00a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0005fd0 tx=0x7ff1d0009520 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 msgr2=0x7ff1d419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc031500 tx=0x7ff1cc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 shutdown_connections 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff1c00a3c40 0x7ff1c00a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1d4107c70 0x7ff1d419f770 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1d4105340 0x7ff1d4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 --2- 172.21.15.33:0/3118319842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1d4102a10 0x7ff1d41981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 >> 172.21.15.33:0/3118319842 conn(0x7ff1d40fc400 msgr2=0x7ff1d41033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 shutdown_connections 2024-04-23T18:25:52.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.672+0000 7ff1dbb95640 1 -- 172.21.15.33:0/3118319842 wait complete. 2024-04-23T18:25:52.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 -- 172.21.15.33:0/2341390554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 msgr2=0x7f7924101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 --2- 172.21.15.33:0/2341390554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924101330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f791c009900 tx=0x7f791c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 -- 172.21.15.33:0/2341390554 shutdown_connections 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 --2- 172.21.15.33:0/2341390554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102a20 0x7f7924109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 --2- 172.21.15.33:0/2341390554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102060 0x7f79241024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 --2- 172.21.15.33:0/2341390554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924101330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 -- 172.21.15.33:0/2341390554 >> 172.21.15.33:0/2341390554 conn(0x7f79240fc4a0 msgr2=0x7f79240fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 -- 172.21.15.33:0/2341390554 shutdown_connections 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.784+0000 7f792b231640 1 -- 172.21.15.33:0/2341390554 wait complete. 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 Processor -- start 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 -- start start 2024-04-23T18:25:52.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 0x7f7924198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 0x7f792419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f792410c9f0 con 0x7f7924100f10 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f792410c870 con 0x7f7924102060 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f792b231640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f792410cb70 con 0x7f7924102a20 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32916/0 (socket says 172.21.15.33:32916) 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 -- 172.21.15.33:0/1825129592 learned_addr learned my addr 172.21.15.33:0/1825129592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f79297ce640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 0x7f792419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7923fff640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 0x7f7924198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 -- 172.21.15.33:0/1825129592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 msgr2=0x7f792419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 0x7f792419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 -- 172.21.15.33:0/1825129592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 msgr2=0x7f7924198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:52.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 0x7f7924198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7928fcd640 1 -- 172.21.15.33:0/1825129592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792419fe60 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f7923fff640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 0x7f7924198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.785+0000 7f79297ce640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 0x7f792419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f7928fcd640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f791c005940 tx=0x7f791c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c03f030 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791c003cb0 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79241a00f0 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c035070 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.786+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241a05e0 con 0x7f7924100f10 2024-04-23T18:25:52.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.787+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79241a0840 con 0x7f7924100f10 2024-04-23T18:25:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.788+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f791c042030 con 0x7f7924100f10 2024-04-23T18:25:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.789+0000 7f7921ffb640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 0x7f79100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.789+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f791c0f1210 con 0x7f7924100f10 2024-04-23T18:25:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.789+0000 7f7923fff640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 0x7f79100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:52.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.789+0000 7f7923fff640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 0x7f79100a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7924199750 tx=0x7f7914009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:52.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:52.792+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f791c0bda90 con 0x7f7924100f10 2024-04-23T18:25:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.021+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79241a0a10 con 0x7f7924100f10 2024-04-23T18:25:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.022+0000 7f7921ffb640 1 -- 172.21.15.33:0/1825129592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7f791c003e60 con 0x7f7924100f10 2024-04-23T18:25:53.015 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:53.015 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:25:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 msgr2=0x7f79100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 0x7f79100a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7924199750 tx=0x7f7914009400 comp rx=0 tx=0).stop 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 msgr2=0x7f7924198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f791c005940 tx=0x7f791c004430 comp rx=0 tx=0).stop 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 shutdown_connections 2024-04-23T18:25:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f79100a3c40 0x7f79100a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7924102a20 0x7f792419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7924102060 0x7f7924198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 --2- 172.21.15.33:0/1825129592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7924100f10 0x7f7924198240 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 >> 172.21.15.33:0/1825129592 conn(0x7f79240fc4a0 msgr2=0x7f79240fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 shutdown_connections 2024-04-23T18:25:53.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.024+0000 7f792b231640 1 -- 172.21.15.33:0/1825129592 wait complete. 2024-04-23T18:25:53.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:53.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 -- 172.21.15.33:0/4147024516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c101fc0 msgr2=0x7f4d1c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 --2- 172.21.15.33:0/4147024516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c1023e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009900 tx=0x7f4d0c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:53.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 -- 172.21.15.33:0/4147024516 shutdown_connections 2024-04-23T18:25:53.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 --2- 172.21.15.33:0/4147024516 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c10a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 --2- 172.21.15.33:0/4147024516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 --2- 172.21.15.33:0/4147024516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c1023e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 -- 172.21.15.33:0/4147024516 >> 172.21.15.33:0/4147024516 conn(0x7f4d1c0fd550 msgr2=0x7f4d1c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 -- 172.21.15.33:0/4147024516 shutdown_connections 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.138+0000 7f4d2155d640 1 -- 172.21.15.33:0/4147024516 wait complete. 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 Processor -- start 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 -- start start 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10db00 con 0x7f4d1c1031b0 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10d980 con 0x7f4d1c103b70 2024-04-23T18:25:53.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d2155d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c10dc80 con 0x7f4d1c101fc0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32934/0 (socket says 172.21.15.33:32934) 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 -- 172.21.15.33:0/1886241403 learned_addr learned my addr 172.21.15.33:0/1886241403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d1affd640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d1b7fe640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 -- 172.21.15.33:0/1886241403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 msgr2=0x7f4d1c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 -- 172.21.15.33:0/1886241403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 msgr2=0x7f4d1c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 -- 172.21.15.33:0/1886241403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c19fe50 con 0x7f4d1c1031b0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d1b7fe640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d1affd640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.139+0000 7f4d13fff640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d770 tx=0x7f4d0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.140+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04017070 con 0x7f4d1c1031b0 2024-04-23T18:25:53.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.140+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d040054d0 con 0x7f4d1c1031b0 2024-04-23T18:25:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.140+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04004c30 con 0x7f4d1c1031b0 2024-04-23T18:25:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.140+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a0140 con 0x7f4d1c1031b0 2024-04-23T18:25:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.140+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a0690 con 0x7f4d1c1031b0 2024-04-23T18:25:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.141+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c109450 con 0x7f4d1c1031b0 2024-04-23T18:25:53.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.142+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4d04004dd0 con 0x7f4d1c1031b0 2024-04-23T18:25:53.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.142+0000 7f4d18ff9640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 0x7f4d080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.142+0000 7f4d1affd640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 0x7f4d080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.142+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f4d040d04b0 con 0x7f4d1c1031b0 2024-04-23T18:25:53.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.143+0000 7f4d1affd640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 0x7f4d080a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.146+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d0409cdb0 con 0x7f4d1c1031b0 2024-04-23T18:25:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2942593224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1415148789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:52 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3118319842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.335+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d1c02d810 con 0x7f4d1c1031b0 2024-04-23T18:25:53.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.336+0000 7f4d18ff9640 1 -- 172.21.15.33:0/1886241403 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f4d0409c6b0 con 0x7f4d1c1031b0 2024-04-23T18:25:53.329 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:53.329 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:53.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 msgr2=0x7f4d080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 0x7f4d080a6180 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 msgr2=0x7f4d1c1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d770 tx=0x7f4d0400dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 shutdown_connections 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4d080a3cc0 0x7f4d080a6180 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d1c103b70 0x7f4d1c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d1c1031b0 0x7f4d1c1987c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 --2- 172.21.15.33:0/1886241403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d1c101fc0 0x7f4d1c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 >> 172.21.15.33:0/1886241403 conn(0x7f4d1c0fd550 msgr2=0x7f4d1c106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.338+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 shutdown_connections 2024-04-23T18:25:53.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.339+0000 7f4d2155d640 1 -- 172.21.15.33:0/1886241403 wait complete. 2024-04-23T18:25:53.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:25:53.344 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:25:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:52 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2942593224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:52 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1415148789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:52 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3118319842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:52 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2942593224' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:52 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1415148789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:52 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3118319842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:53.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 -- 172.21.15.33:0/3445360988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0102a30 msgr2=0x7f36c0104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 --2- 172.21.15.33:0/3445360988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0102a30 0x7f36c0104e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009900 tx=0x7f36b002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:53.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 -- 172.21.15.33:0/3445360988 shutdown_connections 2024-04-23T18:25:53.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 --2- 172.21.15.33:0/3445360988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0107c90 0x7f36c010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 --2- 172.21.15.33:0/3445360988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0105360 0x7f36c0107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 --2- 172.21.15.33:0/3445360988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0102a30 0x7f36c0104e20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 -- 172.21.15.33:0/3445360988 >> 172.21.15.33:0/3445360988 conn(0x7f36c00fc440 msgr2=0x7f36c00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 -- 172.21.15.33:0/3445360988 shutdown_connections 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.456+0000 7f36c538a640 1 -- 172.21.15.33:0/3445360988 wait complete. 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 Processor -- start 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 -- start start 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 0x7f36c01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 0x7f36c019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010daf0 con 0x7f36c0105360 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010d970 con 0x7f36c0102a30 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36c538a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36c010dc70 con 0x7f36c0107c90 2024-04-23T18:25:53.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36be7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36be7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32952/0 (socket says 172.21.15.33:32952) 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36be7fc640 1 -- 172.21.15.33:0/1942887222 learned_addr learned my addr 172.21.15.33:0/1942887222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36bf7fe640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 0x7f36c019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36beffd640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 0x7f36c01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.457+0000 7f36be7fc640 1 -- 172.21.15.33:0/1942887222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 msgr2=0x7f36c019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36be7fc640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 0x7f36c019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36be7fc640 1 -- 172.21.15.33:0/1942887222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 msgr2=0x7f36c01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36be7fc640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 0x7f36c01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36be7fc640 1 -- 172.21.15.33:0/1942887222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c019fe10 con 0x7f36c0105360 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36bf7fe640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 0x7f36c019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36beffd640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 0x7f36c01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36be7fc640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c970 tx=0x7f36b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4007bb0 con 0x7f36c0105360 2024-04-23T18:25:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b4007d50 con 0x7f36c0105360 2024-04-23T18:25:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c01a0100 con 0x7f36c0105360 2024-04-23T18:25:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4018730 con 0x7f36c0105360 2024-04-23T18:25:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.458+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c01a0650 con 0x7f36c0105360 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.459+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c00fdf30 con 0x7f36c0105360 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.460+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f36b400f990 con 0x7f36c0105360 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.461+0000 7f36a7fff640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 0x7f36a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.461+0000 7f36beffd640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 0x7f36a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.461+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f36b40d0620 con 0x7f36c0105360 2024-04-23T18:25:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.461+0000 7f36beffd640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 0x7f36a80a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f36b0006010 tx=0x7f36b0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.465+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36b4014030 con 0x7f36c0105360 2024-04-23T18:25:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.649+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f36c002d810 con 0x7f36c0105360 2024-04-23T18:25:53.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.650+0000 7f36a7fff640 1 -- 172.21.15.33:0/1942887222 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f36b409c810 con 0x7f36c0105360 2024-04-23T18:25:53.643 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 msgr2=0x7f36a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 0x7f36a80a6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f36b0006010 tx=0x7f36b0000bc0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 msgr2=0x7f36c0198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c970 tx=0x7f36b400ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 shutdown_connections 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f36a80a3cc0 0x7f36a80a6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36c0107c90 0x7f36c019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36c0105360 0x7f36c0198730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 --2- 172.21.15.33:0/1942887222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c0102a30 0x7f36c01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 >> 172.21.15.33:0/1942887222 conn(0x7f36c00fc440 msgr2=0x7f36c0103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 shutdown_connections 2024-04-23T18:25:53.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.652+0000 7f36c538a640 1 -- 172.21.15.33:0/1942887222 wait complete. 2024-04-23T18:25:53.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:25:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.767+0000 7f4b0086b640 1 -- 172.21.15.33:0/1685308612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc102980 msgr2=0x7f4afc1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.767+0000 7f4b0086b640 1 --2- 172.21.15.33:0/1685308612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc102980 0x7f4afc1092a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009ca0 tx=0x7f4af402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 -- 172.21.15.33:0/1685308612 shutdown_connections 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 --2- 172.21.15.33:0/1685308612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc102980 0x7f4afc1092a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 --2- 172.21.15.33:0/1685308612 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc101fc0 0x7f4afc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 --2- 172.21.15.33:0/1685308612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc100e70 0x7f4afc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 -- 172.21.15.33:0/1685308612 >> 172.21.15.33:0/1685308612 conn(0x7f4afc0fc3e0 msgr2=0x7f4afc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 -- 172.21.15.33:0/1685308612 shutdown_connections 2024-04-23T18:25:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 -- 172.21.15.33:0/1685308612 wait complete. 2024-04-23T18:25:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 Processor -- start 2024-04-23T18:25:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.768+0000 7f4b0086b640 1 -- start start 2024-04-23T18:25:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc101fc0 0x7f4afc19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc102980 0x7f4afc1a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10c9a0 con 0x7f4afc100e70 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10c820 con 0x7f4afc102980 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc10cb20 con 0x7f4afc101fc0 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32982/0 (socket says 172.21.15.33:32982) 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 -- 172.21.15.33:0/4173497159 learned_addr learned my addr 172.21.15.33:0/4173497159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4af9d9b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc101fc0 0x7f4afc19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afad9d640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc102980 0x7f4afc1a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 -- 172.21.15.33:0/4173497159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc101fc0 msgr2=0x7f4afc19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc101fc0 0x7f4afc19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 -- 172.21.15.33:0/4173497159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc102980 msgr2=0x7f4afc1a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc102980 0x7f4afc1a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 -- 172.21.15.33:0/4173497159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc1a1f80 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4afa59c640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00d950 tx=0x7f4aec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec017070 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aec009c30 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc1a2270 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.769+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00f620 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.770+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1a27c0 con 0x7f4afc100e70 2024-04-23T18:25:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.771+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc1a2a20 con 0x7f4afc100e70 2024-04-23T18:25:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.772+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4aec010460 con 0x7f4afc100e70 2024-04-23T18:25:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.773+0000 7f4aeb7fe640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 0x7f4ae40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.773+0000 7f4af9d9b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 0x7f4ae40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.773+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f4aec0d0110 con 0x7f4afc100e70 2024-04-23T18:25:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.773+0000 7f4af9d9b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 0x7f4ae40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4afc19b870 tx=0x7f4af000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:53.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.778+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4aec009de0 con 0x7f4afc100e70 2024-04-23T18:25:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.966+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4afc1a2d10 con 0x7f4afc100e70 2024-04-23T18:25:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.967+0000 7f4aeb7fe640 1 -- 172.21.15.33:0/4173497159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f4aec09c340 con 0x7f4afc100e70 2024-04-23T18:25:53.960 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 msgr2=0x7f4ae40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 0x7f4ae40a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4afc19b870 tx=0x7f4af000b390 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 msgr2=0x7f4afc19a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00d950 tx=0x7f4aec00de10 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 shutdown_connections 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f4ae40a3c40 0x7f4ae40a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4afc102980 0x7f4afc1a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4afc101fc0 0x7f4afc19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 --2- 172.21.15.33:0/4173497159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4afc100e70 0x7f4afc19a360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 >> 172.21.15.33:0/4173497159 conn(0x7f4afc0fc3e0 msgr2=0x7f4afc0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 shutdown_connections 2024-04-23T18:25:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:53.969+0000 7f4b0086b640 1 -- 172.21.15.33:0/4173497159 wait complete. 2024-04-23T18:25:53.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:25:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 -- 172.21.15.33:0/2130676604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c101fb0 msgr2=0x7f814c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 --2- 172.21.15.33:0/2130676604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c101fb0 0x7f814c1023d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f813c009900 tx=0x7f813c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 -- 172.21.15.33:0/2130676604 shutdown_connections 2024-04-23T18:25:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 --2- 172.21.15.33:0/2130676604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c103b60 0x7f814c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 --2- 172.21.15.33:0/2130676604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c1031a0 0x7f814c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 --2- 172.21.15.33:0/2130676604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c101fb0 0x7f814c1023d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 -- 172.21.15.33:0/2130676604 >> 172.21.15.33:0/2130676604 conn(0x7f814c0fd520 msgr2=0x7f814c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 -- 172.21.15.33:0/2130676604 shutdown_connections 2024-04-23T18:25:54.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.084+0000 7f8151b22640 1 -- 172.21.15.33:0/2130676604 wait complete. 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 Processor -- start 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 -- start start 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c101fb0 0x7f814c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c103b60 0x7f814c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10db40 con 0x7f814c1031a0 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10d9c0 con 0x7f814c103b60 2024-04-23T18:25:54.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8151b22640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f814c10dcc0 con 0x7f814c101fb0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8142ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8142ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32992/0 (socket says 172.21.15.33:32992) 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.085+0000 7f8142ffd640 1 -- 172.21.15.33:0/1726899173 learned_addr learned my addr 172.21.15.33:0/1726899173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f814b7fe640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c101fb0 0x7f814c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 -- 172.21.15.33:0/1726899173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c101fb0 msgr2=0x7f814c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c101fb0 0x7f814c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 -- 172.21.15.33:0/1726899173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c103b60 msgr2=0x7f814c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c103b60 0x7f814c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 -- 172.21.15.33:0/1726899173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c075110 con 0x7f814c1031a0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8142ffd640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f813400d770 tx=0x7f813400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134017070 con 0x7f814c1031a0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c1a22e0 con 0x7f814c1031a0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81340054d0 con 0x7f814c1031a0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134004c30 con 0x7f814c1031a0 2024-04-23T18:25:54.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.086+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1a2830 con 0x7f814c1031a0 2024-04-23T18:25:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.087+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c109320 con 0x7f814c1031a0 2024-04-23T18:25:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.088+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8134004ed0 con 0x7f814c1031a0 2024-04-23T18:25:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.089+0000 7f81497fa640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 0x7f81380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.089+0000 7f814b7fe640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 0x7f81380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.089+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f81340cffd0 con 0x7f814c1031a0 2024-04-23T18:25:54.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.089+0000 7f814b7fe640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 0x7f81380a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f813c005ab0 tx=0x7f813c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.093+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f813409c910 con 0x7f814c1031a0 2024-04-23T18:25:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:53 smithi033 ceph-mon[29274]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1825129592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1886241403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1942887222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4173497159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.308+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f814c02d770 con 0x7f814c1031a0 2024-04-23T18:25:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.309+0000 7f81497fa640 1 -- 172.21.15.33:0/1726899173 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f813409c210 con 0x7f814c1031a0 2024-04-23T18:25:54.301 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 msgr2=0x7f81380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 0x7f81380a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f813c005ab0 tx=0x7f813c005a20 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 msgr2=0x7f814c074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f813400d770 tx=0x7f813400dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 shutdown_connections 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f81380a3c40 0x7f81380a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f814c103b60 0x7f814c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f814c1031a0 0x7f814c074340 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 --2- 172.21.15.33:0/1726899173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f814c101fb0 0x7f814c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 >> 172.21.15.33:0/1726899173 conn(0x7f814c0fd520 msgr2=0x7f814c1064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 shutdown_connections 2024-04-23T18:25:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.311+0000 7f8151b22640 1 -- 172.21.15.33:0/1726899173 wait complete. 2024-04-23T18:25:54.315 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:54.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:53 smithi106 ceph-mon[32716]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:54.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1825129592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1886241403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1942887222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4173497159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 -- 172.21.15.33:0/1291138246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 msgr2=0x7fb504101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 --2- 172.21.15.33:0/1291138246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504101280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 -- 172.21.15.33:0/1291138246 shutdown_connections 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 --2- 172.21.15.33:0/1291138246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102a00 0x7fb504109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 --2- 172.21.15.33:0/1291138246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102040 0x7fb5041024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 --2- 172.21.15.33:0/1291138246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504101280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 -- 172.21.15.33:0/1291138246 >> 172.21.15.33:0/1291138246 conn(0x7fb5040fc440 msgr2=0x7fb5040fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 -- 172.21.15.33:0/1291138246 shutdown_connections 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.424+0000 7fb508cae640 1 -- 172.21.15.33:0/1291138246 wait complete. 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 Processor -- start 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 -- start start 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 0x7fb5041034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102a00 0x7fb5041039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410cad0 con 0x7fb504100e60 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410c950 con 0x7fb504102040 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb508cae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410cc50 con 0x7fb504102a00 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33010/0 (socket says 172.21.15.33:33010) 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 -- 172.21.15.33:0/2362970976 learned_addr learned my addr 172.21.15.33:0/2362970976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb501d9b640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 0x7fb5041034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb502d9d640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102a00 0x7fb5041039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 -- 172.21.15.33:0/2362970976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102a00 msgr2=0x7fb5041039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102a00 0x7fb5041039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 -- 172.21.15.33:0/2362970976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 msgr2=0x7fb5041034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 0x7fb5041034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb50259c640 1 -- 172.21.15.33:0/2362970976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb504104280 con 0x7fb504100e60 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.425+0000 7fb501d9b640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 0x7fb5041034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:54.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb50259c640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f403f030 con 0x7fb504100e60 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4f4003cb0 con 0x7fb504100e60 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4035070 con 0x7fb504100e60 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a2450 con 0x7fb504100e60 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.426+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a2a20 con 0x7fb504100e60 2024-04-23T18:25:54.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.427+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5041a2d40 con 0x7fb504100e60 2024-04-23T18:25:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.428+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb4f4042030 con 0x7fb504100e60 2024-04-23T18:25:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.429+0000 7fb4f37fe640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 0x7fb4ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.429+0000 7fb501d9b640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 0x7fb4ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.429+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fb4f40f1130 con 0x7fb504100e60 2024-04-23T18:25:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.429+0000 7fb501d9b640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 0x7fb4ec0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb504104be0 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.432+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb4f40bd9b0 con 0x7fb504100e60 2024-04-23T18:25:54.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:53 smithi122 ceph-mon[32546]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:25:54.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1825129592' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1886241403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:54.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1942887222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4173497159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:25:54.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.657+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5041a2f10 con 0x7fb504100e60 2024-04-23T18:25:54.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.658+0000 7fb4f37fe640 1 -- 172.21.15.33:0/2362970976 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7fb4f40b7e20 con 0x7fb504100e60 2024-04-23T18:25:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:54.651 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:25:54.651 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 msgr2=0x7fb4ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 0x7fb4ec0a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb504104be0 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 msgr2=0x7fb504104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f4004430 comp rx=0 tx=0).stop 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 shutdown_connections 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fb4ec0a3c40 0x7fb4ec0a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504102a00 0x7fb5041039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb504102040 0x7fb5041034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 --2- 172.21.15.33:0/2362970976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb504100e60 0x7fb504104e70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 >> 172.21.15.33:0/2362970976 conn(0x7fb5040fc440 msgr2=0x7fb50406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 shutdown_connections 2024-04-23T18:25:54.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.660+0000 7fb508cae640 1 -- 172.21.15.33:0/2362970976 wait complete. 2024-04-23T18:25:54.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 -- 172.21.15.33:0/1908876531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 msgr2=0x7fc2541012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 --2- 172.21.15.33:0/1908876531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc2541012b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc244009900 tx=0x7fc24402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 -- 172.21.15.33:0/1908876531 shutdown_connections 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 --2- 172.21.15.33:0/1908876531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2541029a0 0x7fc2541092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 --2- 172.21.15.33:0/1908876531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 0x7fc254102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 --2- 172.21.15.33:0/1908876531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc2541012b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 -- 172.21.15.33:0/1908876531 >> 172.21.15.33:0/1908876531 conn(0x7fc2540fc400 msgr2=0x7fc2540fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 -- 172.21.15.33:0/1908876531 shutdown_connections 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.772+0000 7fc25b091640 1 -- 172.21.15.33:0/1908876531 wait complete. 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 Processor -- start 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 -- start start 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 0x7fc254198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2541029a0 0x7fc25419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410ca60 con 0x7fc254100e90 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410c8e0 con 0x7fc2541029a0 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc25b091640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc25410cbe0 con 0x7fc254101fe0 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33022/0 (socket says 172.21.15.33:33022) 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 -- 172.21.15.33:0/2408092359 learned_addr learned my addr 172.21.15.33:0/2408092359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc253fff640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 0x7fc254198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 -- 172.21.15.33:0/2408092359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 msgr2=0x7fc254198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 0x7fc254198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 -- 172.21.15.33:0/2408092359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2541029a0 msgr2=0x7fc25419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2541029a0 0x7fc25419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.773+0000 7fc258e2d640 1 -- 172.21.15.33:0/2408092359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25419fe50 con 0x7fc254100e90 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc258e2d640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc244031500 tx=0x7fc2440043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24403f030 con 0x7fc254100e90 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc244003800 con 0x7fc254100e90 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2541a00e0 con 0x7fc254100e90 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244035070 con 0x7fc254100e90 2024-04-23T18:25:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.774+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541a06a0 con 0x7fc254100e90 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.775+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2541a0900 con 0x7fc254100e90 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.776+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc2440039b0 con 0x7fc254100e90 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.777+0000 7fc251ffb640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 0x7fc23c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.777+0000 7fc253fff640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 0x7fc23c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.777+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc2440f1000 con 0x7fc254100e90 2024-04-23T18:25:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.778+0000 7fc253fff640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 0x7fc23c0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc254199740 tx=0x7fc248008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.782+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2440bd880 con 0x7fc254100e90 2024-04-23T18:25:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.975+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2541a0c90 con 0x7fc254100e90 2024-04-23T18:25:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.976+0000 7fc251ffb640 1 -- 172.21.15.33:0/2408092359 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fc2440bd180 con 0x7fc254100e90 2024-04-23T18:25:54.969 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:54.970 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:25:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 msgr2=0x7fc23c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 0x7fc23c0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc254199740 tx=0x7fc248008040 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 msgr2=0x7fc254198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc244031500 tx=0x7fc2440043c0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 shutdown_connections 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc23c0a3cc0 0x7fc23c0a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2541029a0 0x7fc25419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc254101fe0 0x7fc254198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 --2- 172.21.15.33:0/2408092359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc254100e90 0x7fc254198230 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.978+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 >> 172.21.15.33:0/2408092359 conn(0x7fc2540fc400 msgr2=0x7fc2540fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.979+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 shutdown_connections 2024-04-23T18:25:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:54.979+0000 7fc25b091640 1 -- 172.21.15.33:0/2408092359 wait complete. 2024-04-23T18:25:54.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:25:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.095+0000 7fcccd260640 1 -- 172.21.15.33:0/1281458192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc81031b0 msgr2=0x7fccc8103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.095+0000 7fcccd260640 1 --2- 172.21.15.33:0/1281458192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc81031b0 0x7fccc8103630 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fccbc009900 tx=0x7fccbc02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 -- 172.21.15.33:0/1281458192 shutdown_connections 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 --2- 172.21.15.33:0/1281458192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc8103b70 0x7fccc810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 --2- 172.21.15.33:0/1281458192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc81031b0 0x7fccc8103630 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 --2- 172.21.15.33:0/1281458192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 0x7fccc81023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 -- 172.21.15.33:0/1281458192 >> 172.21.15.33:0/1281458192 conn(0x7fccc80fd550 msgr2=0x7fccc80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 -- 172.21.15.33:0/1281458192 shutdown_connections 2024-04-23T18:25:55.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 -- 172.21.15.33:0/1281458192 wait complete. 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.096+0000 7fcccd260640 1 Processor -- start 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 -- start start 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 0x7fccc8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 0x7fccc8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810dbe0 con 0x7fccc8103b70 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810da60 con 0x7fccc8101fc0 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fcccd260640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fccc810dd60 con 0x7fccc81031b0 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33034/0 (socket says 172.21.15.33:33034) 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 -- 172.21.15.33:0/3845079489 learned_addr learned my addr 172.21.15.33:0/3845079489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc659c640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 0x7fccc8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc6d9d640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 0x7fccc8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 -- 172.21.15.33:0/3845079489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 msgr2=0x7fccc8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 0x7fccc8198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 -- 172.21.15.33:0/3845079489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 msgr2=0x7fccc8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 0x7fccc8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 -- 172.21.15.33:0/3845079489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc819ff70 con 0x7fccc8103b70 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc659c640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 0x7fccc8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccc759e640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b970 tx=0x7fccc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.097+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0019070 con 0x7fccc8103b70 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.098+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc81a0260 con 0x7fccc8103b70 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.098+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccc0014440 con 0x7fccc8103b70 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.098+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00075d0 con 0x7fccc8103b70 2024-04-23T18:25:55.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.098+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a0830 con 0x7fccc8103b70 2024-04-23T18:25:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.099+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc8109450 con 0x7fccc8103b70 2024-04-23T18:25:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.100+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fccc001c070 con 0x7fccc8103b70 2024-04-23T18:25:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.100+0000 7fccaffff640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 0x7fccb00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.100+0000 7fccc6d9d640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 0x7fccb00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.100+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fccc00cfa40 con 0x7fccc8103b70 2024-04-23T18:25:55.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.101+0000 7fccc6d9d640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 0x7fccb00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccc8103010 tx=0x7fccb8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.105+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fccc00d1e20 con 0x7fccc8103b70 2024-04-23T18:25:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:54 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1726899173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:54 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2362970976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:54 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2408092359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.332+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc8062b50 con 0x7fccc8103b70 2024-04-23T18:25:55.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.333+0000 7fccaffff640 1 -- 172.21.15.33:0/3845079489 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7fccc009c420 con 0x7fccc8103b70 2024-04-23T18:25:55.326 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:25:55.326 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:25:55.326 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 msgr2=0x7fccb00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 0x7fccb00a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fccc8103010 tx=0x7fccb8008040 comp rx=0 tx=0).stop 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 msgr2=0x7fccc819f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b970 tx=0x7fccc000be30 comp rx=0 tx=0).stop 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 shutdown_connections 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fccb00a3bb0 0x7fccb00a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8103b70 0x7fccc819f900 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fccc81031b0 0x7fccc8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 --2- 172.21.15.33:0/3845079489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccc8101fc0 0x7fccc8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 >> 172.21.15.33:0/3845079489 conn(0x7fccc80fd550 msgr2=0x7fccc8106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 shutdown_connections 2024-04-23T18:25:55.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.335+0000 7fcccd260640 1 -- 172.21.15.33:0/3845079489 wait complete. 2024-04-23T18:25:55.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:24:26.661481+0000', 'modified': '2024-04-23T18:24:34.355479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24551, 'mds_1': 24515, 'mds_2': 24490, 'mds_3': 24482, 'mds_4': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/1373616976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 1373616976}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 1373616976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/363309287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 363309287}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 363309287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6833/2194426420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 2194426420}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 2194426420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6833/637332963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 637332963}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 637332963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6835/3209242154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 3209242154}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 3209242154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.122:6837/3846576090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 3846576090}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 3846576090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.106:6835/3607877814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3607877814}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3607877814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6839/538103378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 538103378}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 538103378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.106:6837/319312146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 319312146}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 319312146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/830372376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 830372376}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 830372376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24490, 24551, 24482, 24515, 24466]} 2024-04-23T18:25:55.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:25:55.340 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:25:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:54 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1726899173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:54 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2362970976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:54 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2408092359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 -- 172.21.15.33:0/1425691344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8107cf0 msgr2=0x7f26d810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 --2- 172.21.15.33:0/1425691344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8107cf0 0x7f26d810a0e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009ca0 tx=0x7f26d002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 -- 172.21.15.33:0/1425691344 shutdown_connections 2024-04-23T18:25:55.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 --2- 172.21.15.33:0/1425691344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8107cf0 0x7f26d810a0e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 --2- 172.21.15.33:0/1425691344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d81053c0 0x7f26d81077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 --2- 172.21.15.33:0/1425691344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d8102a90 0x7f26d8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 -- 172.21.15.33:0/1425691344 >> 172.21.15.33:0/1425691344 conn(0x7f26d80fc440 msgr2=0x7f26d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 -- 172.21.15.33:0/1425691344 shutdown_connections 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.449+0000 7f26de322640 1 -- 172.21.15.33:0/1425691344 wait complete. 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 Processor -- start 2024-04-23T18:25:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 -- start start 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 0x7f26d81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 0x7f26d819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810db70 con 0x7f26d8102a90 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810d9f0 con 0x7f26d8107cf0 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26de322640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26d810dcf0 con 0x7f26d81053c0 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33044/0 (socket says 172.21.15.33:33044) 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 -- 172.21.15.33:0/1766475621 learned_addr learned my addr 172.21.15.33:0/1766475621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 0x7f26d819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d77fe640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 0x7f26d81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 -- 172.21.15.33:0/1766475621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 msgr2=0x7f26d81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 0x7f26d81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 -- 172.21.15.33:0/1766475621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 msgr2=0x7f26d819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 0x7f26d819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26d7fff640 1 -- 172.21.15.33:0/1766475621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d819fdd0 con 0x7f26d8102a90 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.450+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 0x7f26d819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26d77fe640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 0x7f26d81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26d7fff640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d770 tx=0x7f26c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26d57fa640 1 -- 172.21.15.33:0/1766475621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8017070 con 0x7f26d8102a90 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26d57fa640 1 -- 172.21.15.33:0/1766475621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c80054d0 con 0x7f26d8102a90 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81a00c0 con 0x7f26d8102a90 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26d57fa640 1 -- 172.21.15.33:0/1766475621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8004c30 con 0x7f26d8102a90 2024-04-23T18:25:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.451+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81a0690 con 0x7f26d8102a90 2024-04-23T18:25:55.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.453+0000 7f26d57fa640 1 -- 172.21.15.33:0/1766475621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f26c8004e10 con 0x7f26d8102a90 2024-04-23T18:25:55.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.453+0000 7f26e1030740 1 Processor -- start 2024-04-23T18:25:55.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.453+0000 7f26e1030740 1 -- start start 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e7705cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e76fc1c50 con 0x556e7705dd50 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e76f5e830 con 0x556e7705e710 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26e1030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e76fbdc10 con 0x556e7705cb80 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d57fa640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 0x7f26c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33048/0 (socket says 172.21.15.33:33048) 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 -- 172.21.15.33:0/2463336779 learned_addr learned my addr 172.21.15.33:0/2463336779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 0x7f26c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e7705cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d57fa640 1 -- 172.21.15.33:0/1766475621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f26c80cfef0 con 0x7f26d8102a90 2024-04-23T18:25:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 -- 172.21.15.33:0/2463336779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 msgr2=0x556e7705cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e7705cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 -- 172.21.15.33:0/2463336779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 msgr2=0x556e77064fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77064fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 -- 172.21.15.33:0/2463336779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e76cd42d0 con 0x556e7705dd50 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26c8086d90 tx=0x7f26c8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90c4e1409863ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.454+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 0x7f26c00a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x556e7705dc50 tx=0x7f26d0005ef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26be7fc640 1 -- 172.21.15.33:0/2463336779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8020030 con 0x556e7705dd50 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26be7fc640 1 -- 172.21.15.33:0/2463336779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c809c8d0 con 0x556e7705dd50 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26be7fc640 1 -- 172.21.15.33:0/2463336779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80d3020 con 0x556e7705dd50 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 -- 172.21.15.33:0/2463336779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 msgr2=0x556e7705e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f26c8086d90 tx=0x7f26c8009f90 comp rx=0 tx=0).stop 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 -- 172.21.15.33:0/2463336779 shutdown_connections 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7705e1d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 --2- 172.21.15.33:0/2463336779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e7705cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 -- 172.21.15.33:0/2463336779 >> 172.21.15.33:0/2463336779 conn(0x556e77058070 msgr2=0x556e77058470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 -- 172.21.15.33:0/2463336779 shutdown_connections 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.455+0000 7f26e1030740 1 -- 172.21.15.33:0/2463336779 wait complete. 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 -- start start 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 Processor -- start 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705dd50 0x556e77067510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705e710 0x556e7706e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e77065660 con 0x556e7705cb80 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e76f5c470 con 0x556e7705dd50 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e770657e0 con 0x556e7705e710 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26e1030740 10 client.0 _ll_register_callbacks cb 0x556e76f81f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33058/0 (socket says 172.21.15.33:33058) 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 -- 172.21.15.33:0/2656631686 learned_addr learned my addr 172.21.15.33:0/2656631686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705dd50 0x556e77067510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 -- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705e710 msgr2=0x556e7706e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705e710 0x556e7706e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 -- 172.21.15.33:0/2656631686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705dd50 msgr2=0x556e77067510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705dd50 0x556e77067510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.456+0000 7f26d7fff640 1 -- 172.21.15.33:0/2656631686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e76d9e6b0 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26c809c830 tx=0x7f26c80cdd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3a9e2b07e30e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80b4020 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f26c80a4bc0 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26c809c8d0 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26c8020040 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 10 client.15057 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e770718c0 con 0x556e7705cb80 2024-04-23T18:25:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c809c8d0 con 0x556e7705cb80 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f26c8097f20 con 0x556e7705cb80 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 10 client.15057 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 20 client.15057 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e77073a90 0x556e77074f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 4 client.15057 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.457+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e76fcf670 con 0x556e77073a90 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.458+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e77073a90 0x556e77074f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.458+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e77073a90 0x556e77074f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=81efdba1ccbb7202 server_cookie=ffcc0b785addd2da in_seq=0 out_seq=0 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.458+0000 7f26b77fe640 10 client.15057 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26b77fe640 1 -- 172.21.15.33:0/2656631686 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e76fcf670 con 0x556e77073a90 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26b77fe640 10 client.15057 handle_command_reply: tid=0 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 client.15057 shutdown 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 msgr2=0x556e77066fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f26c809c830 tx=0x7f26c80cdd20 comp rx=0 tx=0).stop 2024-04-23T18:25:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 shutdown_connections 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e77073a90 0x556e77074f00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705e710 0x556e7706e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705dd50 0x556e77067510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 --2- 172.21.15.33:0/2656631686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705cb80 0x556e77066fd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26b77fe640 0 client.15057 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 >> 172.21.15.33:0/2656631686 conn(0x556e77058070 msgr2=0x556e770571d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 shutdown_connections 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.459+0000 7f26e1030740 1 -- 172.21.15.33:0/2656631686 wait complete. 2024-04-23T18:25:55.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.460+0000 7f26e1030740 20 client.15057 trim_cache size 0 max 16384 2024-04-23T18:25:55.458 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:54 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1726899173' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:25:55.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:54 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2362970976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:54 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2408092359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 Processor -- start 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 -- start start 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e77057b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77072c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e77073240 con 0x556e7705dd50 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e77073910 con 0x556e7705e710 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26e1030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e77073fe0 con 0x556e7705cb80 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33070/0 (socket says 172.21.15.33:33070) 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26d77fe640 1 -- 172.21.15.33:0/1068397008 learned_addr learned my addr 172.21.15.33:0/1068397008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26d7fff640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e77057b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.478+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 -- 172.21.15.33:0/1068397008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 msgr2=0x556e77057b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e77057b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 -- 172.21.15.33:0/1068397008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 msgr2=0x556e77072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77072c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 -- 172.21.15.33:0/1068397008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e7705d570 con 0x556e7705dd50 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26d77fe640 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00aac0 tx=0x7f26cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bf88476e1b56a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26bdffb640 1 -- 172.21.15.33:0/1068397008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc00ae30 con 0x556e7705dd50 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26bdffb640 1 -- 172.21.15.33:0/1068397008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26cc020030 con 0x556e7705dd50 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.479+0000 7f26bdffb640 1 -- 172.21.15.33:0/1068397008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc0243d0 con 0x556e7705dd50 2024-04-23T18:25:55.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- 172.21.15.33:0/1068397008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 msgr2=0x556e7706e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00aac0 tx=0x7f26cc008040 comp rx=0 tx=0).stop 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- 172.21.15.33:0/1068397008 shutdown_connections 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705e710 0x556e77072c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705dd50 0x556e7706e4e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- 172.21.15.33:0/1068397008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705cb80 0x556e77057b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- 172.21.15.33:0/1068397008 >> 172.21.15.33:0/1068397008 conn(0x556e77058070 msgr2=0x556e77060d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- 172.21.15.33:0/1068397008 shutdown_connections 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- 172.21.15.33:0/1068397008 wait complete. 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 -- start start 2024-04-23T18:25:55.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 Processor -- start 2024-04-23T18:25:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705cb80 0x556e77060510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705dd50 0x556e77060a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.480+0000 7f26e1030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556e77074d80 con 0x556e7705e710 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556e770746b0 con 0x556e7705cb80 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e77075450 con 0x556e7705dd50 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 10 client.0 _ll_register_callbacks cb 0x7f26cc00a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33076/0 (socket says 172.21.15.33:33076) 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 -- 172.21.15.33:0/2470445763 learned_addr learned my addr 172.21.15.33:0/2470445763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705cb80 0x556e77060510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26d77fe640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705dd50 0x556e77060a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 -- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705dd50 msgr2=0x556e77060a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705dd50 0x556e77060a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 -- 172.21.15.33:0/2470445763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705cb80 msgr2=0x556e77060510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705cb80 0x556e77060510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 -- 172.21.15.33:0/2470445763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e770d8010 con 0x556e7705e710 2024-04-23T18:25:55.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26dc8bf640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26d0032e50 tx=0x7f26d002f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6800b7e7ee5293fd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004040 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f26d00041e0 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26cc020030 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f26d005c050 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 10 client.15063 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.481+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556e770d9310 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26cc020030 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f26d002fc10 con 0x556e7705e710 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 10 client.15063 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 10 client.15063 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 20 client.15063 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e770de050 0x556e770dcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 4 client.15063 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "status"}) -- 0x556e76fc94e0 con 0x556e770de050 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e770de050 0x556e770dcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26d7fff640 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e770de050 0x556e770dcd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2ce0de7a0ad96e99 server_cookie=e4087d8b42b62738 in_seq=0 out_seq=0 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.482+0000 7f26b77fe640 10 client.15063 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26b77fe640 1 -- 172.21.15.33:0/2470445763 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556e76fc94e0 con 0x556e770de050 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26b77fe640 10 client.15063 handle_command_reply: tid=0 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 client.15063 shutdown 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 msgr2=0x556e770d7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26d0032e50 tx=0x7f26d002f390 comp rx=0 tx=0).stop 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 shutdown_connections 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x556e770de050 0x556e770dcd60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556e7705e710 0x556e770d7a30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e7705dd50 0x556e77060a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 --2- 172.21.15.33:0/2470445763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556e7705cb80 0x556e77060510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 >> 172.21.15.33:0/2470445763 conn(0x556e77058070 msgr2=0x556e77066bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26b77fe640 0 client.15063 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 shutdown_connections 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 1 -- 172.21.15.33:0/2470445763 wait complete. 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26e1030740 20 client.15063 trim_cache size 0 max 16384 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24466, 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 82.144028492000004, 2024-04-23T18:25:55.482 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 106.867540757 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 msgr2=0x7f26c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 0x7f26c00a60f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x556e7705dc50 tx=0x7f26d0005ef0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 msgr2=0x7f26d81981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.483+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f26c800d770 tx=0x7f26c800dc30 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 shutdown_connections 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f26c00a3c30 0x7f26c00a60f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26d8107cf0 0x7f26d819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26d81053c0 0x7f26d81986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 --2- 172.21.15.33:0/1766475621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26d8102a90 0x7f26d81981b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 >> 172.21.15.33:0/1766475621 conn(0x7f26d80fc440 msgr2=0x7f26d81033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 shutdown_connections 2024-04-23T18:25:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.484+0000 7f26de322640 1 -- 172.21.15.33:0/1766475621 wait complete. 2024-04-23T18:25:55.487 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-23T18:25:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 -- 172.21.15.33:0/1604091928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78105340 msgr2=0x7faa78107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1604091928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78105340 0x7faa78107730 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faa60009900 tx=0x7faa6002a410 comp rx=0 tx=0).stop 2024-04-23T18:25:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 -- 172.21.15.33:0/1604091928 shutdown_connections 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1604091928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78107c70 0x7faa7810a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1604091928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78105340 0x7faa78107730 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1604091928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78102a10 0x7faa78104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 -- 172.21.15.33:0/1604091928 >> 172.21.15.33:0/1604091928 conn(0x7faa780fc400 msgr2=0x7faa780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 -- 172.21.15.33:0/1604091928 shutdown_connections 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.597+0000 7faa7e34a640 1 -- 172.21.15.33:0/1604091928 wait complete. 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 Processor -- start 2024-04-23T18:25:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 -- start start 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78105340 0x7faa781009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 0x7faa78100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810daa0 con 0x7faa78102a10 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810d920 con 0x7faa78105340 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7e34a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa7810dc20 con 0x7faa78107c70 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33092/0 (socket says 172.21.15.33:33092) 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 -- 172.21.15.33:0/1239052834 learned_addr learned my addr 172.21.15.33:0/1239052834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa777fe640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78105340 0x7faa781009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 0x7faa78100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 -- 172.21.15.33:0/1239052834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 msgr2=0x7faa78100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 0x7faa78100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 -- 172.21.15.33:0/1239052834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78105340 msgr2=0x7faa781009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78105340 0x7faa781009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa77fff640 1 -- 172.21.15.33:0/1239052834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa78101790 con 0x7faa78102a10 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.598+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 0x7faa78100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa77fff640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa6800d950 tx=0x7faa6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa757fa640 1 -- 172.21.15.33:0/1239052834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68017070 con 0x7faa78102a10 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa757fa640 1 -- 172.21.15.33:0/1239052834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa68009c30 con 0x7faa78102a10 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa781a2300 con 0x7faa78102a10 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa757fa640 1 -- 172.21.15.33:0/1239052834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6800f620 con 0x7faa78102a10 2024-04-23T18:25:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.599+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a2820 con 0x7faa78102a10 2024-04-23T18:25:55.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa757fa640 1 -- 172.21.15.33:0/1239052834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa68010460 con 0x7faa78102a10 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 Processor -- start 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 -- start start 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 0x55dab22ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ecd50 0x55dab22ed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dab2250c50 con 0x55dab22ed710 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dab21ed830 con 0x55dab22ebb80 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa8105a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dab224cc10 con 0x55dab22ecd50 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.601+0000 7faa757fa640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 0x7faa640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33100/0 (socket says 172.21.15.33:33100) 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 -- 172.21.15.33:0/577534103 learned_addr learned my addr 172.21.15.33:0/577534103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 0x55dab22ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa77fff640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ecd50 0x55dab22ed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 0x7faa640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa757fa640 1 -- 172.21.15.33:0/1239052834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7faa680d0030 con 0x7faa78102a10 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 -- 172.21.15.33:0/577534103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ecd50 msgr2=0x55dab22ed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ecd50 0x55dab22ed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 -- 172.21.15.33:0/577534103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 msgr2=0x55dab22ebfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 0x55dab22ebfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 -- 172.21.15.33:0/577534103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab1f632d0 con 0x55dab22ed710 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 0x55dab22ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 0x7faa640a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dab22ecc50 tx=0x7faa700094a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa777fe640 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa78069c40 tx=0x7faa60031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8271e5ba7d7d28c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa6e7fc640 1 -- 172.21.15.33:0/577534103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60035030 con 0x55dab22ed710 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa6e7fc640 1 -- 172.21.15.33:0/577534103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa60040070 con 0x55dab22ed710 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.602+0000 7faa6e7fc640 1 -- 172.21.15.33:0/577534103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600039e0 con 0x55dab22ed710 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- 172.21.15.33:0/577534103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 msgr2=0x55dab22f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faa78069c40 tx=0x7faa60031b90 comp rx=0 tx=0).stop 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- 172.21.15.33:0/577534103 shutdown_connections 2024-04-23T18:25:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f3fa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ecd50 0x55dab22ed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 --2- 172.21.15.33:0/577534103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ebb80 0x55dab22ebfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- 172.21.15.33:0/577534103 >> 172.21.15.33:0/577534103 conn(0x55dab22e7070 msgr2=0x55dab22e7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- 172.21.15.33:0/577534103 shutdown_connections 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- 172.21.15.33:0/577534103 wait complete. 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.603+0000 7faa8105a740 1 -- start start 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 Processor -- start 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dab22f46c0 con 0x55dab22ed710 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dab21eb470 con 0x55dab22ecd50 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dab22f4840 con 0x55dab22ebb80 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa8105a740 10 client.0 _ll_register_callbacks cb 0x55dab2210f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33106/0 (socket says 172.21.15.33:33106) 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3145470550 learned_addr learned my addr 172.21.15.33:0/3145470550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa777fe640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa77fff640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3145470550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 msgr2=0x55dab22f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 msgr2=0x55dab22f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3145470550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab202d6b0 con 0x55dab22ed710 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa70012fb0 tx=0x7faa70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7dcf140b554c994 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.604+0000 7faa577fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70025070 con 0x55dab22ed710 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7faa70010da0 con 0x55dab22ed710 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60040070 con 0x55dab22ed710 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa70031050 con 0x55dab22ed710 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dab2300910 con 0x55dab22ed710 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa60040070 con 0x55dab22ed710 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa7001e050 con 0x55dab22ed710 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 10 client.15075 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 20 client.15075 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab2304430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 4 client.15075 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dab225e670 con 0x55dab2302fc0 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.605+0000 7faa77fff640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab2304430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.606+0000 7faa577fe640 10 client.15075 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.606+0000 7faa77fff640 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab2304430 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d50031b12b7f5b23 server_cookie=b048da13bca9c626 in_seq=0 out_seq=0 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa577fe640 1 -- 172.21.15.33:0/3145470550 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7faa68081ae0 con 0x55dab2302fc0 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa577fe640 10 client.15075 handle_command_reply: tid=0 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 client.15075 shutdown 2024-04-23T18:25:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 msgr2=0x55dab22fd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faa70012fb0 tx=0x7faa70004a40 comp rx=0 tx=0).stop 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 shutdown_connections 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab2304430 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22fd530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa577fe640 0 client.15075 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 --2- 172.21.15.33:0/3145470550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 >> 172.21.15.33:0/3145470550 conn(0x55dab22e7070 msgr2=0x55dab22e61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 shutdown_connections 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 1 -- 172.21.15.33:0/3145470550 wait complete. 2024-04-23T18:25:55.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.607+0000 7faa8105a740 20 client.15075 trim_cache size 0 max 16384 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 Processor -- start 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 -- start start 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22fd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dab2302770 con 0x55dab22ed710 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dab2302e40 con 0x55dab22ecd50 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.625+0000 7faa8105a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dab2303510 con 0x55dab22ebb80 2024-04-23T18:25:55.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33112/0 (socket says 172.21.15.33:33112) 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3933513919 learned_addr learned my addr 172.21.15.33:0/3933513919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa77fff640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa777fe640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3933513919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 msgr2=0x55dab22e20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22e20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3933513919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 msgr2=0x55dab22fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 -- 172.21.15.33:0/3933513919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dab22f3b10 con 0x55dab22ed710 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa777fe640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa7c8e7640 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa700125a0 tx=0x7faa70000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3d413000122239 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa6dffb640 1 -- 172.21.15.33:0/3933513919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70025070 con 0x55dab22ed710 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa6dffb640 1 -- 172.21.15.33:0/3933513919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa7002e030 con 0x55dab22ed710 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.626+0000 7faa6dffb640 1 -- 172.21.15.33:0/3933513919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70020e80 con 0x55dab22ed710 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- 172.21.15.33:0/3933513919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 msgr2=0x55dab22f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7faa700125a0 tx=0x7faa70000f40 comp rx=0 tx=0).stop 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- 172.21.15.33:0/3933513919 shutdown_connections 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ed710 0x55dab22f5fb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 --2- 172.21.15.33:0/3933513919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ebb80 0x55dab22e20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- 172.21.15.33:0/3933513919 >> 172.21.15.33:0/3933513919 conn(0x55dab22e7070 msgr2=0x55dab22e7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- 172.21.15.33:0/3933513919 shutdown_connections 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- 172.21.15.33:0/3933513919 wait complete. 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 -- start start 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.627+0000 7faa8105a740 1 Processor -- start 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ed710 0x55dab22fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dab23042b0 con 0x55dab22ebb80 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dab2303be0 con 0x55dab22ecd50 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dab2304980 con 0x55dab22ed710 2024-04-23T18:25:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 10 client.0 _ll_register_callbacks cb 0x7faa700124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33124/0 (socket says 172.21.15.33:33124) 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 -- 172.21.15.33:0/123220396 learned_addr learned my addr 172.21.15.33:0/123220396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa777fe640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 -- 172.21.15.33:0/123220396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ed710 msgr2=0x55dab22fc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ed710 0x55dab22fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 -- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 msgr2=0x55dab22f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 -- 172.21.15.33:0/123220396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dab22fc6d0 con 0x55dab22ebb80 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa77fff640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faa68021000 tx=0x7faa68066a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca03b0be488e68b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa577fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68096060 con 0x55dab22ebb80 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7faa68098980 con 0x55dab22ebb80 2024-04-23T18:25:55.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa7002e030 con 0x55dab22ebb80 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faa680b5070 con 0x55dab22ebb80 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 10 client.15081 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.628+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dab2368f70 con 0x55dab22ebb80 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa7002e030 con 0x55dab22ebb80 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7faa680a4af0 con 0x55dab22ebb80 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 10 client.15081 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 10 client.15081 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 20 client.15081 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab236c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 4 client.15081 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "status"}) -- 0x55dab22584e0 con 0x55dab2302fc0 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.629+0000 7faa777fe640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab236c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa777fe640 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab236c720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8cf35d76472ec23b server_cookie=5c48c1afe8267222 in_seq=0 out_seq=0 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa577fe640 10 client.15081 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa577fe640 1 -- 172.21.15.33:0/123220396 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dab22584e0 con 0x55dab2302fc0 2024-04-23T18:25:55.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa577fe640 10 client.15081 handle_command_reply: tid=0 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 client.15081 shutdown 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 msgr2=0x55dab22f8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7faa68021000 tx=0x7faa68066a10 comp rx=0 tx=0).stop 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 shutdown_connections 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55dab2302fc0 0x55dab236c720 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dab22ed710 0x55dab22fc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa577fe640 0 client.15081 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dab22ecd50 0x55dab22f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 --2- 172.21.15.33:0/123220396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dab22ebb80 0x55dab22f8b80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 >> 172.21.15.33:0/123220396 conn(0x55dab22e7070 msgr2=0x55dab22e20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 shutdown_connections 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 1 -- 172.21.15.33:0/123220396 wait complete. 2024-04-23T18:25:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.630+0000 7faa8105a740 20 client.15081 trim_cache size 0 max 16384 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24482, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 83.297683336999995, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 112.63955257400001 2024-04-23T18:25:55.629 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 msgr2=0x7faa640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 0x7faa640a6070 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dab22ecc50 tx=0x7faa700094a0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 msgr2=0x7faa78102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faa6800d950 tx=0x7faa6800de10 comp rx=0 tx=0).stop 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 shutdown_connections 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7faa640a3bb0 0x7faa640a6070 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa78107c70 0x7faa78100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa78105340 0x7faa781009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 --2- 172.21.15.33:0/1239052834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78102a10 0x7faa78102380 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 >> 172.21.15.33:0/1239052834 conn(0x7faa780fc400 msgr2=0x7faa78104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 shutdown_connections 2024-04-23T18:25:55.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.631+0000 7faa7e34a640 1 -- 172.21.15.33:0/1239052834 wait complete. 2024-04-23T18:25:55.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:25:55.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 -- 172.21.15.33:0/1879602157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c102a10 msgr2=0x7f807c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 --2- 172.21.15.33:0/1879602157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c102a10 0x7f807c104e00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8074009900 tx=0x7f807402a410 comp rx=0 tx=0).stop 2024-04-23T18:25:55.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 -- 172.21.15.33:0/1879602157 shutdown_connections 2024-04-23T18:25:55.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 --2- 172.21.15.33:0/1879602157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 0x7f807c10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 --2- 172.21.15.33:0/1879602157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c105340 0x7f807c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 --2- 172.21.15.33:0/1879602157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c102a10 0x7f807c104e00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 -- 172.21.15.33:0/1879602157 >> 172.21.15.33:0/1879602157 conn(0x7f807c0fc400 msgr2=0x7f807c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 -- 172.21.15.33:0/1879602157 shutdown_connections 2024-04-23T18:25:55.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.744+0000 7f80811fe640 1 -- 172.21.15.33:0/1879602157 wait complete. 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 Processor -- start 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 -- start start 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 0x7f807c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 0x7f807c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f807c10dae0 con 0x7f807c105340 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f807c10d960 con 0x7f807c102a10 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f80811fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f807c10dc60 con 0x7f807c107c70 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.745+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33146/0 (socket says 172.21.15.33:33146) 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 -- 172.21.15.33:0/1846814295 learned_addr learned my addr 172.21.15.33:0/1846814295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807b59e640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 0x7f807c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807ad9d640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 0x7f807c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 -- 172.21.15.33:0/1846814295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 msgr2=0x7f807c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 0x7f807c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 -- 172.21.15.33:0/1846814295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 msgr2=0x7f807c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 0x7f807c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 -- 172.21.15.33:0/1846814295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c19fda0 con 0x7f807c105340 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807ad9d640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 0x7f807c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f807a59c640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f8073fff640 1 -- 172.21.15.33:0/1846814295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064014070 con 0x7f807c105340 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1a0090 con 0x7f807c105340 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f8073fff640 1 -- 172.21.15.33:0/1846814295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8064004440 con 0x7f807c105340 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f8073fff640 1 -- 172.21.15.33:0/1846814295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064003e50 con 0x7f807c105340 2024-04-23T18:25:55.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.746+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1a0630 con 0x7f807c105340 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.748+0000 7f8073fff640 1 -- 172.21.15.33:0/1846814295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8064017070 con 0x7f807c105340 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 Processor -- start 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8073fff640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 0x7f80680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 -- start start 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f807b59e640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 0x7f80680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8073fff640 1 -- 172.21.15.33:0/1846814295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f80640d0260 con 0x7f807c105340 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a84fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a2099e9c50 con 0x55a209a85d50 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a209986830 con 0x55a209a86710 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f8083ff2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a2099e5c10 con 0x55a209a84b80 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33158/0 (socket says 172.21.15.33:33158) 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.749+0000 7f807a59c640 1 -- 172.21.15.33:0/3136440070 learned_addr learned my addr 172.21.15.33:0/3136440070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807b59e640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 0x7f80680a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f806c005fd0 tx=0x7f806c009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807ad9d640 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a84fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 -- 172.21.15.33:0/3136440070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 msgr2=0x55a209a84fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a84fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 -- 172.21.15.33:0/3136440070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 msgr2=0x55a209a8cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 -- 172.21.15.33:0/3136440070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2096fc2d0 con 0x55a209a85d50 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807ad9d640 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a84fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f807a59c640 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f806400afd0 tx=0x7f80640038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ea261e45315dff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f80717fa640 1 -- 172.21.15.33:0/3136440070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640a08f0 con 0x55a209a85d50 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f80717fa640 1 -- 172.21.15.33:0/3136440070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80640b5070 con 0x55a209a85d50 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.750+0000 7f80717fa640 1 -- 172.21.15.33:0/3136440070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064098290 con 0x55a209a85d50 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- 172.21.15.33:0/3136440070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 msgr2=0x55a209a861d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f806400afd0 tx=0x7f80640038b0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- 172.21.15.33:0/3136440070 shutdown_connections 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a861d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3136440070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a84fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- 172.21.15.33:0/3136440070 >> 172.21.15.33:0/3136440070 conn(0x55a209a80070 msgr2=0x55a209a80470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- 172.21.15.33:0/3136440070 shutdown_connections 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- 172.21.15.33:0/3136440070 wait complete. 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- start start 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 Processor -- start 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a84b80 0x55a209a8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a209a8d6c0 con 0x55a209a86710 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.751+0000 7f8083ff2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a209984470 con 0x55a209a84b80 2024-04-23T18:25:55.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f8083ff2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a209a8d840 con 0x55a209a85d50 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f8083ff2740 10 client.0 _ll_register_callbacks cb 0x55a2099a9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:51158/0 (socket says 172.21.15.33:51158) 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33172/0 (socket says 172.21.15.33:33172) 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807a59c640 1 -- 172.21.15.33:0/3372620296 learned_addr learned my addr 172.21.15.33:0/3372620296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807ad9d640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a84b80 0x55a209a8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 -- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 msgr2=0x55a209a8f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 -- 172.21.15.33:0/3372620296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a84b80 msgr2=0x55a209a8efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a84b80 0x55a209a8efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 -- 172.21.15.33:0/3372620296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2097c66b0 con 0x55a209a86710 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807a59c640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f807b59e640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f806c010bd0 tx=0x7f806c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4e710a88856a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f80627fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c023070 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f806c011cf0 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.752+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80640b5070 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f806c02f050 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a209a99990 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80640b5070 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f806c00f040 con 0x55a209a86710 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 10 client.15093 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 20 client.15093 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209a9c000 0x55a209a9d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 4 client.15093 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2099f7670 con 0x55a209a9c000 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.753+0000 7f807ad9d640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209a9c000 0x55a209a9d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.754+0000 7f807ad9d640 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209a9c000 0x55a209a9d470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3901afe0583b7c server_cookie=5030d76c2c5be212 in_seq=0 out_seq=0 2024-04-23T18:25:55.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.754+0000 7f80627fc640 10 client.15093 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.754+0000 7f80627fc640 1 -- 172.21.15.33:0/3372620296 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2099f7670 con 0x55a209a9c000 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f80627fc640 10 client.15093 handle_command_reply: tid=0 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 client.15093 shutdown 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 msgr2=0x55a209a96550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f806c010bd0 tx=0x7f806c009960 comp rx=0 tx=0).stop 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 shutdown_connections 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209a9c000 0x55a209a9d470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209a96550 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a85d50 0x55a209a8f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f80627fc640 0 client.15093 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 --2- 172.21.15.33:0/3372620296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a84b80 0x55a209a8efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 >> 172.21.15.33:0/3372620296 conn(0x55a209a80070 msgr2=0x55a209a7f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 shutdown_connections 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 1 -- 172.21.15.33:0/3372620296 wait complete. 2024-04-23T18:25:55.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.755+0000 7f8083ff2740 20 client.15093 trim_cache size 0 max 16384 2024-04-23T18:25:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 Processor -- start 2024-04-23T18:25:55.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 -- start start 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9b7b0 con 0x55a209a85d50 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9be80 con 0x55a209a86710 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f8083ff2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9c550 con 0x55a209a84b80 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f807a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33184/0 (socket says 172.21.15.33:33184) 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.773+0000 7f807a59c640 1 -- 172.21.15.33:0/2290309576 learned_addr learned my addr 172.21.15.33:0/2290309576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 -- 172.21.15.33:0/2290309576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 msgr2=0x55a209a8dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 -- 172.21.15.33:0/2290309576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 msgr2=0x55a209a8efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 -- 172.21.15.33:0/2290309576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a209a857d0 con 0x55a209a85d50 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f807a59c640 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8064098ec0 tx=0x7f806409c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b205611e37f202 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f8070ff9640 1 -- 172.21.15.33:0/2290309576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640a51b0 con 0x55a209a85d50 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f8070ff9640 1 -- 172.21.15.33:0/2290309576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80640b9030 con 0x55a209a85d50 2024-04-23T18:25:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.774+0000 7f8070ff9640 1 -- 172.21.15.33:0/2290309576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80640984f0 con 0x55a209a85d50 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- 172.21.15.33:0/2290309576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 msgr2=0x55a209a96510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8064098ec0 tx=0x7f806409c600 comp rx=0 tx=0).stop 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- 172.21.15.33:0/2290309576 shutdown_connections 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a86710 0x55a209a8efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a85d50 0x55a209a96510 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- 172.21.15.33:0/2290309576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a8dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- 172.21.15.33:0/2290309576 >> 172.21.15.33:0/2290309576 conn(0x55a209a80070 msgr2=0x55a209a7b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- 172.21.15.33:0/2290309576 shutdown_connections 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- 172.21.15.33:0/2290309576 wait complete. 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- start start 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 Processor -- start 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a91500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 0x55a209a91a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9d2f0 con 0x55a209a86710 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9cc20 con 0x55a209a85d50 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a209a9d9c0 con 0x55a209a84b80 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f8083ff2740 10 client.0 _ll_register_callbacks cb 0x7f806c016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f807b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f807b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33200/0 (socket says 172.21.15.33:33200) 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.775+0000 7f807b59e640 1 -- 172.21.15.33:0/1119645217 learned_addr learned my addr 172.21.15.33:0/1119645217 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807a59c640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 0x55a209a91a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807ad9d640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a91500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 -- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 msgr2=0x55a209a91500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a91500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 -- 172.21.15.33:0/1119645217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 msgr2=0x55a209a91a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 0x55a209a91a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 -- 172.21.15.33:0/1119645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a209b00560 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807a59c640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 0x55a209a91a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f807b59e640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f806c027660 tx=0x7f806c00f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c84da983a57992 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f80627fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c023030 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f806c027b80 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80640b9030 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f806c0127f0 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f8083ff2740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a209b01890 con 0x55a209a86710 2024-04-23T18:25:55.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.776+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80640b9030 con 0x55a209a86710 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f806c03b030 con 0x55a209a86710 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 20 client.15099 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209b064e0 0x55a209b051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 4 client.15099 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "status"}) -- 0x55a2099f14e0 con 0x55a209b064e0 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f807ad9d640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209b064e0 0x55a209b051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f807ad9d640 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209b064e0 0x55a209b051f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cce3b2cc3e53759f server_cookie=465e37ec0ddb3cff in_seq=0 out_seq=0 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f80627fc640 10 client.15099 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f80627fc640 1 -- 172.21.15.33:0/1119645217 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a2099f14e0 con 0x55a209b064e0 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f80627fc640 10 client.15099 handle_command_reply: tid=0 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.777+0000 7f8083ff2740 1 client.15099 shutdown 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 msgr2=0x55a209afffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f806c027660 tx=0x7f806c00f270 comp rx=0 tx=0).stop 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 shutdown_connections 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x55a209b064e0 0x55a209b051f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a209a86710 0x55a209afffa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a209a85d50 0x55a209a91a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80627fc640 0 client.15099 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 --2- 172.21.15.33:0/1119645217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a209a84b80 0x55a209a91500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 >> 172.21.15.33:0/1119645217 conn(0x55a209a80070 msgr2=0x55a209a8dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 shutdown_connections 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 1 -- 172.21.15.33:0/1119645217 wait complete. 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f8083ff2740 20 client.15099 trim_cache size 0 max 16384 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:25:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24490, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 84.455793366999998, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 101.494360081 2024-04-23T18:25:55.777 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 msgr2=0x7f80680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 0x7f80680a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f806c005fd0 tx=0x7f806c009600 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 msgr2=0x7f807c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f806400c970 tx=0x7f806400ce30 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 shutdown_connections 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f80680a3d50 0x7f80680a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c107c70 0x7f807c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807c105340 0x7f807c1986c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 --2- 172.21.15.33:0/1846814295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807c102a10 0x7f807c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 >> 172.21.15.33:0/1846814295 conn(0x7f807c0fc400 msgr2=0x7f807c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 shutdown_connections 2024-04-23T18:25:55.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.778+0000 7f80811fe640 1 -- 172.21.15.33:0/1846814295 wait complete. 2024-04-23T18:25:55.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 -- 172.21.15.33:0/939366891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102040 msgr2=0x7fc5741024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 --2- 172.21.15.33:0/939366891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102040 0x7fc5741024c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc568009900 tx=0x7fc56802a410 comp rx=0 tx=0).stop 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 -- 172.21.15.33:0/939366891 shutdown_connections 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 --2- 172.21.15.33:0/939366891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102a00 0x7fc574109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 --2- 172.21.15.33:0/939366891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102040 0x7fc5741024c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 --2- 172.21.15.33:0/939366891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 0x7fc574101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 -- 172.21.15.33:0/939366891 >> 172.21.15.33:0/939366891 conn(0x7fc5740fc440 msgr2=0x7fc5740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 -- 172.21.15.33:0/939366891 shutdown_connections 2024-04-23T18:25:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 -- 172.21.15.33:0/939366891 wait complete. 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.892+0000 7fc57a223640 1 Processor -- start 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- start start 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 0x7fc574198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102040 0x7fc574198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410ca40 con 0x7fc574102a00 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410c8c0 con 0x7fc574100e60 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410cbc0 con 0x7fc574102040 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33222/0 (socket says 172.21.15.33:33222) 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 -- 172.21.15.33:0/142991686 learned_addr learned my addr 172.21.15.33:0/142991686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc572ffd640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102040 0x7fc574198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc5737fe640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 0x7fc574198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 -- 172.21.15.33:0/142991686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102040 msgr2=0x7fc574198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102040 0x7fc574198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 -- 172.21.15.33:0/142991686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 msgr2=0x7fc574198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 0x7fc574198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 -- 172.21.15.33:0/142991686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419fe60 con 0x7fc574102a00 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc573fff640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00efd0 tx=0x7fc56c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc570ff9640 1 -- 172.21.15.33:0/142991686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c003b70 con 0x7fc574102a00 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc570ff9640 1 -- 172.21.15.33:0/142991686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56c003d10 con 0x7fc574102a00 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5741a0150 con 0x7fc574102a00 2024-04-23T18:25:55.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc570ff9640 1 -- 172.21.15.33:0/142991686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c0108a0 con 0x7fc574102a00 2024-04-23T18:25:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.893+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a0670 con 0x7fc574102a00 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.895+0000 7fc570ff9640 1 -- 172.21.15.33:0/142991686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc56c004020 con 0x7fc574102a00 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 Processor -- start 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 -- start start 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5b83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207d510c50 con 0x56207d5aff90 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56207d4ad830 con 0x56207d5b1b20 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc57d011740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56207d50cc10 con 0x56207d5b1160 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33236/0 (socket says 172.21.15.33:33236) 2024-04-23T18:25:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 -- 172.21.15.33:0/2525693607 learned_addr learned my addr 172.21.15.33:0/2525693607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc570ff9640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 0x7fc55c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc5737fe640 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc573fff640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 0x7fc55c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 -- 172.21.15.33:0/2525693607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 msgr2=0x56207d5b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 -- 172.21.15.33:0/2525693607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 msgr2=0x56207d5b83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5b83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc572ffd640 1 -- 172.21.15.33:0/2525693607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207d2232d0 con 0x56207d5aff90 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.896+0000 7fc570ff9640 1 -- 172.21.15.33:0/142991686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc56c0d08f0 con 0x7fc574102a00 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc572ffd640 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc568031500 tx=0x7fc568004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bb53b1c1fd1222 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc573fff640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 0x7fc55c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc56c099030 tx=0x7fc56c06f710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc559ffb640 1 -- 172.21.15.33:0/2525693607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568035030 con 0x56207d5aff90 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc559ffb640 1 -- 172.21.15.33:0/2525693607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680054f0 con 0x56207d5aff90 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc559ffb640 1 -- 172.21.15.33:0/2525693607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568003850 con 0x56207d5aff90 2024-04-23T18:25:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 -- 172.21.15.33:0/2525693607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 msgr2=0x56207d5b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc568031500 tx=0x7fc568004110 comp rx=0 tx=0).stop 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 -- 172.21.15.33:0/2525693607 shutdown_connections 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5b83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 --2- 172.21.15.33:0/2525693607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b03b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 -- 172.21.15.33:0/2525693607 >> 172.21.15.33:0/2525693607 conn(0x56207d5a7070 msgr2=0x56207d5a7470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 -- 172.21.15.33:0/2525693607 shutdown_connections 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.897+0000 7fc57d011740 1 -- 172.21.15.33:0/2525693607 wait complete. 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 -- start start 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 Processor -- start 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5ba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207d5b8ad0 con 0x56207d5aff90 2024-04-23T18:25:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56207d4ab470 con 0x56207d5b1b20 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56207d5b8c50 con 0x56207d5b1160 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc57d011740 10 client.0 _ll_register_callbacks cb 0x56207d4d0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33248/0 (socket says 172.21.15.33:33248) 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.898+0000 7fc572ffd640 1 -- 172.21.15.33:0/3911848403 learned_addr learned my addr 172.21.15.33:0/3911848403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5ba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 -- 172.21.15.33:0/3911848403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 msgr2=0x56207d5ba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5ba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 -- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 msgr2=0x56207d5c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 -- 172.21.15.33:0/3911848403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207d2ed6b0 con 0x56207d5aff90 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5ba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc568004620 tx=0x7fc568009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e9e3c7db2c9dec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568035070 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc5680037f0 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5680054f0 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc568058050 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc57d011740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56207d5c4d90 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5680054f0 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc5680319f0 con 0x56207d5aff90 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc57d011740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.899+0000 7fc57d011740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc57d011740 20 client.15111 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d5c6f60 0x56207d5c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc57d011740 4 client.15111 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207d51e670 con 0x56207d5c6f60 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d5c6f60 0x56207d5c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d5c6f60 0x56207d5c83d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a819b7f9c09e041 server_cookie=c953203a3b885dc3 in_seq=0 out_seq=0 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.900+0000 7fc552ffd640 10 client.15111 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc552ffd640 1 -- 172.21.15.33:0/3911848403 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56207d51e670 con 0x56207d5c6f60 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc552ffd640 10 client.15111 handle_command_reply: tid=0 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 client.15111 shutdown 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 msgr2=0x56207d5ba4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc568004620 tx=0x7fc568009520 comp rx=0 tx=0).stop 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 shutdown_connections 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d5c6f60 0x56207d5c83d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1b20 0x56207d5c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1160 0x56207d5ba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 --2- 172.21.15.33:0/3911848403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5ba4a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc552ffd640 0 client.15111 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:25:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 >> 172.21.15.33:0/3911848403 conn(0x56207d5a7070 msgr2=0x56207d5a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 shutdown_connections 2024-04-23T18:25:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 1 -- 172.21.15.33:0/3911848403 wait complete. 2024-04-23T18:25:55.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.901+0000 7fc57d011740 20 client.15111 trim_cache size 0 max 16384 2024-04-23T18:25:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 Processor -- start 2024-04-23T18:25:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 -- start start 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5aff90 0x56207d5b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c6710 con 0x56207d5b1160 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c6de0 con 0x56207d5aff90 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc57d011740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c74b0 con 0x56207d5b1b20 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33260/0 (socket says 172.21.15.33:33260) 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 -- 172.21.15.33:0/3565091759 learned_addr learned my addr 172.21.15.33:0/3565091759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5aff90 0x56207d5b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc572ffd640 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 -- 172.21.15.33:0/3565091759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 msgr2=0x56207d5ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5ba0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 -- 172.21.15.33:0/3565091759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5aff90 msgr2=0x56207d5b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5aff90 0x56207d5b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 -- 172.21.15.33:0/3565091759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207d5bb490 con 0x56207d5b1160 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.920+0000 7fc5737fe640 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc56400bc20 tx=0x7fc564005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdd71a8292d7a10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc5597fa640 1 -- 172.21.15.33:0/3565091759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564013c90 con 0x56207d5b1160 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc5597fa640 1 -- 172.21.15.33:0/3565091759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56401e030 con 0x56207d5b1160 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc5597fa640 1 -- 172.21.15.33:0/3565091759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc564011a20 con 0x56207d5b1160 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- 172.21.15.33:0/3565091759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 msgr2=0x56207d5c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc56400bc20 tx=0x7fc564005c50 comp rx=0 tx=0).stop 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- 172.21.15.33:0/3565091759 shutdown_connections 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5ba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5b1160 0x56207d5c19e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 --2- 172.21.15.33:0/3565091759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5aff90 0x56207d5b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- 172.21.15.33:0/3565091759 >> 172.21.15.33:0/3565091759 conn(0x56207d5a7070 msgr2=0x56207d5a20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- 172.21.15.33:0/3565091759 shutdown_connections 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- 172.21.15.33:0/3565091759 wait complete. 2024-04-23T18:25:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.921+0000 7fc57d011740 1 -- start start 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 Processor -- start 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1160 0x56207d5b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5c00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c8250 con 0x56207d5aff90 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c7b80 con 0x56207d5b1160 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56207d5c8920 con 0x56207d5b1b20 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc57d011740 10 client.0 _ll_register_callbacks cb 0x7fc568009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33266/0 (socket says 172.21.15.33:33266) 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 -- 172.21.15.33:0/1509341936 learned_addr learned my addr 172.21.15.33:0/1509341936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc5737fe640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1160 0x56207d5b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5c00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 -- 172.21.15.33:0/1509341936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 msgr2=0x56207d5c00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5c00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 -- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1160 msgr2=0x56207d5b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1160 0x56207d5b73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.922+0000 7fc572ffd640 1 -- 172.21.15.33:0/1509341936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207d5c06c0 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc572ffd640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc568009590 tx=0x7fc5680056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7888b108ca5016e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568064060 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fc5680315a0 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc56401e030 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc568035040 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56207d5c19c0 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56401e030 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fc568069070 con 0x56207d5aff90 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:25:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 20 client.15117 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d631500 0x56207d630210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 4 client.15117 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.923+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "status"}) -- 0x56207d5184e0 con 0x56207d631500 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc5737fe640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d631500 0x56207d630210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc552ffd640 10 client.15117 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc5737fe640 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d631500 0x56207d630210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4206826c6ad284b server_cookie=8648e6481eebbe19 in_seq=0 out_seq=0 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc552ffd640 1 -- 172.21.15.33:0/1509341936 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56207d5184e0 con 0x56207d631500 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc552ffd640 10 client.15117 handle_command_reply: tid=0 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 client.15117 shutdown 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 msgr2=0x56207d5b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc568009590 tx=0x7fc5680056a0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 shutdown_connections 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x56207d631500 0x56207d630210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56207d5b1b20 0x56207d5c00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc552ffd640 0 client.15117 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56207d5b1160 0x56207d5b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 --2- 172.21.15.33:0/1509341936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d5aff90 0x56207d5b6eb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 >> 172.21.15.33:0/1509341936 conn(0x56207d5a7070 msgr2=0x56207d5a9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.924+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 shutdown_connections 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57d011740 1 -- 172.21.15.33:0/1509341936 wait complete. 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57d011740 20 client.15117 trim_cache size 0 max 16384 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24515, 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:25:55.923 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 85.609245002999998, 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 104.411858841 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:25:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 msgr2=0x7fc55c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 0x7fc55c0a60f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc56c099030 tx=0x7fc56c06f710 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 msgr2=0x7fc57419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00efd0 tx=0x7fc56c00c6a0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 shutdown_connections 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fc55c0a3c30 0x7fc55c0a60f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc574102a00 0x7fc57419f7f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc574102040 0x7fc574198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 --2- 172.21.15.33:0/142991686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc574100e60 0x7fc574198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 >> 172.21.15.33:0/142991686 conn(0x7fc5740fc440 msgr2=0x7fc5740fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 shutdown_connections 2024-04-23T18:25:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:55.925+0000 7fc57a223640 1 -- 172.21.15.33:0/142991686 wait complete. 2024-04-23T18:25:55.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 -- 172.21.15.33:0/3537611804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07581031b0 msgr2=0x7f0758103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 --2- 172.21.15.33:0/3537611804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07581031b0 0x7f0758103630 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f074c009900 tx=0x7f074c02a410 comp rx=0 tx=0).stop 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 -- 172.21.15.33:0/3537611804 shutdown_connections 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 --2- 172.21.15.33:0/3537611804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758103b70 0x7f075810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 --2- 172.21.15.33:0/3537611804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07581031b0 0x7f0758103630 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 --2- 172.21.15.33:0/3537611804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0758101fc0 0x7f07581023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 -- 172.21.15.33:0/3537611804 >> 172.21.15.33:0/3537611804 conn(0x7f07580fd530 msgr2=0x7f07580ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 -- 172.21.15.33:0/3537611804 shutdown_connections 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.039+0000 7f075eff7640 1 -- 172.21.15.33:0/3537611804 wait complete. 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 Processor -- start 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 -- start start 2024-04-23T18:25:56.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758101fc0 0x7f0758198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07581031b0 0x7f07581988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f075810dbe0 con 0x7f0758103b70 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f075810da60 con 0x7f0758101fc0 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075eff7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f075810dd60 con 0x7f07581031b0 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33278/0 (socket says 172.21.15.33:33278) 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 -- 172.21.15.33:0/1874082312 learned_addr learned my addr 172.21.15.33:0/1874082312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f0757fff640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07581031b0 0x7f07581988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075cd93640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758101fc0 0x7f0758198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 -- 172.21.15.33:0/1874082312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07581031b0 msgr2=0x7f07581988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07581031b0 0x7f07581988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 -- 172.21.15.33:0/1874082312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758101fc0 msgr2=0x7f0758198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758101fc0 0x7f0758198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.040+0000 7f075d594640 1 -- 172.21.15.33:0/1874082312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075819ff80 con 0x7f0758103b70 2024-04-23T18:25:56.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f075d594640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f075000b970 tx=0x7f075000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f0755ffb640 1 -- 172.21.15.33:0/1874082312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750019070 con 0x7f0758103b70 2024-04-23T18:25:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f0755ffb640 1 -- 172.21.15.33:0/1874082312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0750014440 con 0x7f0758103b70 2024-04-23T18:25:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581a0270 con 0x7f0758103b70 2024-04-23T18:25:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f0755ffb640 1 -- 172.21.15.33:0/1874082312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500075d0 con 0x7f0758103b70 2024-04-23T18:25:56.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.041+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a0810 con 0x7f0758103b70 2024-04-23T18:25:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.043+0000 7f0755ffb640 1 -- 172.21.15.33:0/1874082312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f075001c070 con 0x7f0758103b70 2024-04-23T18:25:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.043+0000 7f0761de2740 1 Processor -- start 2024-04-23T18:25:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.043+0000 7f0761de2740 1 -- start start 2024-04-23T18:25:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.043+0000 7f0755ffb640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 0x7f07400a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4affa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e40cc50 con 0x55fb9e4a8d50 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e3a9830 con 0x55fb9e4a7b80 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0761de2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e408c10 con 0x55fb9e4a9710 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0757fff640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 0x7f07400a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0755ffb640 1 -- 172.21.15.33:0/1874082312 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f07500cf920 con 0x7f0758103b70 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0757fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:60196/0 (socket says 172.21.15.33:60196) 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 -- 172.21.15.33:0/1553431192 learned_addr learned my addr 172.21.15.33:0/1553431192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075d594640 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 -- 172.21.15.33:0/1553431192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 msgr2=0x55fb9e4affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 -- 172.21.15.33:0/1553431192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 msgr2=0x55fb9e4a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 -- 172.21.15.33:0/1553431192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9e11f2d0 con 0x55fb9e4a7b80 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f0757fff640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 0x7f07400a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f074c000c00 tx=0x7f074c032c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f075cd93640 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f074800a8e0 tx=0x7f074800ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71cb54e8f13b8352 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.044+0000 7f073effd640 1 -- 172.21.15.33:0/1553431192 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748015070 con 0x55fb9e4a7b80 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f073effd640 1 -- 172.21.15.33:0/1553431192 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07480113f0 con 0x55fb9e4a7b80 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f073effd640 1 -- 172.21.15.33:0/1553431192 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07480105b0 con 0x55fb9e4a7b80 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- 172.21.15.33:0/1553431192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 msgr2=0x55fb9e4a7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f074800a8e0 tx=0x7f074800ada0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- 172.21.15.33:0/1553431192 shutdown_connections 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 --2- 172.21.15.33:0/1553431192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a7fa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- 172.21.15.33:0/1553431192 >> 172.21.15.33:0/1553431192 conn(0x55fb9e4a3070 msgr2=0x55fb9e4a3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- 172.21.15.33:0/1553431192 shutdown_connections 2024-04-23T18:25:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- 172.21.15.33:0/1553431192 wait complete. 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.045+0000 7f0761de2740 1 -- start start 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 Processor -- start 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4b0720 con 0x55fb9e4a8d50 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e3a7470 con 0x55fb9e4a7b80 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4b08a0 con 0x55fb9e4a9710 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0761de2740 10 client.0 _ll_register_callbacks cb 0x55fb9e3ccf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33300/0 (socket says 172.21.15.33:33300) 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 -- 172.21.15.33:0/3925093043 learned_addr learned my addr 172.21.15.33:0/3925093043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f075cd93640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f075d594640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 -- 172.21.15.33:0/3925093043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 msgr2=0x55fb9e4b94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4b94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 -- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 msgr2=0x55fb9e4b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.046+0000 7f0757fff640 1 -- 172.21.15.33:0/3925093043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9e1e96b0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0757fff640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f074c005860 tx=0x7f074c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb4545a134d1df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c04f070 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f074c031d20 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07480113f0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f074c05b050 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb9e4bc8a0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07480113f0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f074c049040 con 0x55fb9e4a8d50 2024-04-23T18:25:56.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 10 client.15129 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 20 client.15129 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e4bef60 0x55fb9e4c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 4 client.15129 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.047+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb9e41a670 con 0x55fb9e4bef60 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.048+0000 7f075cd93640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e4bef60 0x55fb9e4c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.048+0000 7f075cd93640 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e4bef60 0x55fb9e4c03d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b137121b6832116b server_cookie=a93680944d8fa18a in_seq=0 out_seq=0 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.048+0000 7f0737fff640 10 client.15129 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0737fff640 1 -- 172.21.15.33:0/3925093043 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb9e41a670 con 0x55fb9e4bef60 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0737fff640 10 client.15129 handle_command_reply: tid=0 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 client.15129 shutdown 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 msgr2=0x55fb9e4b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f074c005860 tx=0x7f074c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 shutdown_connections 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e4bef60 0x55fb9e4c03d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a9710 0x55fb9e4b94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0737fff640 0 client.15129 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4b2480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 --2- 172.21.15.33:0/3925093043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 >> 172.21.15.33:0/3925093043 conn(0x55fb9e4a3070 msgr2=0x55fb9e4a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 shutdown_connections 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 1 -- 172.21.15.33:0/3925093043 wait complete. 2024-04-23T18:25:56.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.049+0000 7f0761de2740 20 client.15129 trim_cache size 0 max 16384 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 Processor -- start 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 -- start start 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4be710 con 0x55fb9e4a9710 2024-04-23T18:25:56.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4bede0 con 0x55fb9e4a7b80 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f0761de2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4bf4b0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.067+0000 7f075d594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33302/0 (socket says 172.21.15.33:33302) 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 -- 172.21.15.33:0/2961545570 learned_addr learned my addr 172.21.15.33:0/2961545570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f0757fff640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075cd93640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 -- 172.21.15.33:0/2961545570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a8d50 msgr2=0x55fb9e4a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 -- 172.21.15.33:0/2961545570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 msgr2=0x55fb9e4a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 -- 172.21.15.33:0/2961545570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb9e4a87d0 con 0x55fb9e4a9710 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075cd93640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f075d594640 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f075009bbf0 tx=0x7f07500bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e92e394812458 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f073e7fc640 1 -- 172.21.15.33:0/2961545570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500b5070 con 0x55fb9e4a9710 2024-04-23T18:25:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f073e7fc640 1 -- 172.21.15.33:0/2961545570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f075009ffb0 con 0x55fb9e4a9710 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.068+0000 7f073e7fc640 1 -- 172.21.15.33:0/2961545570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750098320 con 0x55fb9e4a9710 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- 172.21.15.33:0/2961545570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 msgr2=0x55fb9e4a23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f075009bbf0 tx=0x7f07500bc040 comp rx=0 tx=0).stop 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- 172.21.15.33:0/2961545570 shutdown_connections 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a9710 0x55fb9e4a23c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a8d50 0x55fb9e4a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- 172.21.15.33:0/2961545570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a7b80 0x55fb9e4a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- 172.21.15.33:0/2961545570 >> 172.21.15.33:0/2961545570 conn(0x55fb9e4a3070 msgr2=0x55fb9e4ac750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- 172.21.15.33:0/2961545570 shutdown_connections 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- 172.21.15.33:0/2961545570 wait complete. 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- start start 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 Processor -- start 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 0x55fb9e521340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 0x55fb9e528890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4c0250 con 0x55fb9e4a8d50 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4bfb80 con 0x55fb9e4a9710 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fb9e4c0920 con 0x55fb9e4a7b80 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0761de2740 10 client.0 _ll_register_callbacks cb 0x7f074c0057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0757fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.069+0000 7f0757fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33306/0 (socket says 172.21.15.33:33306) 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 -- 172.21.15.33:0/1293954674 learned_addr learned my addr 172.21.15.33:0/1293954674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f075cd93640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 0x55fb9e521340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f075d594640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 0x55fb9e528890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 -- 172.21.15.33:0/1293954674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 msgr2=0x55fb9e521340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 0x55fb9e521340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 -- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 msgr2=0x55fb9e528890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 0x55fb9e528890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 -- 172.21.15.33:0/1293954674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb9e528e80 con 0x55fb9e4a8d50 2024-04-23T18:25:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f075d594640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 0x55fb9e528890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f075cd93640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 0x55fb9e521340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0757fff640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f074c004650 tx=0x7f074c0327c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13dcc29bb59f9ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0737fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074c058030 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f074c031f30 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f075009ffb0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f074c04f070 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0761de2740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fb9e52a0f0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.070+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f075009ffb0 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f074c003b30 con 0x55fb9e4a8d50 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 20 client.15135 populate_metadata read hostname 'smithi033' 2024-04-23T18:25:56.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e52ecf0 0x55fb9e52da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 4 client.15135 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "status"}) -- 0x55fb9e4144e0 con 0x55fb9e52ecf0 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f075cd93640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e52ecf0 0x55fb9e52da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0737fff640 10 client.15135 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f075cd93640 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e52ecf0 0x55fb9e52da00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247a0564fc09ed9 server_cookie=9c3eb73941e5ceef in_seq=0 out_seq=0 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0737fff640 1 -- 172.21.15.33:0/1293954674 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fb9e4144e0 con 0x55fb9e52ecf0 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.071+0000 7f0737fff640 10 client.15135 handle_command_reply: tid=0 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 client.15135 shutdown 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 msgr2=0x55fb9e521880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f074c004650 tx=0x7f074c0327c0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 shutdown_connections 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fb9e52ecf0 0x55fb9e52da00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb9e4a9710 0x55fb9e528890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0737fff640 0 client.15135 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:25:56.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb9e4a8d50 0x55fb9e521880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 --2- 172.21.15.33:0/1293954674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fb9e4a7b80 0x55fb9e521340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 >> 172.21.15.33:0/1293954674 conn(0x55fb9e4a3070 msgr2=0x55fb9e4a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 shutdown_connections 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 1 -- 172.21.15.33:0/1293954674 wait complete. 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f0761de2740 20 client.15135 trim_cache size 0 max 16384 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:25:56.071 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 88.790282723999994, 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 92.919400053000004 2024-04-23T18:25:56.072 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 msgr2=0x7f07400a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 0x7f07400a60f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f074c000c00 tx=0x7f074c032c00 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 msgr2=0x7f075819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f075000b970 tx=0x7f075000be30 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.072+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 shutdown_connections 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f07400a3c30 0x7f07400a60f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0758103b70 0x7f075819f910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07581031b0 0x7f07581988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 --2- 172.21.15.33:0/1874082312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0758101fc0 0x7f0758198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 >> 172.21.15.33:0/1874082312 conn(0x7f07580fd530 msgr2=0x7f07581065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 shutdown_connections 2024-04-23T18:25:56.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:56.073+0000 7f075eff7640 1 -- 172.21.15.33:0/1874082312 wait complete. 2024-04-23T18:25:56.076 INFO:tasks.cephadm:Setup complete, yielding 2024-04-23T18:25:56.077 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:25:56.086 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:25:56.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-23T18:25:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:55 smithi033 ceph-mon[29274]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:55 smithi033 ceph-mon[29274]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:55 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3845079489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:56.331 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:25:56.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:55 smithi106 ceph-mon[32716]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:56.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:55 smithi106 ceph-mon[32716]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:56.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:55 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3845079489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:56.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:55 smithi122 ceph-mon[32546]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:25:56.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:55 smithi122 ceph-mon[32546]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-23T18:25:56.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:55 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3845079489' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 -- 172.21.15.33:0/102825434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 msgr2=0x7f308c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 --2- 172.21.15.33:0/102825434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c10ac20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f308400b0a0 tx=0x7f308400b3b0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 -- 172.21.15.33:0/102825434 shutdown_connections 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 --2- 172.21.15.33:0/102825434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c10ac20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 --2- 172.21.15.33:0/102825434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c1035b0 0x7f308c1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 --2- 172.21.15.33:0/102825434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c105ee0 0x7f308c1082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 -- 172.21.15.33:0/102825434 >> 172.21.15.33:0/102825434 conn(0x7f308c0fcf60 msgr2=0x7f308c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 -- 172.21.15.33:0/102825434 shutdown_connections 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.934+0000 7f30923ad700 1 -- 172.21.15.33:0/102825434 wait complete. 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 Processor -- start 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 -- start start 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 0x7f308c102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c105ee0 0x7f308c101560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f308c10e520 con 0x7f308c108810 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f308c10e3a0 con 0x7f308c1035b0 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f30923ad700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f308c10e6a0 con 0x7f308c105ee0 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34202/0 (socket says 172.21.15.33:34202) 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 -- 172.21.15.33:0/782854295 learned_addr learned my addr 172.21.15.33:0/782854295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:25:57.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f308b7fe700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c105ee0 0x7f308c101560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f308bfff700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 0x7f308c102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 -- 172.21.15.33:0/782854295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c105ee0 msgr2=0x7f308c101560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c105ee0 0x7f308c101560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 -- 172.21.15.33:0/782854295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 msgr2=0x7f308c102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 0x7f308c102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f309094a700 1 -- 172.21.15.33:0/782854295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c102350 con 0x7f308c108810 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.935+0000 7f308bfff700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 0x7f308c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f309094a700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f308400b070 tx=0x7f308400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084021070 con 0x7f308c108810 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3084007d00 con 0x7f308c108810 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c1a2c50 con 0x7f308c108810 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308401d450 con 0x7f308c108810 2024-04-23T18:25:57.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.936+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1a3110 con 0x7f308c108810 2024-04-23T18:25:57.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.937+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c069730 con 0x7f308c108810 2024-04-23T18:25:57.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.938+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3084003a00 con 0x7f308c108810 2024-04-23T18:25:57.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.939+0000 7f30897fa700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 0x7f30780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:25:57.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.939+0000 7f308bfff700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 0x7f30780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:25:57.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.939+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f30840d1940 con 0x7f308c108810 2024-04-23T18:25:57.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.939+0000 7f308bfff700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 0x7f30780a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3074005950 tx=0x7f3074005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:25:57.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:57.944+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f308409e370 con 0x7f308c108810 2024-04-23T18:25:58.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.169+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f308c062e80 con 0x7f30780a3c40 2024-04-23T18:25:58.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.171+0000 7f30897fa700 1 -- 172.21.15.33:0/782854295 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f308c062e80 con 0x7f30780a3c40 2024-04-23T18:25:58.163 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-04-23T18:25:58.163 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-04-23T18:25:58.164 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-04-23T18:25:58.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.172+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 msgr2=0x7f30780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.172+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 0x7f30780a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3074005950 tx=0x7f3074005f50 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.172+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 msgr2=0x7f308c101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.172+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f308400b070 tx=0x7f308400ced0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 shutdown_connections 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f30780a3c40 0x7f30780a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f308c108810 0x7f308c101aa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f308c1035b0 0x7f308c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 --2- 172.21.15.33:0/782854295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f308c105ee0 0x7f308c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 >> 172.21.15.33:0/782854295 conn(0x7f308c0fcf60 msgr2=0x7f308c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 shutdown_connections 2024-04-23T18:25:58.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:25:58.173+0000 7f30923ad700 1 -- 172.21.15.33:0/782854295 wait complete. 2024-04-23T18:25:58.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:57 smithi106 ceph-mon[32716]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:58.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:57 smithi122 ceph-mon[32546]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:58.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:57 smithi033 ceph-mon[29274]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-23T18:25:58.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-23T18:25:59.120 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:00.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:59 smithi033 ceph-mon[29274]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:00.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:25:59 smithi033 ceph-mon[29274]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:00.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:59 smithi106 ceph-mon[32716]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:00.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:25:59 smithi106 ceph-mon[32716]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:00.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:59 smithi122 ceph-mon[32546]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:00.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:25:59 smithi122 ceph-mon[32546]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 -- 172.21.15.33:0/3323509054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c102aa0 msgr2=0x7f727c102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 --2- 172.21.15.33:0/3323509054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c102aa0 0x7f727c102ec0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f726c0099c0 tx=0x7f726c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 -- 172.21.15.33:0/3323509054 shutdown_connections 2024-04-23T18:26:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 --2- 172.21.15.33:0/3323509054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c104640 0x7f727c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 --2- 172.21.15.33:0/3323509054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c102aa0 0x7f727c102ec0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 --2- 172.21.15.33:0/3323509054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c103ca0 0x7f727c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 -- 172.21.15.33:0/3323509054 >> 172.21.15.33:0/3323509054 conn(0x7f727c0fe040 msgr2=0x7f727c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 -- 172.21.15.33:0/3323509054 shutdown_connections 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.900+0000 7f7281884700 1 -- 172.21.15.33:0/3323509054 wait complete. 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 Processor -- start 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 -- start start 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 0x7f727c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c103ca0 0x7f727c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 0x7f727c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10e590 con 0x7f727c104640 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10e410 con 0x7f727c102aa0 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.901+0000 7f7281884700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f727c10e710 con 0x7f727c103ca0 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727affd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 0x7f727c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 0x7f727c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727affd700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 0x7f727c0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:33714/0 (socket says 172.21.15.33:33714) 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727affd700 1 -- 172.21.15.33:0/642020563 learned_addr learned my addr 172.21.15.33:0/642020563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727a7fc700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c103ca0 0x7f727c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 -- 172.21.15.33:0/642020563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c103ca0 msgr2=0x7f727c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c103ca0 0x7f727c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 -- 172.21.15.33:0/642020563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 msgr2=0x7f727c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 0x7f727c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 -- 172.21.15.33:0/642020563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c075c20 con 0x7f727c104640 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f727b7fe700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 0x7f727c075370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7274007f10 tx=0x7f727400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274004db0 con 0x7f727c104640 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7274009e20 con 0x7f727c104640 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1a2e10 con 0x7f727c104640 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274015400 con 0x7f727c104640 2024-04-23T18:26:00.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.902+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a3270 con 0x7f727c104640 2024-04-23T18:26:00.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.904+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c069730 con 0x7f727c104640 2024-04-23T18:26:00.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.905+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7274005260 con 0x7f727c104640 2024-04-23T18:26:00.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.906+0000 7f7280882700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 0x7f72680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:00.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.906+0000 7f727affd700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 0x7f72680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:00.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.906+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f72740d0150 con 0x7f727c104640 2024-04-23T18:26:00.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.906+0000 7f727affd700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 0x7f72680a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f726c005cd0 tx=0x7f726c005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:00.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:00.909+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7274010070 con 0x7f727c104640 2024-04-23T18:26:01.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.127+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f727c062e80 con 0x7f72680a3cc0 2024-04-23T18:26:01.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.141+0000 7f7280882700 1 -- 172.21.15.33:0/642020563 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f727c062e80 con 0x7f72680a3cc0 2024-04-23T18:26:01.134 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-23T18:26:01.134 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (101s) 16s ago 100s 16.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8db0770a5d17 2024-04-23T18:26:01.134 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi106 running (98s) 31s ago 97s 28.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7bea2d1eab25 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi122 running (112s) 31s ago 112s 19.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3af0d6340c40 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (2m) 16s ago 2m 14.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 76a9367cdeb3 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi106 running (118s) 31s ago 118s 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 be4874427eaa 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi122 running (115s) 31s ago 115s 20.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 bfb12c9012e2 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (104s) 16s ago 103s 17.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0a7757587474 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi106 running (110s) 31s ago 109s 18.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a20af9def49a 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi122 running (107s) 31s ago 106s 17.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0dfa79d04a53 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (2m) 16s ago 2m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 66762eb185a3 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi106 running (2m) 31s ago 2m 18.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 00784d7ce481 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi122 running (2m) 31s ago 2m 16.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1b6ba1fc8d8f 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (9m) 16s ago 9m 514M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d7656f9fb004 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi106 *:8443,8765 running (7m) 31s ago 7m 455M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8f55751ed7ec 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi122 *:8443,8765 running (7m) 31s ago 7m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 313fc5dbb8cf 2024-04-23T18:26:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (9m) 16s ago 9m 53.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 5bf33b2d8c0e 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi106 running (7m) 31s ago 7m 38.3M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 7c4223f6e9fe 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi122 running (7m) 31s ago 7m 39.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 3aafede04df2 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 16s ago 7m 56.6M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 90820a1016aa 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi106 running (6m) 31s ago 6m 57.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 872ac92dbdf4 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi122 running (6m) 31s ago 6m 56.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c96e93307e4e 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (5m) 16s ago 5m 57.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 64d95280fadc 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi106 running (5m) 31s ago 5m 44.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 75319b60c8f6 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi122 running (5m) 31s ago 5m 44.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ff11a459760e 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (4m) 16s ago 4m 52.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 3a3df1f940d8 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi106 running (4m) 31s ago 4m 54.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 eeace3722f09 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi122 running (3m) 31s ago 3m 45.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4dd55a98d46a 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (3m) 16s ago 3m 51.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 a33438cddff8 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi106 running (2m) 31s ago 2m 55.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 563b95a28b39 2024-04-23T18:26:01.136 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi122 running (2m) 31s ago 2m 51.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5d9fca4a1c4c 2024-04-23T18:26:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 msgr2=0x7f72680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 0x7f72680a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f726c005cd0 tx=0x7f726c005c00 comp rx=0 tx=0).stop 2024-04-23T18:26:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 msgr2=0x7f727c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 0x7f727c075370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7274007f10 tx=0x7f727400d3b0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 shutdown_connections 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f72680a3cc0 0x7f72680a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727c104640 0x7f727c075370 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f727c102aa0 0x7f727c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 --2- 172.21.15.33:0/642020563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f727c103ca0 0x7f727c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.143+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 >> 172.21.15.33:0/642020563 conn(0x7f727c0fe040 msgr2=0x7f727c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.144+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 shutdown_connections 2024-04-23T18:26:01.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:01.144+0000 7f7281884700 1 -- 172.21.15.33:0/642020563 wait complete. 2024-04-23T18:26:02.073 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-23T18:26:02.331 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:02.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:01 smithi106 ceph-mon[32716]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:26:02.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:01 smithi122 ceph-mon[32546]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:26:02.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:01 smithi033 ceph-mon[29274]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T18:26:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:03 smithi106 ceph-mon[32716]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:03.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:03 smithi122 ceph-mon[32546]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:03.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:03 smithi033 ceph-mon[29274]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:04.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:04 smithi033 ceph-mon[29274]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 -- 172.21.15.33:0/2421646231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 msgr2=0x7f2e00103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2421646231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e00103020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2df40099c0 tx=0x7f2df4009cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 -- 172.21.15.33:0/2421646231 shutdown_connections 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2421646231 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 0x7f2e00109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2421646231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e00101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2421646231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e00103020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 -- 172.21.15.33:0/2421646231 >> 172.21.15.33:0/2421646231 conn(0x7f2e000fcf60 msgr2=0x7f2e000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 -- 172.21.15.33:0/2421646231 shutdown_connections 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.152+0000 7f2e065e1700 1 -- 172.21.15.33:0/2421646231 wait complete. 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 Processor -- start 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 -- start start 2024-04-23T18:26:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e0006bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 0x7f2e0019ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d460 con 0x7f2e00102bc0 2024-04-23T18:26:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d2e0 con 0x7f2e001019c0 2024-04-23T18:26:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e065e1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010d5e0 con 0x7f2e00103560 2024-04-23T18:26:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:04.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34242/0 (socket says 172.21.15.33:34242) 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dff7fe700 1 -- 172.21.15.33:0/2326822007 learned_addr learned my addr 172.21.15.33:0/2326822007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2e04b7e700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 0x7f2e0019ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dfffff700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e0006bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dff7fe700 1 -- 172.21.15.33:0/2326822007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 msgr2=0x7f2e0019ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.153+0000 7f2dff7fe700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 0x7f2e0019ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dff7fe700 1 -- 172.21.15.33:0/2326822007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 msgr2=0x7f2e0006bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dff7fe700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e0006bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dff7fe700 1 -- 172.21.15.33:0/2326822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e001a0870 con 0x7f2e00102bc0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dfffff700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e0006bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dff7fe700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2df4003790 tx=0x7f2df4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4027070 con 0x7f2e00102bc0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df4004160 con 0x7f2e00102bc0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e001a0b00 con 0x7f2e00102bc0 2024-04-23T18:26:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4022d90 con 0x7f2e00102bc0 2024-04-23T18:26:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.154+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001a0fe0 con 0x7f2e00102bc0 2024-04-23T18:26:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.155+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e00069730 con 0x7f2e00102bc0 2024-04-23T18:26:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.156+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2df402a070 con 0x7f2e00102bc0 2024-04-23T18:26:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.157+0000 7f2dfd7fa700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 0x7f2de80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.157+0000 7f2dfffff700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 0x7f2de80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.157+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f2df40d13c0 con 0x7f2e00102bc0 2024-04-23T18:26:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.157+0000 7f2dfffff700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 0x7f2de80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e00102a20 tx=0x7f2df0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:04.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.161+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2df4013070 con 0x7f2e00102bc0 2024-04-23T18:26:04.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:04 smithi106 ceph-mon[32716]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.379+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2e00062f40 con 0x7f2de80a3c40 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.385+0000 7f2dfd7fa700 1 -- 172.21.15.33:0/2326822007 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2e00062f40 con 0x7f2de80a3c40 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 34s ago 2m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 34s ago 7m smithi033=x;smithi106=y;smithi122=z;count:3 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 34s ago 8m smithi033:172.21.15.33=a;smithi106:172.21.15.106=b;smithi122:172.21.15.122=c;count:3 2024-04-23T18:26:04.378 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 34s ago - 2024-04-23T18:26:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 msgr2=0x7f2de80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 0x7f2de80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2e00102a20 tx=0x7f2df0009670 comp rx=0 tx=0).stop 2024-04-23T18:26:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 msgr2=0x7f2e0019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2df4003790 tx=0x7f2df4005b00 comp rx=0 tx=0).stop 2024-04-23T18:26:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 shutdown_connections 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f2de80a3c40 0x7f2de80a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e00103560 0x7f2e0019ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e001019c0 0x7f2e0006bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 --2- 172.21.15.33:0/2326822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e00102bc0 0x7f2e0019d960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 >> 172.21.15.33:0/2326822007 conn(0x7f2e000fcf60 msgr2=0x7f2e00105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 shutdown_connections 2024-04-23T18:26:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:04.387+0000 7f2e065e1700 1 -- 172.21.15.33:0/2326822007 wait complete. 2024-04-23T18:26:04.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:04 smithi122 ceph-mon[32546]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-23T18:26:05.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-23T18:26:05.375 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:06.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:06 smithi033 ceph-mon[29274]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:06.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:06 smithi033 ceph-mon[29274]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:06.251 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:06 smithi122 ceph-mon[32546]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:06.251 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:06 smithi122 ceph-mon[32546]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:06.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:06 smithi106 ceph-mon[32716]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:06.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:06 smithi106 ceph-mon[32716]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 -- 172.21.15.33:0/602337959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228102ad0 msgr2=0x7f9228102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 --2- 172.21.15.33:0/602337959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228102ad0 0x7f9228102ef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f921c0099c0 tx=0x7f921c009cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 -- 172.21.15.33:0/602337959 shutdown_connections 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 --2- 172.21.15.33:0/602337959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228104670 0x7f922810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 --2- 172.21.15.33:0/602337959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228102ad0 0x7f9228102ef0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 --2- 172.21.15.33:0/602337959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228103cd0 0x7f9228104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 -- 172.21.15.33:0/602337959 >> 172.21.15.33:0/602337959 conn(0x7f92280fe090 msgr2=0x7f92281004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 -- 172.21.15.33:0/602337959 shutdown_connections 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.150+0000 7f922fda4700 1 -- 172.21.15.33:0/602337959 wait complete. 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 Processor -- start 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 -- start start 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228102ad0 0x7f9228198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 0x7f92281991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f922810e5c0 con 0x7f9228104670 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f922810e440 con 0x7f9228102ad0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922fda4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f922810e740 con 0x7f9228103cd0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54910/0 (socket says 172.21.15.33:54910) 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 -- 172.21.15.33:0/2248118569 learned_addr learned my addr 172.21.15.33:0/2248118569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922db40700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228102ad0 0x7f9228198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922d33f700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 0x7f92281991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 -- 172.21.15.33:0/2248118569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 msgr2=0x7f92281991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 0x7f92281991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 -- 172.21.15.33:0/2248118569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228102ad0 msgr2=0x7f9228198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228102ad0 0x7f9228198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 -- 172.21.15.33:0/2248118569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92281a08e0 con 0x7f9228104670 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922d33f700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 0x7f92281991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.151+0000 7f922e341700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f922000b970 tx=0x7f922000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.152+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220019070 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.152+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92281a0bd0 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.152+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f922000cde0 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.152+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220007370 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.152+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92281a1090 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.153+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9228069730 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.154+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f922001c070 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.154+0000 7f921affd700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 0x7f92140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.154+0000 7f922db40700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 0x7f92140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.154+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f92200cffa0 con 0x7f9228104670 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.155+0000 7f922db40700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 0x7f92140a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f921c0038c0 tx=0x7f921c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:07.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.158+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9220023070 con 0x7f9228104670 2024-04-23T18:26:07.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.373+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9228062e80 con 0x7f92140a3c40 2024-04-23T18:26:07.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.375+0000 7f921affd700 1 -- 172.21.15.33:0/2248118569 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9228062e80 con 0x7f92140a3c40 2024-04-23T18:26:07.368 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-04-23T18:26:07.368 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-04-23T18:26:07.368 INFO:teuthology.orchestra.run.smithi033.stdout:smithi106 172.21.15.106 2024-04-23T18:26:07.368 INFO:teuthology.orchestra.run.smithi033.stdout:smithi122 172.21.15.122 2024-04-23T18:26:07.369 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-04-23T18:26:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 msgr2=0x7f92140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 0x7f92140a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f921c0038c0 tx=0x7f921c00b540 comp rx=0 tx=0).stop 2024-04-23T18:26:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 msgr2=0x7f92281a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f922000b970 tx=0x7f922000bc80 comp rx=0 tx=0).stop 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 shutdown_connections 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f92140a3c40 0x7f92140a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9228104670 0x7f92281a0210 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9228102ad0 0x7f9228198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 --2- 172.21.15.33:0/2248118569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9228103cd0 0x7f92281991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 >> 172.21.15.33:0/2248118569 conn(0x7f92280fe090 msgr2=0x7f92281070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:07.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 shutdown_connections 2024-04-23T18:26:07.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:07.377+0000 7f922fda4700 1 -- 172.21.15.33:0/2248118569 wait complete. 2024-04-23T18:26:08.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-23T18:26:08.283 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:08.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:08 smithi106 ceph-mon[32716]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:08.405 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:08 smithi033 ceph-mon[29274]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:08.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:08 smithi122 ceph-mon[32546]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-04-23T18:26:09.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:09 smithi106 ceph-mon[32716]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:09.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:09 smithi033 ceph-mon[29274]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:09.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:09 smithi122 ceph-mon[32546]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 -- 172.21.15.33:0/1526860828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 msgr2=0x7f53c0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 --2- 172.21.15.33:0/1526860828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c0109df0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f53b800b0d0 tx=0x7f53b800b3e0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 -- 172.21.15.33:0/1526860828 shutdown_connections 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 --2- 172.21.15.33:0/1526860828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c0109df0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 --2- 172.21.15.33:0/1526860828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 0x7f53c0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 --2- 172.21.15.33:0/1526860828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 0x7f53c0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 -- 172.21.15.33:0/1526860828 >> 172.21.15.33:0/1526860828 conn(0x7f53c00fcf40 msgr2=0x7f53c00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 -- 172.21.15.33:0/1526860828 shutdown_connections 2024-04-23T18:26:09.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.998+0000 7f53c6755700 1 -- 172.21.15.33:0/1526860828 wait complete. 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 Processor -- start 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 -- start start 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 0x7f53c006b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 0x7f53c019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f53c010d420 con 0x7f53c0103540 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f53c010d2a0 con 0x7f53c01019a0 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c6755700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53c010d5a0 con 0x7f53c0102ba0 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54924/0 (socket says 172.21.15.33:54924) 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 -- 172.21.15.33:0/2956450555 learned_addr learned my addr 172.21.15.33:0/2956450555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53b7fff700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 0x7f53c019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53bffff700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 0x7f53c006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 -- 172.21.15.33:0/2956450555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 msgr2=0x7f53c019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 0x7f53c019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 -- 172.21.15.33:0/2956450555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 msgr2=0x7f53c006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:09.999+0000 7f53c4cf2700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 0x7f53c006b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53c4cf2700 1 -- 172.21.15.33:0/2956450555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c01a07d0 con 0x7f53c0103540 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53c4cf2700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d010 tx=0x7f53b8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8021070 con 0x7f53c0103540 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c01a0a60 con 0x7f53c0103540 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b8007e50 con 0x7f53c0103540 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b801bb00 con 0x7f53c0103540 2024-04-23T18:26:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.000+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c01a0f60 con 0x7f53c0103540 2024-04-23T18:26:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.001+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a0005360 con 0x7f53c0103540 2024-04-23T18:26:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.003+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f53b8024070 con 0x7f53c0103540 2024-04-23T18:26:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.003+0000 7f53bdffb700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 0x7f53ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.004+0000 7f53bffff700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 0x7f53ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.004+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f53b800f020 con 0x7f53c0103540 2024-04-23T18:26:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.004+0000 7f53bffff700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 0x7f53ac0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53c0102a00 tx=0x7f53b000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:10.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.008+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53b802b020 con 0x7f53c0103540 2024-04-23T18:26:10.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.224+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f53a0000bf0 con 0x7f53ac0a3c40 2024-04-23T18:26:10.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.226+0000 7f53bdffb700 1 -- 172.21.15.33:0/2956450555 <== mgr.14152 v2:172.21.15.33:6800/2158874591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f53a0000bf0 con 0x7f53ac0a3c40 2024-04-23T18:26:10.218 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-23T18:26:10.219 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-23T18:26:10.219 INFO:teuthology.orchestra.run.smithi033.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:26:10.219 INFO:teuthology.orchestra.run.smithi033.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 msgr2=0x7f53ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 0x7f53ac0a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53c0102a00 tx=0x7f53b000a3b0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 msgr2=0x7f53c019ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f53b800d010 tx=0x7f53b8007a20 comp rx=0 tx=0).stop 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 shutdown_connections 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f53ac0a3c40 0x7f53ac0a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53c0103540 0x7f53c019ff00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53c01019a0 0x7f53c006b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 --2- 172.21.15.33:0/2956450555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f53c0102ba0 0x7f53c019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 >> 172.21.15.33:0/2956450555 conn(0x7f53c00fcf40 msgr2=0x7f53c0105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 shutdown_connections 2024-04-23T18:26:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:10.228+0000 7f53c6755700 1 -- 172.21.15.33:0/2956450555 wait complete. 2024-04-23T18:26:10.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:10 smithi106 ceph-mon[32716]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:26:10.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:10 smithi122 ceph-mon[32546]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:26:10.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:10 smithi033 ceph-mon[29274]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-23T18:26:10.975 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-23T18:26:10.984 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:26:10.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-23T18:26:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:12.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:12 smithi106 ceph-mon[32716]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:26:12.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:12 smithi106 ceph-mon[32716]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:12.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:12 smithi122 ceph-mon[32546]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:26:12.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:12 smithi122 ceph-mon[32546]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:12.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:12 smithi033 ceph-mon[29274]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:26:12.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:12 smithi033 ceph-mon[29274]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 -- 172.21.15.33:0/1656289930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 msgr2=0x7f1c5c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 --2- 172.21.15.33:0/1656289930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c102ee0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00b0d0 tx=0x7f1c4c00b3e0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 -- 172.21.15.33:0/1656289930 shutdown_connections 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 --2- 172.21.15.33:0/1656289930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 --2- 172.21.15.33:0/1656289930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c102ee0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 --2- 172.21.15.33:0/1656289930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c5c103cc0 0x7f1c5c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 -- 172.21.15.33:0/1656289930 >> 172.21.15.33:0/1656289930 conn(0x7f1c5c0fe060 msgr2=0x7f1c5c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 -- 172.21.15.33:0/1656289930 shutdown_connections 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.048+0000 7f1c6376c700 1 -- 172.21.15.33:0/1656289930 wait complete. 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 Processor -- start 2024-04-23T18:26:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 -- start start 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c5c103cc0 0x7f1c5c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c10e530 con 0x7f1c5c102ac0 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c10e3b0 con 0x7f1c5c103cc0 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c6376c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5c10e6b0 con 0x7f1c5c104660 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c61508700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c61508700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54930/0 (socket says 172.21.15.33:54930) 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c61508700 1 -- 172.21.15.33:0/790674662 learned_addr learned my addr 172.21.15.33:0/790674662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.049+0000 7f1c61d09700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 -- 172.21.15.33:0/790674662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 msgr2=0x7f1c5c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 -- 172.21.15.33:0/790674662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c5c103cc0 msgr2=0x7f1c5c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c5c103cc0 0x7f1c5c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 -- 172.21.15.33:0/790674662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5c075c20 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61d09700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c61508700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00d010 tx=0x7f1c4c004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c01f030 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c4c007c70 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c004810 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5c1a2c20 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.050+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c5c1a3070 con 0x7f1c5c102ac0 2024-04-23T18:26:13.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.051+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5c02df80 con 0x7f1c5c102ac0 2024-04-23T18:26:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.052+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1c4c003a00 con 0x7f1c5c102ac0 2024-04-23T18:26:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.053+0000 7f1c5a7fc700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 0x7f1c440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.053+0000 7f1c60d07700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 0x7f1c440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.053+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f1c4c00f020 con 0x7f1c5c102ac0 2024-04-23T18:26:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.053+0000 7f1c60d07700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 0x7f1c440a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005fd0 tx=0x7f1c50005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.057+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1c4c029020 con 0x7f1c5c102ac0 2024-04-23T18:26:13.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.311+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1c5c02e150 con 0x7f1c5c102ac0 2024-04-23T18:26:13.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.311+0000 7f1c5a7fc700 1 -- 172.21.15.33:0/790674662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3088 (secure 0 0 0) 0x7f1c4c09e620 con 0x7f1c5c102ac0 2024-04-23T18:26:13.304 INFO:teuthology.orchestra.run.smithi033.stdout:e27 2024-04-23T18:26:13.304 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 27 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-23T18:24:26.661481+0000 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-23T18:24:34.355479+0000 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-23T18:26:13.305 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 5 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2,3,4 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24551,1=24515,2=24490,3=24482,4=24466} 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-23T18:26:13.306 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 24551 members: 24490,24551,24482,24515,24466 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:24551} state up:active seq 3 addr [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.122:6834/3209242154,v1:172.21.15.122:6835/3209242154] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{1:24515} state up:active seq 6 addr [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{1:24446} state up:standby-replay seq 1 addr [v2:172.21.15.106:6832/637332963,v1:172.21.15.106:6833/637332963] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{2:24490} state up:active seq 6 addr [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{2:24418} state up:standby-replay seq 1 addr [v2:172.21.15.122:6832/2194426420,v1:172.21.15.122:6833/2194426420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{3:24482} state up:active seq 9 addr [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{3:14640} state up:standby-replay seq 1 addr [v2:172.21.15.33:6840/363309287,v1:172.21.15.33:6841/363309287] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{4:24466} state up:active seq 8 addr [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{4:14628} state up:standby-replay seq 1 addr [v2:172.21.15.33:6838/1373616976,v1:172.21.15.33:6839/1373616976] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:13.307 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:13.308 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-23T18:26:13.308 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:13.308 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.33:6834/2258040370,v1:172.21.15.33:6835/2258040370] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.308 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.33:6836/3122213420,v1:172.21.15.33:6837/3122213420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 msgr2=0x7f1c440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 0x7f1c440a6180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1c50005fd0 tx=0x7f1c50005d00 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 msgr2=0x7f1c5c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00d010 tx=0x7f1c4c004ef0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 shutdown_connections 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f1c440a3cc0 0x7f1c440a6180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1c5c104660 0x7f1c5c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c5c102ac0 0x7f1c5c0767e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 --2- 172.21.15.33:0/790674662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c5c103cc0 0x7f1c5c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 >> 172.21.15.33:0/790674662 conn(0x7f1c5c0fe060 msgr2=0x7f1c5c106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 shutdown_connections 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:13.314+0000 7f1c6376c700 1 -- 172.21.15.33:0/790674662 wait complete. 2024-04-23T18:26:13.309 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:26:14.000 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-23T18:26:14.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:14 smithi033 ceph-mon[29274]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:26:14.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:14 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/790674662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:26:14.253 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:14.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:14 smithi106 ceph-mon[32716]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:26:14.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:14 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/790674662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:26:14.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:14 smithi122 ceph-mon[32546]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T18:26:14.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:14 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/790674662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-23T18:26:16.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 -- 172.21.15.33:0/2031506574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4102ba0 msgr2=0x7f0fd4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/2031506574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103000 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0fc80099c0 tx=0x7f0fc8009cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 -- 172.21.15.33:0/2031506574 shutdown_connections 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/2031506574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd4103540 0x7f0fd4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/2031506574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd41019a0 0x7f0fd4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/2031506574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103000 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 -- 172.21.15.33:0/2031506574 >> 172.21.15.33:0/2031506574 conn(0x7f0fd40fcf40 msgr2=0x7f0fd40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 -- 172.21.15.33:0/2031506574 shutdown_connections 2024-04-23T18:26:16.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.010+0000 7f0fd8a90700 1 -- 172.21.15.33:0/2031506574 wait complete. 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 Processor -- start 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 -- start start 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd41019a0 0x7f0fd41058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410d410 con 0x7f0fd4103540 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410d290 con 0x7f0fd41019a0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.011+0000 7f0fd8a90700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410d590 con 0x7f0fd4102ba0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54948/0 (socket says 172.21.15.33:54948) 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 -- 172.21.15.33:0/4031658887 learned_addr learned my addr 172.21.15.33:0/4031658887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd1d9b700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd259c700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd41019a0 0x7f0fd41058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 -- 172.21.15.33:0/4031658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd4102ba0 msgr2=0x7f0fd4103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 -- 172.21.15.33:0/4031658887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd41019a0 msgr2=0x7f0fd41058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd41019a0 0x7f0fd41058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 -- 172.21.15.33:0/4031658887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4104d90 con 0x7f0fd4103540 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd2d9d700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00c970 tx=0x7f0fcc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00ce60 con 0x7f0fd4103540 2024-04-23T18:26:16.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd41a2d50 con 0x7f0fd4103540 2024-04-23T18:26:16.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fcc004830 con 0x7f0fd4103540 2024-04-23T18:26:16.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0054c0 con 0x7f0fd4103540 2024-04-23T18:26:16.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.012+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd41a31b0 con 0x7f0fd4103540 2024-04-23T18:26:16.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.013+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4069730 con 0x7f0fd4103540 2024-04-23T18:26:16.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.014+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0fcc0049e0 con 0x7f0fd4103540 2024-04-23T18:26:16.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.015+0000 7f0fc37fe700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 0x7f0fbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:16.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.015+0000 7f0fd259c700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 0x7f0fbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:16.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.015+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f0fcc0cff90 con 0x7f0fd4103540 2024-04-23T18:26:16.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.016+0000 7f0fd259c700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 0x7f0fbc0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4102a00 tx=0x7f0fc4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:16.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.020+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fcc014030 con 0x7f0fd4103540 2024-04-23T18:26:16.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.229+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f0fd41a3b50 con 0x7f0fd4103540 2024-04-23T18:26:16.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:16 smithi033 ceph-mon[29274]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:26:16.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.230+0000 7f0fc37fe700 1 -- 172.21.15.33:0/4031658887 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) ==== 56+0+5318 (secure 0 0 0) 0x7f0fcc09c970 con 0x7f0fd4103540 2024-04-23T18:26:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 87 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-23T18:16:27.249730+0000 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-23T18:25:25.905871+0000 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 26 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-23T18:26:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/3729126407,v1:172.21.15.33:6803/3729126407] [v2:172.21.15.33:6804/3729126407,v1:172.21.15.33:6805/3729126407] exists,up 5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6800/1202694989,v1:172.21.15.106:6801/1202694989] [v2:172.21.15.106:6802/1202694989,v1:172.21.15.106:6803/1202694989] exists,up c3bf75bc-fda4-44d5-be29-dbb282a8a56a 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/2314180839,v1:172.21.15.122:6801/2314180839] [v2:172.21.15.122:6802/2314180839,v1:172.21.15.122:6803/2314180839] exists,up 77497994-8bdd-45c2-8521-089e1016f545 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/1512542347,v1:172.21.15.33:6811/1512542347] [v2:172.21.15.33:6812/1512542347,v1:172.21.15.33:6813/1512542347] exists,up 70af3b74-4df3-4c39-8562-3b0052a1319f 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6808/2880981819,v1:172.21.15.106:6809/2880981819] [v2:172.21.15.106:6810/2880981819,v1:172.21.15.106:6811/2880981819] exists,up b8f450e4-1ae9-4833-90de-3668d806c829 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/2410305978,v1:172.21.15.122:6809/2410305978] [v2:172.21.15.122:6810/2410305978,v1:172.21.15.122:6811/2410305978] exists,up a3bd5fd8-f114-476a-be25-de2b536d9601 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/3512939411,v1:172.21.15.33:6819/3512939411] [v2:172.21.15.33:6820/3512939411,v1:172.21.15.33:6821/3512939411] exists,up c84631e0-7950-46a1-9e3a-24f56cab53f5 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6816/4041578359,v1:172.21.15.106:6817/4041578359] [v2:172.21.15.106:6818/4041578359,v1:172.21.15.106:6819/4041578359] exists,up 8a454eca-15b1-4305-bf15-24a47be985d4 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/497939734,v1:172.21.15.122:6817/497939734] [v2:172.21.15.122:6818/497939734,v1:172.21.15.122:6819/497939734] exists,up 574d7ce1-62ac-4720-9f2a-d40cdf230ffc 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/3157866333,v1:172.21.15.33:6827/3157866333] [v2:172.21.15.33:6828/3157866333,v1:172.21.15.33:6829/3157866333] exists,up 0449b46b-87c2-4174-b0fc-623efb4f453d 2024-04-23T18:26:16.225 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6824/4120078808,v1:172.21.15.106:6825/4120078808] [v2:172.21.15.106:6826/4120078808,v1:172.21.15.106:6827/4120078808] exists,up 7fab0d97-f3b4-484d-8f60-12b5327a1a5e 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/3755562411,v1:172.21.15.122:6825/3755562411] [v2:172.21.15.122:6826/3755562411,v1:172.21.15.122:6827/3755562411] exists,up 95bc8f26-44c0-429c-893b-566a272d0dbf 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.a [2,11] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.e [2,4] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1d [8,11] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.2 [3,9] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.17 [4,6] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.19 [3,7] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1b [4,8] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.25 [5,9] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.2c [4,9] 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/4245296611 expires 2024-04-24T18:17:24.067933+0000 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/4110897883 expires 2024-04-24T18:17:24.067933+0000 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2953329503 expires 2024-04-24T18:17:24.067933+0000 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/1526532469 expires 2024-04-24T18:16:57.302656+0000 2024-04-23T18:26:16.226 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/4245296611 expires 2024-04-24T18:17:24.067933+0000 2024-04-23T18:26:16.227 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/4072285085 expires 2024-04-24T18:16:57.302656+0000 2024-04-23T18:26:16.227 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/438669199 expires 2024-04-24T18:16:57.302656+0000 2024-04-23T18:26:16.227 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/1526532469 expires 2024-04-24T18:16:57.302656+0000 2024-04-23T18:26:16.227 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3363990252 expires 2024-04-24T18:16:57.302656+0000 2024-04-23T18:26:16.227 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/1895522836 expires 2024-04-24T18:17:24.067933+0000 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.233+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 msgr2=0x7f0fbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.233+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 0x7f0fbc0a6100 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4102a00 tx=0x7f0fc4009040 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.233+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 msgr2=0x7f0fd41044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.233+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00c970 tx=0x7f0fcc00cc80 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 shutdown_connections 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0fbc0a3c40 0x7f0fbc0a6100 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fd4103540 0x7f0fd41044c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fd41019a0 0x7f0fd41058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 --2- 172.21.15.33:0/4031658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fd4102ba0 0x7f0fd4103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 >> 172.21.15.33:0/4031658887 conn(0x7f0fd40fcf40 msgr2=0x7f0fd406a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 shutdown_connections 2024-04-23T18:26:16.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:16.234+0000 7f0fd8a90700 1 -- 172.21.15.33:0/4031658887 wait complete. 2024-04-23T18:26:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:16 smithi106 ceph-mon[32716]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:26:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:16 smithi122 ceph-mon[32546]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:26:16.945 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-23T18:26:16.956 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:17.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 -- 172.21.15.33:0/2776352470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102040 msgr2=0x7fa3c41024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/2776352470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102040 0x7fa3c41024c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 -- 172.21.15.33:0/2776352470 shutdown_connections 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/2776352470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102a00 0x7fa3c4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/2776352470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102040 0x7fa3c41024c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/2776352470 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4100e60 0x7fa3c4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 -- 172.21.15.33:0/2776352470 >> 172.21.15.33:0/2776352470 conn(0x7fa3c40fc440 msgr2=0x7fa3c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 -- 172.21.15.33:0/2776352470 shutdown_connections 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.069+0000 7fa3cbeef640 1 -- 172.21.15.33:0/2776352470 wait complete. 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 Processor -- start 2024-04-23T18:26:17.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 -- start start 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 0x7fa3c41034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4102a00 0x7fa3c41039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410ca00 con 0x7fa3c4100e60 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410c880 con 0x7fa3c4102a00 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3cbeef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c410cb80 con 0x7fa3c4102040 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48518/0 (socket says 172.21.15.33:48518) 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 -- 172.21.15.33:0/140172929 learned_addr learned my addr 172.21.15.33:0/140172929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c948a640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 0x7fa3c41034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 -- 172.21.15.33:0/140172929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 msgr2=0x7fa3c41034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 0x7fa3c41034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3ca48c640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4102a00 0x7fa3c41039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 -- 172.21.15.33:0/140172929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4102a00 msgr2=0x7fa3c41039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4102a00 0x7fa3c41039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c9c8b640 1 -- 172.21.15.33:0/140172929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c41042b0 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.070+0000 7fa3c948a640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 0x7fa3c41034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3c9c8b640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c970 tx=0x7fa3c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0004220 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3c00043c0 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f450 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c41a2340 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.071+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a2810 con 0x7fa3c4100e60 2024-04-23T18:26:17.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.072+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c41a2ad0 con 0x7fa3c4100e60 2024-04-23T18:26:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.073+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa3c0003610 con 0x7fa3c4100e60 2024-04-23T18:26:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.073+0000 7fa3baffd640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 0x7fa3b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.074+0000 7fa3c948a640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 0x7fa3b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.074+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fa3c00d0270 con 0x7fa3c4100e60 2024-04-23T18:26:17.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.074+0000 7fa3c948a640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 0x7fa3b00a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006010 tx=0x7fa3b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.078+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3c0014030 con 0x7fa3c4100e60 2024-04-23T18:26:17.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4031658887' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:26:17.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.267+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c41a3280 con 0x7fa3c4100e60 2024-04-23T18:26:17.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.269+0000 7fa3baffd640 1 -- 172.21.15.33:0/140172929 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7fa3c009cc10 con 0x7fa3c4100e60 2024-04-23T18:26:17.262 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:17.262 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 msgr2=0x7fa3b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 0x7fa3b00a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4006010 tx=0x7fa3b4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 msgr2=0x7fa3c4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000c970 tx=0x7fa3c000ce30 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 shutdown_connections 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa3b00a3c30 0x7fa3b00a60f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3c4102a00 0x7fa3c41039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3c4102040 0x7fa3c41034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 --2- 172.21.15.33:0/140172929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4100e60 0x7fa3c4104e20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.271+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 >> 172.21.15.33:0/140172929 conn(0x7fa3c40fc440 msgr2=0x7fa3c40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.272+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 shutdown_connections 2024-04-23T18:26:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.272+0000 7fa3cbeef640 1 -- 172.21.15.33:0/140172929 wait complete. 2024-04-23T18:26:17.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:17.276 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:17.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4031658887' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 -- 172.21.15.33:0/4074751663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 msgr2=0x7f4928101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 --2- 172.21.15.33:0/4074751663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928101ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4910009900 tx=0x7f491002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 -- 172.21.15.33:0/4074751663 shutdown_connections 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 --2- 172.21.15.33:0/4074751663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49281021e0 0x7f4928108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 --2- 172.21.15.33:0/4074751663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928101ca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 --2- 172.21.15.33:0/4074751663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4928100630 0x7f4928100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 -- 172.21.15.33:0/4074751663 >> 172.21.15.33:0/4074751663 conn(0x7f49280fbb80 msgr2=0x7f49280fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 -- 172.21.15.33:0/4074751663 shutdown_connections 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.389+0000 7f492c930640 1 -- 172.21.15.33:0/4074751663 wait complete. 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 Processor -- start 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 -- start start 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4928100630 0x7f49281981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49281021e0 0x7f492819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f492810b940 con 0x7f4928101820 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f492810b7c0 con 0x7f4928100630 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492c930640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f492810bac0 con 0x7f49281021e0 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48530/0 (socket says 172.21.15.33:48530) 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 -- 172.21.15.33:0/2498470884 learned_addr learned my addr 172.21.15.33:0/2498470884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:17.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4926d9d640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49281021e0 0x7f492819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f492659c640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4928100630 0x7f49281981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 -- 172.21.15.33:0/2498470884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49281021e0 msgr2=0x7f492819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49281021e0 0x7f492819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 -- 172.21.15.33:0/2498470884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4928100630 msgr2=0x7f49281981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4928100630 0x7f49281981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 -- 172.21.15.33:0/2498470884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492819fe00 con 0x7f4928101820 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.390+0000 7f4925d9b640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4910005ab0 tx=0x7f4910003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.391+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491003f070 con 0x7f4928101820 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.391+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f491003b460 con 0x7f4928101820 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.391+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49281a0090 con 0x7f4928101820 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.391+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910035030 con 0x7f4928101820 2024-04-23T18:26:17.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.391+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281a0580 con 0x7f4928101820 2024-04-23T18:26:17.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.392+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49281078f0 con 0x7f4928101820 2024-04-23T18:26:17.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.393+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4910042070 con 0x7f4928101820 2024-04-23T18:26:17.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.393+0000 7f491f7fe640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 0x7f49140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.393+0000 7f492659c640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 0x7f49140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.394+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f49100f0db0 con 0x7f4928101820 2024-04-23T18:26:17.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.394+0000 7f492659c640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 0x7f49140a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4918005c40 tx=0x7f4918005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.398+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49100bd790 con 0x7f4928101820 2024-04-23T18:26:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4031658887' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-23T18:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.586+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4928062b50 con 0x7f4928101820 2024-04-23T18:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.587+0000 7f491f7fe640 1 -- 172.21.15.33:0/2498470884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f49100bd090 con 0x7f4928101820 2024-04-23T18:26:17.580 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 msgr2=0x7f49140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 0x7f49140a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4918005c40 tx=0x7f4918005bd0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 msgr2=0x7f4928198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4910005ab0 tx=0x7f4910003cd0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 shutdown_connections 2024-04-23T18:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49140a3bb0 0x7f49140a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49281021e0 0x7f492819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4928101820 0x7f4928198720 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 --2- 172.21.15.33:0/2498470884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4928100630 0x7f49281981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 >> 172.21.15.33:0/2498470884 conn(0x7f49280fbb80 msgr2=0x7f4928104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 shutdown_connections 2024-04-23T18:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.589+0000 7f492c930640 1 -- 172.21.15.33:0/2498470884 wait complete. 2024-04-23T18:26:17.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 -- 172.21.15.33:0/1976182528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 msgr2=0x7f144c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 --2- 172.21.15.33:0/1976182528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c103620 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1440009900 tx=0x7f144002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 -- 172.21.15.33:0/1976182528 shutdown_connections 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 --2- 172.21.15.33:0/1976182528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c103b60 0x7f144c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 --2- 172.21.15.33:0/1976182528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c103620 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 --2- 172.21.15.33:0/1976182528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c101fb0 0x7f144c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 -- 172.21.15.33:0/1976182528 >> 172.21.15.33:0/1976182528 conn(0x7f144c0fd540 msgr2=0x7f144c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 -- 172.21.15.33:0/1976182528 shutdown_connections 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 -- 172.21.15.33:0/1976182528 wait complete. 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.705+0000 7f1451c13640 1 Processor -- start 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 -- start start 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 0x7f144c075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c103b60 0x7f144c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10db40 con 0x7f144c1031a0 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10d9c0 con 0x7f144c103b60 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f1451c13640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f144c10dcc0 con 0x7f144c101fb0 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48538/0 (socket says 172.21.15.33:48538) 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 -- 172.21.15.33:0/1114155984 learned_addr learned my addr 172.21.15.33:0/1114155984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:17.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144b7fe640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 0x7f144c075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144bfff640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c103b60 0x7f144c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 -- 172.21.15.33:0/1114155984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 msgr2=0x7f144c075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 0x7f144c075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 -- 172.21.15.33:0/1114155984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c103b60 msgr2=0x7f144c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c103b60 0x7f144c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 -- 172.21.15.33:0/1114155984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144c075110 con 0x7f144c1031a0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144b7fe640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 0x7f144c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.706+0000 7f144affd640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f1440003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.707+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144003f070 con 0x7f144c1031a0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.707+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1440031c50 con 0x7f144c1031a0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.707+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144c1a2250 con 0x7f144c1031a0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.707+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440035030 con 0x7f144c1031a0 2024-04-23T18:26:17.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.707+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144c1a27a0 con 0x7f144c1031a0 2024-04-23T18:26:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.708+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144c1092d0 con 0x7f144c1031a0 2024-04-23T18:26:17.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.709+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1440042070 con 0x7f144c1031a0 2024-04-23T18:26:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.709+0000 7f1448ff9640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.709+0000 7f144b7fe640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.709+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f14400f0c80 con 0x7f144c1031a0 2024-04-23T18:26:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.710+0000 7f144b7fe640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 0x7f14340a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f144c103000 tx=0x7f143c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:17.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.714+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14400f7080 con 0x7f144c1031a0 2024-04-23T18:26:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.899+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f144c062b50 con 0x7f144c1031a0 2024-04-23T18:26:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.900+0000 7f1448ff9640 1 -- 172.21.15.33:0/1114155984 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f14400bd660 con 0x7f144c1031a0 2024-04-23T18:26:17.893 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 msgr2=0x7f14340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 0x7f14340a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f144c103000 tx=0x7f143c00b390 comp rx=0 tx=0).stop 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 msgr2=0x7f144c074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1440005ab0 tx=0x7f1440003b90 comp rx=0 tx=0).stop 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 shutdown_connections 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f14340a3bb0 0x7f14340a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f144c103b60 0x7f144c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f144c1031a0 0x7f144c074340 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 --2- 172.21.15.33:0/1114155984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f144c101fb0 0x7f144c075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 >> 172.21.15.33:0/1114155984 conn(0x7f144c0fd540 msgr2=0x7f144c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 shutdown_connections 2024-04-23T18:26:17.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:17.902+0000 7f1451c13640 1 -- 172.21.15.33:0/1114155984 wait complete. 2024-04-23T18:26:17.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 -- 172.21.15.33:0/1609121319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 msgr2=0x7f8ac01023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/1609121319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01023d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009900 tx=0x7f8abc02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 -- 172.21.15.33:0/1609121319 shutdown_connections 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/1609121319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac010a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/1609121319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 0x7f8ac0103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/1609121319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01023d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 -- 172.21.15.33:0/1609121319 >> 172.21.15.33:0/1609121319 conn(0x7f8ac00fd520 msgr2=0x7f8ac00ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 -- 172.21.15.33:0/1609121319 shutdown_connections 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.018+0000 7f8ac87d8640 1 -- 172.21.15.33:0/1609121319 wait complete. 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 Processor -- start 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 -- start start 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 0x7f8ac0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac010daf0 con 0x7f8ac0101fb0 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac010d970 con 0x7f8ac0103b60 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac87d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac010dc70 con 0x7f8ac01031a0 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48552/0 (socket says 172.21.15.33:48552) 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 -- 172.21.15.33:0/868989109 learned_addr learned my addr 172.21.15.33:0/868989109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac5d73640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 0x7f8ac0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6d75640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 -- 172.21.15.33:0/868989109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 msgr2=0x7f8ac0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 0x7f8ac0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 -- 172.21.15.33:0/868989109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 msgr2=0x7f8ac019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 -- 172.21.15.33:0/868989109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac019fde0 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6d75640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.019+0000 7f8ac6574640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006010 tx=0x7f8abc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.020+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc03f030 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.020+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8abc003cb0 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.020+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac01a0070 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.020+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc035070 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.020+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac01a0560 con 0x7f8ac0101fb0 2024-04-23T18:26:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.021+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac01093c0 con 0x7f8ac0101fb0 2024-04-23T18:26:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.022+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8abc042030 con 0x7f8ac0101fb0 2024-04-23T18:26:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.022+0000 7f8ab77fe640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 0x7f8aac0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.022+0000 7f8ac5d73640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 0x7f8aac0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.022+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f8abc0f0a40 con 0x7f8ac0101fb0 2024-04-23T18:26:18.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.023+0000 7f8ac5d73640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 0x7f8aac0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ac01996d0 tx=0x7f8ab000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.027+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8abc0f2e20 con 0x7f8ac0101fb0 2024-04-23T18:26:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.215+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ac0062b50 con 0x7f8ac0101fb0 2024-04-23T18:26:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.216+0000 7f8ab77fe640 1 -- 172.21.15.33:0/868989109 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f8abc0bd420 con 0x7f8ac0101fb0 2024-04-23T18:26:18.209 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 msgr2=0x7f8aac0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 0x7f8aac0a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8ac01996d0 tx=0x7f8ab000b390 comp rx=0 tx=0).stop 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 msgr2=0x7f8ac01981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8abc006010 tx=0x7f8abc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 shutdown_connections 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f8aac0a3bb0 0x7f8aac0a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0103b60 0x7f8ac019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac01031a0 0x7f8ac0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 --2- 172.21.15.33:0/868989109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ac0101fb0 0x7f8ac01981c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 >> 172.21.15.33:0/868989109 conn(0x7f8ac00fd520 msgr2=0x7f8ac0106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 shutdown_connections 2024-04-23T18:26:18.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.218+0000 7f8ac87d8640 1 -- 172.21.15.33:0/868989109 wait complete. 2024-04-23T18:26:18.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:18 smithi033 ceph-mon[29274]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:26:18.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/140172929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:18.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2498470884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1114155984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:18.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 -- 172.21.15.33:0/429882725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0101fe0 msgr2=0x7fe7c0102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/429882725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0102460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009900 tx=0x7fe7ac02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 -- 172.21.15.33:0/429882725 shutdown_connections 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/429882725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/429882725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0102460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/429882725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0100e90 0x7fe7c01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.331+0000 7fe7c6f19640 1 -- 172.21.15.33:0/429882725 >> 172.21.15.33:0/429882725 conn(0x7fe7c00fc400 msgr2=0x7fe7c00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- 172.21.15.33:0/429882725 shutdown_connections 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- 172.21.15.33:0/429882725 wait complete. 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 Processor -- start 2024-04-23T18:26:18.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- start start 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c010ca40 con 0x7fe7c0100e90 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c010c8c0 con 0x7fe7c01029a0 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c6f19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c010cbc0 con 0x7fe7c0101fe0 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c4cb5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c4cb5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48556/0 (socket says 172.21.15.33:48556) 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c4cb5640 1 -- 172.21.15.33:0/2425058725 learned_addr learned my addr 172.21.15.33:0/2425058725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7c54b6640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.332+0000 7fe7bffff640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 -- 172.21.15.33:0/2425058725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0101fe0 msgr2=0x7fe7c0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 -- 172.21.15.33:0/2425058725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 msgr2=0x7fe7c019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 -- 172.21.15.33:0/2425058725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c019fee0 con 0x7fe7c0100e90 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c54b6640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c4cb5640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400b4a0 tx=0x7fe7b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4014070 con 0x7fe7c0100e90 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b4010ac0 con 0x7fe7c0100e90 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c01a01d0 con 0x7fe7c0100e90 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017c30 con 0x7fe7c0100e90 2024-04-23T18:26:18.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.333+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01a0720 con 0x7fe7c0100e90 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.334+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c01a09e0 con 0x7fe7c0100e90 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.335+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7c0100e90 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.336+0000 7fe7bdffb640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 0x7fe7b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.336+0000 7fe7bffff640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 0x7fe7b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.336+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fe7b40cf7e0 con 0x7fe7c0100e90 2024-04-23T18:26:18.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.337+0000 7fe7bffff640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 0x7fe7b00a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009230 tx=0x7fe7ac0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.341+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7b409c180 con 0x7fe7c0100e90 2024-04-23T18:26:18.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:18 smithi106 ceph-mon[32716]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:26:18.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:18 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/140172929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:18.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:18 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2498470884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:18 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1114155984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:18 smithi122 ceph-mon[32546]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-04-23T18:26:18.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:18 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/140172929' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:18.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:18 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2498470884' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:18 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1114155984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.566+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7c01a0bb0 con 0x7fe7c0100e90 2024-04-23T18:26:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.567+0000 7fe7bdffb640 1 -- 172.21.15.33:0/2425058725 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7fe7b409ba80 con 0x7fe7c0100e90 2024-04-23T18:26:18.560 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:18.560 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:26:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 msgr2=0x7fe7b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 0x7fe7b00a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009230 tx=0x7fe7ac0314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 msgr2=0x7fe7c01982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400b4a0 tx=0x7fe7b400b960 comp rx=0 tx=0).stop 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 shutdown_connections 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fe7b00a3bb0 0x7fe7b00a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c01029a0 0x7fe7c019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c0101fe0 0x7fe7c0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 --2- 172.21.15.33:0/2425058725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7c0100e90 0x7fe7c01982c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 >> 172.21.15.33:0/2425058725 conn(0x7fe7c00fc400 msgr2=0x7fe7c0069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 shutdown_connections 2024-04-23T18:26:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.569+0000 7fe7c6f19640 1 -- 172.21.15.33:0/2425058725 wait complete. 2024-04-23T18:26:18.573 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/111054581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78103b70 msgr2=0x7f7c7810a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/111054581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78103b70 0x7f7c7810a400 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009ca0 tx=0x7f7c7002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/111054581 shutdown_connections 2024-04-23T18:26:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/111054581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78103b70 0x7f7c7810a400 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/111054581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/111054581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78101fc0 0x7f7c781023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/111054581 >> 172.21.15.33:0/111054581 conn(0x7f7c780fd530 msgr2=0x7f7c780ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/111054581 shutdown_connections 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.683+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/111054581 wait complete. 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 Processor -- start 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 -- start start 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78103b70 0x7f7c78074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810db50 con 0x7f7c78101fc0 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810d9d0 con 0x7f7c78103b70 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7cb9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7810dcd0 con 0x7f7c781031b0 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48566/0 (socket says 172.21.15.33:48566) 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c7659c640 1 -- 172.21.15.33:0/1507821489 learned_addr learned my addr 172.21.15.33:0/1507821489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:18.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.684+0000 7f7c6ffff640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 -- 172.21.15.33:0/1507821489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 msgr2=0x7f7c78074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 -- 172.21.15.33:0/1507821489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78103b70 msgr2=0x7f7c78074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c76d9d640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78103b70 0x7f7c78074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78103b70 0x7f7c78074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 -- 172.21.15.33:0/1507821489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c78075110 con 0x7f7c78101fc0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c6ffff640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7659c640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800d770 tx=0x7f7c6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68017070 con 0x7f7c78101fc0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c781a22a0 con 0x7f7c78101fc0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c68009c30 con 0x7f7c78101fc0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003cc0 con 0x7f7c78101fc0 2024-04-23T18:26:18.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.685+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c781a27f0 con 0x7f7c78101fc0 2024-04-23T18:26:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.686+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c781092e0 con 0x7f7c78101fc0 2024-04-23T18:26:18.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.687+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c68003e60 con 0x7f7c78101fc0 2024-04-23T18:26:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.688+0000 7f7c6f7fe640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 0x7f7c640a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.688+0000 7f7c6ffff640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 0x7f7c640a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.688+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f7c680d0290 con 0x7f7c78101fc0 2024-04-23T18:26:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.688+0000 7f7c6ffff640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 0x7f7c640a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c78075a70 tx=0x7f7c60005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.692+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c68009de0 con 0x7f7c78101fc0 2024-04-23T18:26:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.879+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c7802d870 con 0x7f7c78101fc0 2024-04-23T18:26:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.880+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1507821489 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f7c6809cc30 con 0x7f7c78101fc0 2024-04-23T18:26:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:18.873 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 msgr2=0x7f7c640a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 0x7f7c640a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7c78075a70 tx=0x7f7c60005e50 comp rx=0 tx=0).stop 2024-04-23T18:26:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 msgr2=0x7f7c78075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800d770 tx=0x7f7c6800dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 shutdown_connections 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7c640a3bb0 0x7f7c640a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c78103b70 0x7f7c78074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c781031b0 0x7f7c78074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 --2- 172.21.15.33:0/1507821489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c78101fc0 0x7f7c78075d00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 >> 172.21.15.33:0/1507821489 conn(0x7f7c780fd530 msgr2=0x7f7c781064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 shutdown_connections 2024-04-23T18:26:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:18.882+0000 7f7c7cb9b640 1 -- 172.21.15.33:0/1507821489 wait complete. 2024-04-23T18:26:18.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:18.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 -- 172.21.15.33:0/2195974537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c101fe0 msgr2=0x7f868c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 --2- 172.21.15.33:0/2195974537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c101fe0 0x7f868c102460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8680009900 tx=0x7f868002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 -- 172.21.15.33:0/2195974537 shutdown_connections 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 --2- 172.21.15.33:0/2195974537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c1029a0 0x7f868c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 --2- 172.21.15.33:0/2195974537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c101fe0 0x7f868c102460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 --2- 172.21.15.33:0/2195974537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 0x7f868c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 -- 172.21.15.33:0/2195974537 >> 172.21.15.33:0/2195974537 conn(0x7f868c0fc400 msgr2=0x7f868c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 -- 172.21.15.33:0/2195974537 shutdown_connections 2024-04-23T18:26:18.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.000+0000 7f8691ca0640 1 -- 172.21.15.33:0/2195974537 wait complete. 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 Processor -- start 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 -- start start 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 0x7f868c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c101fe0 0x7f868c19f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10c9d0 con 0x7f868c1029a0 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10c850 con 0x7f868c101fe0 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f8691ca0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f868c10cb50 con 0x7f868c100e90 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48574/0 (socket says 172.21.15.33:48574) 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 -- 172.21.15.33:0/633846868 learned_addr learned my addr 172.21.15.33:0/633846868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 -- 172.21.15.33:0/633846868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 msgr2=0x7f868c069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868b7fe640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 0x7f868c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 0x7f868c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 -- 172.21.15.33:0/633846868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c101fe0 msgr2=0x7f868c19f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868affd640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c101fe0 0x7f868c19f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c101fe0 0x7f868c19f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.001+0000 7f868bfff640 1 -- 172.21.15.33:0/633846868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c1a2090 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f868bfff640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f868400eaa0 tx=0x7f868400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400cdb0 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8684005020 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c1a2320 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86840116d0 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.002+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1a2870 con 0x7f868c1029a0 2024-04-23T18:26:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.003+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c108310 con 0x7f868c1029a0 2024-04-23T18:26:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.004+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f86840051d0 con 0x7f868c1029a0 2024-04-23T18:26:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.004+0000 7f8688ff9640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 0x7f86740a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.005+0000 7f868b7fe640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 0x7f86740a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.005+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f86840d0670 con 0x7f868c1029a0 2024-04-23T18:26:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.005+0000 7f868b7fe640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 0x7f86740a60a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.009+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8684016030 con 0x7f868c1029a0 2024-04-23T18:26:19.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.198+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f868c062bf0 con 0x7f868c1029a0 2024-04-23T18:26:19.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.198+0000 7f8688ff9640 1 -- 172.21.15.33:0/633846868 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7f868409d050 con 0x7f868c1029a0 2024-04-23T18:26:19.191 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:26:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 msgr2=0x7f86740a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 0x7f86740a60a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c005850 comp rx=0 tx=0).stop 2024-04-23T18:26:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 msgr2=0x7f868c1a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f868400eaa0 tx=0x7f868400ef60 comp rx=0 tx=0).stop 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 shutdown_connections 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f86740a3be0 0x7f86740a60a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f868c1029a0 0x7f868c1a18a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f868c101fe0 0x7f868c19f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 --2- 172.21.15.33:0/633846868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f868c100e90 0x7f868c069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 >> 172.21.15.33:0/633846868 conn(0x7f868c0fc400 msgr2=0x7f868c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 shutdown_connections 2024-04-23T18:26:19.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.200+0000 7f8691ca0640 1 -- 172.21.15.33:0/633846868 wait complete. 2024-04-23T18:26:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:19 smithi033 ceph-mon[29274]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:26:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/868989109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2425058725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1507821489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 -- 172.21.15.33:0/2042484555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 msgr2=0x7f0ec01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/2042484555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01023a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009900 tx=0x7f0eb002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 -- 172.21.15.33:0/2042484555 shutdown_connections 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/2042484555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103b30 0x7f0ec010a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/2042484555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103170 0x7f0ec01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/2042484555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01023a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 -- 172.21.15.33:0/2042484555 >> 172.21.15.33:0/2042484555 conn(0x7f0ec00fd510 msgr2=0x7f0ec00ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 -- 172.21.15.33:0/2042484555 shutdown_connections 2024-04-23T18:26:19.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 -- 172.21.15.33:0/2042484555 wait complete. 2024-04-23T18:26:19.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.317+0000 7f0ec5d08640 1 Processor -- start 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- start start 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 0x7f0ec0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 0x7f0ec019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010db50 con 0x7f0ec0101f80 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010d9d0 con 0x7f0ec0103170 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec010dcd0 con 0x7f0ec0103b30 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48588/0 (socket says 172.21.15.33:48588) 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 -- 172.21.15.33:0/3705530569 learned_addr learned my addr 172.21.15.33:0/3705530569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebeffd640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 0x7f0ec0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebffff640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 0x7f0ec019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 -- 172.21.15.33:0/3705530569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 msgr2=0x7f0ec019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 0x7f0ec019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 -- 172.21.15.33:0/3705530569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 msgr2=0x7f0ec0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 0x7f0ec0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 -- 172.21.15.33:0/3705530569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec019fde0 con 0x7f0ec0101f80 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebeffd640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 0x7f0ec0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebffff640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 0x7f0ec019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebf7fe640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0031500 tx=0x7f0eb0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb003f070 con 0x7f0ec0101f80 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0eb003b460 con 0x7f0ec0101f80 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0035030 con 0x7f0ec0101f80 2024-04-23T18:26:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a0070 con 0x7f0ec0101f80 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.318+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a0630 con 0x7f0ec0101f80 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.320+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec01092a0 con 0x7f0ec0101f80 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.321+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0eb0042070 con 0x7f0ec0101f80 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.321+0000 7f0ebcff9640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 0x7f0ea80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.322+0000 7f0ebeffd640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 0x7f0ea80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.322+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f0eb00f0dd0 con 0x7f0ec0101f80 2024-04-23T18:26:19.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.322+0000 7f0ebeffd640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 0x7f0ea80a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006fd0 tx=0x7f0eb4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.326+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0eb00f4030 con 0x7f0ec0101f80 2024-04-23T18:26:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:19 smithi106 ceph-mon[32716]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:26:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/868989109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2425058725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1507821489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:19 smithi122 ceph-mon[32546]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-23T18:26:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/868989109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2425058725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1507821489' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:19.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.511+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0ec0062b50 con 0x7f0ec0101f80 2024-04-23T18:26:19.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.512+0000 7f0ebcff9640 1 -- 172.21.15.33:0/3705530569 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f0eb00bd7b0 con 0x7f0ec0101f80 2024-04-23T18:26:19.505 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 msgr2=0x7f0ea80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 0x7f0ea80a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4006fd0 tx=0x7f0eb4008040 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 msgr2=0x7f0ec01981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0031500 tx=0x7f0eb0003d50 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 shutdown_connections 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0ea80a3c30 0x7f0ea80a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ec0103b30 0x7f0ec019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0ec0103170 0x7f0ec0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 --2- 172.21.15.33:0/3705530569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ec0101f80 0x7f0ec01981c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 >> 172.21.15.33:0/3705530569 conn(0x7f0ec00fd510 msgr2=0x7f0ec0106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 shutdown_connections 2024-04-23T18:26:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.514+0000 7f0ec5d08640 1 -- 172.21.15.33:0/3705530569 wait complete. 2024-04-23T18:26:19.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 -- 172.21.15.33:0/2148296515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102a00 msgr2=0x7f7f38109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/2148296515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102a00 0x7f7f38109320 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7f28009ca0 tx=0x7f7f2802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 -- 172.21.15.33:0/2148296515 shutdown_connections 2024-04-23T18:26:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/2148296515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102a00 0x7f7f38109320 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/2148296515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38102040 0x7f7f381024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/2148296515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38100e60 0x7f7f38101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 -- 172.21.15.33:0/2148296515 >> 172.21.15.33:0/2148296515 conn(0x7f7f380fc420 msgr2=0x7f7f380fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 -- 172.21.15.33:0/2148296515 shutdown_connections 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 -- 172.21.15.33:0/2148296515 wait complete. 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.630+0000 7f7f3d68a640 1 Processor -- start 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 -- start start 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 0x7f7f381981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 0x7f7f3819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810c9f0 con 0x7f7f38102040 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810c870 con 0x7f7f38100e60 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f3d68a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3810cb70 con 0x7f7f38102a00 2024-04-23T18:26:19.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48612/0 (socket says 172.21.15.33:48612) 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 -- 172.21.15.33:0/3980167894 learned_addr learned my addr 172.21.15.33:0/3980167894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f36ffd640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 0x7f7f381981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f377fe640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 0x7f7f3819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 -- 172.21.15.33:0/3980167894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 msgr2=0x7f7f3819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 0x7f7f3819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 -- 172.21.15.33:0/3980167894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 msgr2=0x7f7f381981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 0x7f7f381981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 -- 172.21.15.33:0/3980167894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3819fdc0 con 0x7f7f38102040 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f377fe640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 0x7f7f3819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f36ffd640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 0x7f7f381981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f367fc640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d950 tx=0x7f7f2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.631+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017070 con 0x7f7f38102040 2024-04-23T18:26:19.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.632+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f20009c30 con 0x7f7f38102040 2024-04-23T18:26:19.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.632+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f381a00b0 con 0x7f7f38102040 2024-04-23T18:26:19.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.632+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2000f620 con 0x7f7f38102040 2024-04-23T18:26:19.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.632+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f381a0600 con 0x7f7f38102040 2024-04-23T18:26:19.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.632+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f381a0860 con 0x7f7f38102040 2024-04-23T18:26:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.634+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7f20010460 con 0x7f7f38102040 2024-04-23T18:26:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.634+0000 7f7f2ffff640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 0x7f7f240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.634+0000 7f7f36ffd640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 0x7f7f240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.634+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f7f200d00e0 con 0x7f7f38102040 2024-04-23T18:26:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.635+0000 7f7f36ffd640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 0x7f7f240a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f38101bb0 tx=0x7f7f3000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.639+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f2009cac0 con 0x7f7f38102040 2024-04-23T18:26:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.824+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7f381a0fd0 con 0x7f7f38102040 2024-04-23T18:26:19.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.825+0000 7f7f2ffff640 1 -- 172.21.15.33:0/3980167894 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f7f20009de0 con 0x7f7f38102040 2024-04-23T18:26:19.817 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.826+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 msgr2=0x7f7f240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.826+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 0x7f7f240a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f38101bb0 tx=0x7f7f3000a560 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.826+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 msgr2=0x7f7f38198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.826+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d950 tx=0x7f7f2000de10 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 shutdown_connections 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f7f240a3bb0 0x7f7f240a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f38102a00 0x7f7f3819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f38102040 0x7f7f38198730 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 --2- 172.21.15.33:0/3980167894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f38100e60 0x7f7f381981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 >> 172.21.15.33:0/3980167894 conn(0x7f7f380fc420 msgr2=0x7f7f380fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 shutdown_connections 2024-04-23T18:26:19.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.827+0000 7f7f3d68a640 1 -- 172.21.15.33:0/3980167894 wait complete. 2024-04-23T18:26:19.831 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 -- 172.21.15.33:0/2919560302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64105340 msgr2=0x7f5d64107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2919560302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64105340 0x7f5d64107730 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009900 tx=0x7f5d5802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 -- 172.21.15.33:0/2919560302 shutdown_connections 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2919560302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64107c70 0x7f5d6410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2919560302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64105340 0x7f5d64107730 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2919560302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64102a10 0x7f5d64104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 -- 172.21.15.33:0/2919560302 >> 172.21.15.33:0/2919560302 conn(0x7f5d640fc400 msgr2=0x7f5d640fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 -- 172.21.15.33:0/2919560302 shutdown_connections 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.939+0000 7f5d6af95640 1 -- 172.21.15.33:0/2919560302 wait complete. 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 Processor -- start 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 -- start start 2024-04-23T18:26:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 0x7f5d641986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64107c70 0x7f5d6419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410daa0 con 0x7f5d64102a10 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410d920 con 0x7f5d64105340 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d6af95640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6410dc20 con 0x7f5d64107c70 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d68d31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d68d31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48640/0 (socket says 172.21.15.33:48640) 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d68d31640 1 -- 172.21.15.33:0/2896697687 learned_addr learned my addr 172.21.15.33:0/2896697687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.940+0000 7f5d63fff640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 0x7f5d641986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d69532640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64107c70 0x7f5d6419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 -- 172.21.15.33:0/2896697687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64107c70 msgr2=0x7f5d6419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64107c70 0x7f5d6419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 -- 172.21.15.33:0/2896697687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 msgr2=0x7f5d641986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 0x7f5d641986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 -- 172.21.15.33:0/2896697687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6419fd90 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d63fff640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 0x7f5d641986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d68d31640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c920 tx=0x7f5d5400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54007c70 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d54007e10 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d641a0080 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018820 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.941+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d641a05a0 con 0x7f5d64102a10 2024-04-23T18:26:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.942+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d640fdef0 con 0x7f5d64102a10 2024-04-23T18:26:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.943+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d5400f450 con 0x7f5d64102a10 2024-04-23T18:26:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.944+0000 7f5d61ffb640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.944+0000 7f5d63fff640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.944+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f5d540d05c0 con 0x7f5d64102a10 2024-04-23T18:26:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.944+0000 7f5d63fff640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d580314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:19.948+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d54014030 con 0x7f5d64102a10 2024-04-23T18:26:20.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.175+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d64062b50 con 0x7f5d64102a10 2024-04-23T18:26:20.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.175+0000 7f5d61ffb640 1 -- 172.21.15.33:0/2896697687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11929 (secure 0 0 0) 0x7f5d5409cfa0 con 0x7f5d64102a10 2024-04-23T18:26:20.168 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:20.168 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:24:34.355479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:26:20.169 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 msgr2=0x7f5d4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d580314a0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 msgr2=0x7f5d64198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400c920 tx=0x7f5d5400cde0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 shutdown_connections 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d64107c70 0x7f5d6419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d64105340 0x7f5d641986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 --2- 172.21.15.33:0/2896697687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d64102a10 0x7f5d64198170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 >> 172.21.15.33:0/2896697687 conn(0x7f5d640fc400 msgr2=0x7f5d64103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 shutdown_connections 2024-04-23T18:26:20.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.178+0000 7f5d6af95640 1 -- 172.21.15.33:0/2896697687 wait complete. 2024-04-23T18:26:20.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:20 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/633846868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:20 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3705530569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:20 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3980167894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 -- 172.21.15.33:0/1770739677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57781029a0 msgr2=0x7f57781092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 --2- 172.21.15.33:0/1770739677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57781029a0 0x7f57781092c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5760009900 tx=0x7f576002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:20.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 -- 172.21.15.33:0/1770739677 shutdown_connections 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 --2- 172.21.15.33:0/1770739677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57781029a0 0x7f57781092c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 --2- 172.21.15.33:0/1770739677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5778101fe0 0x7f5778102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 --2- 172.21.15.33:0/1770739677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5778100e90 0x7f57781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 -- 172.21.15.33:0/1770739677 >> 172.21.15.33:0/1770739677 conn(0x7f57780fc400 msgr2=0x7f57780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 -- 172.21.15.33:0/1770739677 shutdown_connections 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.291+0000 7f577f023640 1 -- 172.21.15.33:0/1770739677 wait complete. 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 Processor -- start 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 -- start start 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5778100e90 0x7f57781981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57781029a0 0x7f577819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f577810cab0 con 0x7f5778101fe0 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f577810c930 con 0x7f57781029a0 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577f023640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f577810cc30 con 0x7f5778100e90 2024-04-23T18:26:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f5777fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f5777fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48648/0 (socket says 172.21.15.33:48648) 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f5777fff640 1 -- 172.21.15.33:0/2926303315 learned_addr learned my addr 172.21.15.33:0/2926303315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.292+0000 7f577d5c0640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57781029a0 0x7f577819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 -- 172.21.15.33:0/2926303315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5778100e90 msgr2=0x7f57781981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5778100e90 0x7f57781981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 -- 172.21.15.33:0/2926303315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57781029a0 msgr2=0x7f577819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57781029a0 0x7f577819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 -- 172.21.15.33:0/2926303315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577819fe10 con 0x7f5778101fe0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5777fff640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f576800b450 tx=0x7f576800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768015020 con 0x7f5778101fe0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5768011d40 con 0x7f5778101fe0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57781a0100 con 0x7f5778101fe0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576800f340 con 0x7f5778101fe0 2024-04-23T18:26:20.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.293+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57781a0720 con 0x7f5778101fe0 2024-04-23T18:26:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.294+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57781a09e0 con 0x7f5778101fe0 2024-04-23T18:26:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.295+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5768018020 con 0x7f5778101fe0 2024-04-23T18:26:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.296+0000 7f5775ffb640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.296+0000 7f577cdbf640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.296+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f57680cff00 con 0x7f5778101fe0 2024-04-23T18:26:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.296+0000 7f577cdbf640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 0x7f57640a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5770007be0 tx=0x7f5770005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.300+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f576801f020 con 0x7f5778101fe0 2024-04-23T18:26:20.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:20 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/633846868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:20 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3705530569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:20 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3980167894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:20 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/633846868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:20 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3705530569' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:20 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3980167894' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:20.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.487+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57781a0bb0 con 0x7f5778101fe0 2024-04-23T18:26:20.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.489+0000 7f5775ffb640 1 -- 172.21.15.33:0/2926303315 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+22724 (secure 0 0 0) 0x7f576800d040 con 0x7f5778101fe0 2024-04-23T18:26:20.482 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:20.482 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:25:25.905871+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 msgr2=0x7f57640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 0x7f57640a60f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5770007be0 tx=0x7f5770005f70 comp rx=0 tx=0).stop 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 msgr2=0x7f5778198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f576800b450 tx=0x7f576800b910 comp rx=0 tx=0).stop 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 shutdown_connections 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f57640a3c30 0x7f57640a60f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57781029a0 0x7f577819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5778101fe0 0x7f5778198730 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 --2- 172.21.15.33:0/2926303315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5778100e90 0x7f57781981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 >> 172.21.15.33:0/2926303315 conn(0x7f57780fc400 msgr2=0x7f57780fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 shutdown_connections 2024-04-23T18:26:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.491+0000 7f577f023640 1 -- 172.21.15.33:0/2926303315 wait complete. 2024-04-23T18:26:20.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:24:26.661481+0000', 'modified': '2024-04-23T18:24:34.355479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24551, 'mds_1': 24515, 'mds_2': 24490, 'mds_3': 24482, 'mds_4': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/1373616976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 1373616976}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 1373616976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/363309287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 363309287}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 363309287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6833/2194426420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 2194426420}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 2194426420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6833/637332963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 637332963}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 637332963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6835/3209242154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 3209242154}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 3209242154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.122:6837/3846576090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 3846576090}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 3846576090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.106:6835/3607877814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3607877814}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3607877814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6839/538103378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 538103378}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 538103378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.106:6837/319312146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 319312146}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 319312146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/830372376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 830372376}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 830372376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24490, 24551, 24482, 24515, 24466]} 2024-04-23T18:26:20.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:26:20.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:26:20.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 -- 172.21.15.33:0/1689975264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0102af0 msgr2=0x7f3ab0104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 --2- 172.21.15.33:0/1689975264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0102af0 0x7f3ab0104ee0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 -- 172.21.15.33:0/1689975264 shutdown_connections 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 --2- 172.21.15.33:0/1689975264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 0x7f3ab010a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 --2- 172.21.15.33:0/1689975264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0105420 0x7f3ab0107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 --2- 172.21.15.33:0/1689975264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0102af0 0x7f3ab0104ee0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.606+0000 7f3ab730f640 1 -- 172.21.15.33:0/1689975264 >> 172.21.15.33:0/1689975264 conn(0x7f3ab00fc440 msgr2=0x7f3ab00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- 172.21.15.33:0/1689975264 shutdown_connections 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- 172.21.15.33:0/1689975264 wait complete. 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 Processor -- start 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- start start 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0102af0 0x7f3ab01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 0x7f3ab019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010db60 con 0x7f3ab0105420 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010d9e0 con 0x7f3ab0102af0 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab730f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab010dce0 con 0x7f3ab0107d50 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48658/0 (socket says 172.21.15.33:48658) 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.607+0000 7f3ab48aa640 1 -- 172.21.15.33:0/2792379688 learned_addr learned my addr 172.21.15.33:0/2792379688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 0x7f3ab019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0102af0 0x7f3ab01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 -- 172.21.15.33:0/2792379688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 msgr2=0x7f3ab019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 0x7f3ab019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 -- 172.21.15.33:0/2792379688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0102af0 msgr2=0x7f3ab01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0102af0 0x7f3ab01982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 -- 172.21.15.33:0/2792379688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab019fec0 con 0x7f3ab0105420 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d770 tx=0x7f3aa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3aa67fc640 1 -- 172.21.15.33:0/2792379688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab0105420 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3aa67fc640 1 -- 172.21.15.33:0/2792379688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0009c30 con 0x7f3ab0105420 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab01a01b0 con 0x7f3ab0105420 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3aa67fc640 1 -- 172.21.15.33:0/2792379688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000f620 con 0x7f3ab0105420 2024-04-23T18:26:20.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.608+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab01a0780 con 0x7f3ab0105420 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aa67fc640 1 -- 172.21.15.33:0/2792379688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3aa001a070 con 0x7f3ab0105420 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 Processor -- start 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 -- start start 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663ee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c466346c50 con 0x55c4663e5f90 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4662e3830 con 0x55c4663e7160 2024-04-23T18:26:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.611+0000 7f3aba022740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c466342c10 con 0x55c4663e7b20 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48670/0 (socket says 172.21.15.33:48670) 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 -- 172.21.15.33:0/17090658 learned_addr learned my addr 172.21.15.33:0/17090658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663ee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3aa67fc640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 0x7f3a9c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 -- 172.21.15.33:0/17090658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 msgr2=0x55c4663ee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663ee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 -- 172.21.15.33:0/17090658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 msgr2=0x55c4663e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663e75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 -- 172.21.15.33:0/17090658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4660592d0 con 0x55c4663e5f90 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3aa67fc640 1 -- 172.21.15.33:0/2792379688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f3aa00d0050 con 0x7f3ab0105420 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 0x7f3a9c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b8e0 tx=0x7f3aa800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff592f72731326a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3a9b7fe640 1 -- 172.21.15.33:0/17090658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8016be0 con 0x55c4663e5f90 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 0x7f3a9c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801e000 tx=0x7f3aa801f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.612+0000 7f3a9b7fe640 1 -- 172.21.15.33:0/17090658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa8016d80 con 0x55c4663e5f90 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3a9b7fe640 1 -- 172.21.15.33:0/17090658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8015df0 con 0x55c4663e5f90 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 -- 172.21.15.33:0/17090658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 msgr2=0x55c4663e63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b8e0 tx=0x7f3aa800bda0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 -- 172.21.15.33:0/17090658 shutdown_connections 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663ee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 --2- 172.21.15.33:0/17090658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e63b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 -- 172.21.15.33:0/17090658 >> 172.21.15.33:0/17090658 conn(0x55c4663dd070 msgr2=0x55c4663dd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 -- 172.21.15.33:0/17090658 shutdown_connections 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.613+0000 7f3aba022740 1 -- 172.21.15.33:0/17090658 wait complete. 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 -- start start 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 Processor -- start 2024-04-23T18:26:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 0x55c4663f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4663eeb30 con 0x55c4663e7160 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4662e1470 con 0x55c4663e5f90 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4663eecb0 con 0x55c4663e7b20 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3aba022740 10 client.0 _ll_register_callbacks cb 0x55c466306f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48680/0 (socket says 172.21.15.33:48680) 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 -- 172.21.15.33:0/1360225772 learned_addr learned my addr 172.21.15.33:0/1360225772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 0x55c4663f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 -- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 msgr2=0x55c4663f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 -- 172.21.15.33:0/1360225772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 msgr2=0x55c4663f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 0x55c4663f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 -- 172.21.15.33:0/1360225772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4661236b0 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 0x55c4663f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.614+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009610 tx=0x7f3aa0010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e99441fe6609e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0021040 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f3aa00d2110 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa8016d80 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0096e60 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c4663fadb0 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa8016d80 con 0x55c4663e7160 2024-04-23T18:26:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3aa00d3c00 con 0x55c4663e7160 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 10 client.15255 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 20 client.15255 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c4663fd3e0 0x55c4663fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 4 client.15255 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.615+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c466354670 con 0x55c4663fd3e0 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.616+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c4663fd3e0 0x55c4663fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.616+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c4663fd3e0 0x55c4663fe850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77ed2feb8a171fd5 server_cookie=7e5ce59a105a3806 in_seq=0 out_seq=0 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.616+0000 7f3a98ff9640 10 client.15255 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3a98ff9640 1 -- 172.21.15.33:0/1360225772 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c466354670 con 0x55c4663fd3e0 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3a98ff9640 10 client.15255 handle_command_reply: tid=0 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 client.15255 shutdown 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 msgr2=0x55c4663f0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009610 tx=0x7f3aa0010c00 comp rx=0 tx=0).stop 2024-04-23T18:26:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 shutdown_connections 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c4663fd3e0 0x55c4663fe850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f0990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3a98ff9640 0 client.15255 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 --2- 172.21.15.33:0/1360225772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e5f90 0x55c4663f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 >> 172.21.15.33:0/1360225772 conn(0x55c4663dd070 msgr2=0x55c4663dc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 shutdown_connections 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 1 -- 172.21.15.33:0/1360225772 wait complete. 2024-04-23T18:26:20.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.617+0000 7f3aba022740 20 client.15255 trim_cache size 0 max 16384 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 Processor -- start 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 -- start start 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4663fcb90 con 0x55c4663e5f90 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4663fd260 con 0x55c4663e7160 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3aba022740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4663fd930 con 0x55c4663e7b20 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48690/0 (socket says 172.21.15.33:48690) 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 -- 172.21.15.33:0/2891659410 learned_addr learned my addr 172.21.15.33:0/2891659410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 -- 172.21.15.33:0/2891659410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 msgr2=0x55c4663f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 -- 172.21.15.33:0/2891659410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 msgr2=0x55c4663f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 -- 172.21.15.33:0/2891659410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4663edf50 con 0x55c4663e5f90 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3aac004af0 tx=0x7f3aac02afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df46c825c2fde41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3a9affd640 1 -- 172.21.15.33:0/2891659410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac031790 con 0x55c4663e5f90 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3a9affd640 1 -- 172.21.15.33:0/2891659410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x55c4663e5f90 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.636+0000 7f3a9affd640 1 -- 172.21.15.33:0/2891659410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004280 con 0x55c4663e5f90 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- 172.21.15.33:0/2891659410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 msgr2=0x55c4663e7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3aac004af0 tx=0x7f3aac02afd0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- 172.21.15.33:0/2891659410 shutdown_connections 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e7b20 0x55c4663f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7160 0x55c4663f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 --2- 172.21.15.33:0/2891659410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e5f90 0x55c4663e7540 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- 172.21.15.33:0/2891659410 >> 172.21.15.33:0/2891659410 conn(0x55c4663dd070 msgr2=0x55c4663d80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- 172.21.15.33:0/2891659410 shutdown_connections 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- 172.21.15.33:0/2891659410 wait complete. 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 -- start start 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.637+0000 7f3aba022740 1 Processor -- start 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e5f90 0x55c4663f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7b20 0x55c4663f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c4663fe6d0 con 0x55c4663e7160 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4663fe000 con 0x55c4663e7b20 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4663feda0 con 0x55c4663e5f90 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3aba022740 10 client.0 _ll_register_callbacks cb 0x7f3aa0010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48694/0 (socket says 172.21.15.33:48694) 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 -- 172.21.15.33:0/175373251 learned_addr learned my addr 172.21.15.33:0/175373251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e5f90 0x55c4663f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab58ac640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7b20 0x55c4663f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 -- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e5f90 msgr2=0x55c4663f3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e5f90 0x55c4663f3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 -- 172.21.15.33:0/175373251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7b20 msgr2=0x55c4663f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7b20 0x55c4663f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 -- 172.21.15.33:0/175373251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4663f7310 con 0x55c4663e7160 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3ab48aa640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0000c00 tx=0x7f3aa00d37f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a8f368e1672b60 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.638+0000 7f3a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0014020 con 0x55c4663e7160 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f3aa00d3180 con 0x55c4663e7160 2024-04-23T18:26:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac036040 con 0x55c4663e7160 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3aa0021070 con 0x55c4663e7160 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c466463b00 con 0x55c4663e7160 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x55c4663e7160 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3aa009b1b0 con 0x55c4663e7160 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 10 client.15261 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 20 client.15261 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c466468650 0x55c466467360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 4 client.15261 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.639+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "status"}) -- 0x55c46634e4e0 con 0x55c466468650 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c466468650 0x55c466467360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3a98ff9640 10 client.15261 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3ab50ab640 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c466468650 0x55c466467360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bbf251b9d3e09bbe server_cookie=69ba1dbbcd053f64 in_seq=0 out_seq=0 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3a98ff9640 1 -- 172.21.15.33:0/175373251 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c46634e4e0 con 0x55c466468650 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3a98ff9640 10 client.15261 handle_command_reply: tid=0 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 client.15261 shutdown 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 msgr2=0x55c4663f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0000c00 tx=0x7f3aa00d37f0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 shutdown_connections 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c466468650 0x55c466467360 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4663e7b20 0x55c4663f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4663e7160 0x55c4663f4100 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3a98ff9640 0 client.15261 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 --2- 172.21.15.33:0/175373251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4663e5f90 0x55c4663f3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.640+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 >> 172.21.15.33:0/175373251 conn(0x55c4663dd070 msgr2=0x55c4663f0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 shutdown_connections 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3aba022740 1 -- 172.21.15.33:0/175373251 wait complete. 2024-04-23T18:26:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3aba022740 20 client.15261 trim_cache size 0 max 16384 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24466, 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 107.30161174200001, 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:26:20.640 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 132.02512345100001 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 msgr2=0x7f3a9c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 0x7f3a9c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3aa801e000 tx=0x7f3aa801f000 comp rx=0 tx=0).stop 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 msgr2=0x7f3ab01987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d770 tx=0x7f3aa000dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 shutdown_connections 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f3a9c0a3bb0 0x7f3a9c0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab0107d50 0x7f3ab019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0105420 0x7f3ab01987e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 --2- 172.21.15.33:0/2792379688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ab0102af0 0x7f3ab01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 >> 172.21.15.33:0/2792379688 conn(0x7f3ab00fc440 msgr2=0x7f3ab0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 shutdown_connections 2024-04-23T18:26:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.641+0000 7f3ab730f640 1 -- 172.21.15.33:0/2792379688 wait complete. 2024-04-23T18:26:20.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-23T18:26:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 -- 172.21.15.33:0/2762780869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 msgr2=0x7f0c2c074380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2762780869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c074380 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c2002a480 comp rx=0 tx=0).stop 2024-04-23T18:26:20.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 -- 172.21.15.33:0/2762780869 shutdown_connections 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2762780869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c109e50 0x7f0c2c10c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2762780869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c074950 0x7f0c2c074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2762780869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c074380 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 -- 172.21.15.33:0/2762780869 >> 172.21.15.33:0/2762780869 conn(0x7f0c2c0fd530 msgr2=0x7f0c2c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 -- 172.21.15.33:0/2762780869 shutdown_connections 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.754+0000 7f0c32a20640 1 -- 172.21.15.33:0/2762780869 wait complete. 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 Processor -- start 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 -- start start 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c074950 0x7f0c2c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c109e50 0x7f0c2c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c10fd50 con 0x7f0c2c075f30 2024-04-23T18:26:20.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c10fbd0 con 0x7f0c2c074950 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c32a20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c10fed0 con 0x7f0c2c109e50 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c074950 0x7f0c2c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48720/0 (socket says 172.21.15.33:48720) 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c2b7fe640 1 -- 172.21.15.33:0/2492828293 learned_addr learned my addr 172.21.15.33:0/2492828293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.755+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c109e50 0x7f0c2c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 -- 172.21.15.33:0/2492828293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c109e50 msgr2=0x7f0c2c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c109e50 0x7f0c2c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 -- 172.21.15.33:0/2492828293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c074950 msgr2=0x7f0c2c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c074950 0x7f0c2c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 -- 172.21.15.33:0/2492828293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c1a1ff0 con 0x7f0c2c075f30 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c890 tx=0x7f0c1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c297fa640 1 -- 172.21.15.33:0/2492828293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c004220 con 0x7f0c2c075f30 2024-04-23T18:26:20.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c297fa640 1 -- 172.21.15.33:0/2492828293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c1c0043c0 con 0x7f0c2c075f30 2024-04-23T18:26:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c1a22e0 con 0x7f0c2c075f30 2024-04-23T18:26:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c297fa640 1 -- 172.21.15.33:0/2492828293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c004a00 con 0x7f0c2c075f30 2024-04-23T18:26:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.756+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c1a2830 con 0x7f0c2c075f30 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.758+0000 7f0c297fa640 1 -- 172.21.15.33:0/2492828293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c1c0079b0 con 0x7f0c2c075f30 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.758+0000 7f0c3580f740 1 Processor -- start 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.758+0000 7f0c3580f740 1 -- start start 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x56395156b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951571fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639514cec50 con 0x563951569b80 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56395146b830 con 0x56395156ad50 2024-04-23T18:26:20.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c3580f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639514cac10 con 0x56395156b710 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48726/0 (socket says 172.21.15.33:48726) 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 -- 172.21.15.33:0/986053546 learned_addr learned my addr 172.21.15.33:0/986053546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x56395156b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c297fa640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 0x7f0c140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951571fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 -- 172.21.15.33:0/986053546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 msgr2=0x563951571fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951571fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 -- 172.21.15.33:0/986053546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 msgr2=0x56395156b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x56395156b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 -- 172.21.15.33:0/986053546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639511e12d0 con 0x563951569b80 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c297fa640 1 -- 172.21.15.33:0/2492828293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f0c1c0cfff0 con 0x7f0c2c075f30 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae90 tx=0x7f0c2400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d0b6f599f73dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 0x7f0c140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.759+0000 7f0c127fc640 1 -- 172.21.15.33:0/986053546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24004c80 con 0x563951569b80 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c127fc640 1 -- 172.21.15.33:0/986053546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c24010c10 con 0x563951569b80 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c127fc640 1 -- 172.21.15.33:0/986053546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24018430 con 0x563951569b80 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 -- 172.21.15.33:0/986053546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 msgr2=0x563951569fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 0x7f0c140a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c24019000 tx=0x7f0c24018ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae90 tx=0x7f0c2400f790 comp rx=0 tx=0).stop 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 -- 172.21.15.33:0/986053546 shutdown_connections 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951571fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x56395156b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 --2- 172.21.15.33:0/986053546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x563951569fa0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 -- 172.21.15.33:0/986053546 >> 172.21.15.33:0/986053546 conn(0x563951565070 msgr2=0x563951565470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 -- 172.21.15.33:0/986053546 shutdown_connections 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.760+0000 7f0c3580f740 1 -- 172.21.15.33:0/986053546 wait complete. 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 -- start start 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 Processor -- start 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 0x563951573f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951574480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563951572720 con 0x56395156b710 2024-04-23T18:26:20.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563951469470 con 0x56395156ad50 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639515728a0 con 0x563951569b80 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c3580f740 10 client.0 _ll_register_callbacks cb 0x56395148ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48734/0 (socket says 172.21.15.33:48734) 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 -- 172.21.15.33:0/1607199335 learned_addr learned my addr 172.21.15.33:0/1607199335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 0x563951573f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 -- 172.21.15.33:0/1607199335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 msgr2=0x563951573f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 0x563951573f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 -- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 msgr2=0x563951574480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951574480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c30fbd640 1 -- 172.21.15.33:0/1607199335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639512ab6b0 con 0x56395156b710 2024-04-23T18:26:20.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.761+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 0x563951573f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae60 tx=0x7f0c24022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af1c43f4165832 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24035060 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f0c24018430 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c24010c10 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c24010dc0 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56395157e8a0 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c24010c10 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0c2403a070 con 0x56395156b710 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 10 client.15273 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 20 client.15273 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x563951580c50 0x5639515820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 4 client.15273 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.762+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639514dc670 con 0x563951580c50 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.763+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x563951580c50 0x5639515820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.763+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x563951580c50 0x5639515820c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4869f45a26ac00e4 server_cookie=b1a59513152893e7 in_seq=0 out_seq=0 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.763+0000 7f0c0b7fe640 10 client.15273 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/1607199335 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639514dc670 con 0x563951580c50 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c0b7fe640 10 client.15273 handle_command_reply: tid=0 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 client.15273 shutdown 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 msgr2=0x56395157b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae60 tx=0x7f0c24022000 comp rx=0 tx=0).stop 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 shutdown_connections 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x563951580c50 0x5639515820c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395157b4c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951574480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 --2- 172.21.15.33:0/1607199335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563951569b80 0x563951573f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 >> 172.21.15.33:0/1607199335 conn(0x563951565070 msgr2=0x5639515641d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c0b7fe640 0 client.15273 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 shutdown_connections 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 1 -- 172.21.15.33:0/1607199335 wait complete. 2024-04-23T18:26:20.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.764+0000 7f0c3580f740 20 client.15273 trim_cache size 0 max 16384 2024-04-23T18:26:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.782+0000 7f0c3580f740 1 Processor -- start 2024-04-23T18:26:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 -- start start 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563951569b80 0x563951571dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156ad50 0x56395157b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563951580400 con 0x56395156b710 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563951580ad0 con 0x563951569b80 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c3580f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639515811a0 con 0x56395156ad50 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48748/0 (socket says 172.21.15.33:48748) 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 -- 172.21.15.33:0/2239094453 learned_addr learned my addr 172.21.15.33:0/2239094453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563951569b80 0x563951571dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 -- 172.21.15.33:0/2239094453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156ad50 msgr2=0x56395157b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156ad50 0x56395157b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 -- 172.21.15.33:0/2239094453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563951569b80 msgr2=0x563951571dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563951569b80 0x563951571dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 -- 172.21.15.33:0/2239094453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563951572300 con 0x56395156b710 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c30fbd640 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae90 tx=0x7f0c2400a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8b2ef96f69a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c11ffb640 1 -- 172.21.15.33:0/2239094453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24040040 con 0x56395156b710 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c11ffb640 1 -- 172.21.15.33:0/2239094453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c2402c040 con 0x56395156b710 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.783+0000 7f0c11ffb640 1 -- 172.21.15.33:0/2239094453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c24035040 con 0x56395156b710 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- 172.21.15.33:0/2239094453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 msgr2=0x56395156e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400ae90 tx=0x7f0c2400a4d0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- 172.21.15.33:0/2239094453 shutdown_connections 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56395156b710 0x56395156e750 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156ad50 0x56395157b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 --2- 172.21.15.33:0/2239094453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563951569b80 0x563951571dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- 172.21.15.33:0/2239094453 >> 172.21.15.33:0/2239094453 conn(0x563951565070 msgr2=0x563951565450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- 172.21.15.33:0/2239094453 shutdown_connections 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- 172.21.15.33:0/2239094453 wait complete. 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.784+0000 7f0c3580f740 1 -- start start 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 Processor -- start 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951576b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951579a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563951581f40 con 0x563951569b80 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563951581870 con 0x56395156ad50 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563951582610 con 0x56395156b710 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 10 client.0 _ll_register_callbacks cb 0x7f0c20005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48750/0 (socket says 172.21.15.33:48750) 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 -- 172.21.15.33:0/537078909 learned_addr learned my addr 172.21.15.33:0/537078909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 -- 172.21.15.33:0/537078909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 msgr2=0x563951579a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951579a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 -- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 msgr2=0x563951576b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951576b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 -- 172.21.15.33:0/537078909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56395157a070 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c2bfff640 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c20004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422c72aa5f876d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20049070 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f0c200454c0 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.785+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2402c040 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0c20055050 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56395157b410 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c2402c040 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0c200315a0 con 0x563951569b80 2024-04-23T18:26:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 10 client.15279 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 20 client.15279 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x5639515eb290 0x5639515e9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 4 client.15279 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "status"}) -- 0x5639514d64e0 con 0x5639515eb290 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.786+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x5639515eb290 0x5639515e9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c2b7fe640 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x5639515eb290 0x5639515e9fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6cb418a3987c1 server_cookie=cf32eee974fdf105 in_seq=0 out_seq=0 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c0b7fe640 10 client.15279 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c0b7fe640 1 -- 172.21.15.33:0/537078909 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5639514d64e0 con 0x5639515eb290 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c0b7fe640 10 client.15279 handle_command_reply: tid=0 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 client.15279 shutdown 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 msgr2=0x5639515765c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0c20009900 tx=0x7f0c20004560 comp rx=0 tx=0).stop 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 shutdown_connections 2024-04-23T18:26:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x5639515eb290 0x5639515e9fa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56395156b710 0x563951579a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56395156ad50 0x563951576b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c0b7fe640 0 client.15279 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 --2- 172.21.15.33:0/537078909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563951569b80 0x5639515765c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 >> 172.21.15.33:0/537078909 conn(0x563951565070 msgr2=0x563951573f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 shutdown_connections 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 1 -- 172.21.15.33:0/537078909 wait complete. 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.787+0000 7f0c3580f740 20 client.15279 trim_cache size 0 max 16384 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:20.786 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24482, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 108.454913976, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 137.79678279000001 2024-04-23T18:26:20.787 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 msgr2=0x7f0c140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 0x7f0c140a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c24019000 tx=0x7f0c24018ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 msgr2=0x7f0c2c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00c890 tx=0x7f0c1c00cd50 comp rx=0 tx=0).stop 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 shutdown_connections 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f0c140a3bb0 0x7f0c140a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c2c109e50 0x7f0c2c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c2c075f30 0x7f0c2c19a910 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 --2- 172.21.15.33:0/2492828293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c2c074950 0x7f0c2c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 >> 172.21.15.33:0/2492828293 conn(0x7f0c2c0fd530 msgr2=0x7f0c2c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 shutdown_connections 2024-04-23T18:26:20.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.788+0000 7f0c32a20640 1 -- 172.21.15.33:0/2492828293 wait complete. 2024-04-23T18:26:20.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- 172.21.15.33:0/3706706775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4107c70 msgr2=0x7ff6c410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/3706706775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4107c70 0x7ff6c410a0c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009900 tx=0x7ff6bc02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- 172.21.15.33:0/3706706775 shutdown_connections 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/3706706775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4107c70 0x7ff6c410a0c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/3706706775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4105340 0x7ff6c4107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/3706706775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4102a10 0x7ff6c4104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- 172.21.15.33:0/3706706775 >> 172.21.15.33:0/3706706775 conn(0x7ff6c40fc400 msgr2=0x7ff6c40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- 172.21.15.33:0/3706706775 shutdown_connections 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- 172.21.15.33:0/3706706775 wait complete. 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 Processor -- start 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.902+0000 7ff6c8cae640 1 -- start start 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 0x7ff6c41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4107c70 0x7ff6c419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410db30 con 0x7ff6c4105340 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410d9b0 con 0x7ff6c4102a10 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c8cae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c410dcb0 con 0x7ff6c4107c70 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48762/0 (socket says 172.21.15.33:48762) 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/4260535076 learned_addr learned my addr 172.21.15.33:0/4260535076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c259c640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 0x7ff6c41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/4260535076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4107c70 msgr2=0x7ff6c419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4107c70 0x7ff6c419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4107c70 0x7ff6c419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/4260535076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 msgr2=0x7ff6c41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 0x7ff6c41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/4260535076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c419fdc0 con 0x7ff6c4105340 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c259c640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 0x7ff6c41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c970 tx=0x7ff6ac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.903+0000 7ff6bb7fe640 1 -- 172.21.15.33:0/4260535076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac007bb0 con 0x7ff6c4105340 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.904+0000 7ff6bb7fe640 1 -- 172.21.15.33:0/4260535076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac007d50 con 0x7ff6c4105340 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.904+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a00b0 con 0x7ff6c4105340 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.904+0000 7ff6bb7fe640 1 -- 172.21.15.33:0/4260535076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00f450 con 0x7ff6c4105340 2024-04-23T18:26:20.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.904+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a0650 con 0x7ff6c4105340 2024-04-23T18:26:20.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 Processor -- start 2024-04-23T18:26:20.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6bb7fe640 1 -- 172.21.15.33:0/4260535076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff6ac004f20 con 0x7ff6c4105340 2024-04-23T18:26:20.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 -- start start 2024-04-23T18:26:20.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e0cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e0e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ae1d71c50 con 0x560ae1e0e710 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ae1d0e830 con 0x560ae1e0cb80 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.906+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ae1d6dc10 con 0x560ae1e0dd50 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48772/0 (socket says 172.21.15.33:48772) 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/355226183 learned_addr learned my addr 172.21.15.33:0/355226183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c259c640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e0e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e0cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6bb7fe640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 0x7ff6b00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/355226183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 msgr2=0x560ae1e0e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e0e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/355226183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 msgr2=0x560ae1e0cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e0cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/355226183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae1a842d0 con 0x560ae1e0e710 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e0cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 0x7ff6b00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6bb7fe640 1 -- 172.21.15.33:0/4260535076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7ff6ac0d03a0 con 0x7ff6c4105340 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0a0310 tx=0x7ff6ac098750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4f2fa78a48333c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6b8ff9640 1 -- 172.21.15.33:0/355226183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac01d030 con 0x560ae1e0e710 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6b8ff9640 1 -- 172.21.15.33:0/355226183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac09ff10 con 0x560ae1e0e710 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.907+0000 7ff6b8ff9640 1 -- 172.21.15.33:0/355226183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0a0170 con 0x560ae1e0e710 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/355226183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 msgr2=0x560ae1e14fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 0x7ff6b00a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc006010 tx=0x7ff6bc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0a0310 tx=0x7ff6ac098750 comp rx=0 tx=0).stop 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/355226183 shutdown_connections 2024-04-23T18:26:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e14fa0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e0e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/355226183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e0cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/355226183 >> 172.21.15.33:0/355226183 conn(0x560ae1e08070 msgr2=0x560ae1e08470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/355226183 shutdown_connections 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/355226183 wait complete. 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.908+0000 7ff6cb9c8740 1 -- start start 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 Processor -- start 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e17000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e17540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e15720 con 0x560ae1e0e710 2024-04-23T18:26:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ae1d0c470 con 0x560ae1e0cb80 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e158a0 con 0x560ae1e0dd50 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6cb9c8740 10 client.0 _ll_register_callbacks cb 0x560ae1d31f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48782/0 (socket says 172.21.15.33:48782) 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 -- 172.21.15.33:0/1411710527 learned_addr learned my addr 172.21.15.33:0/1411710527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e17540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c259c640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e17000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 -- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 msgr2=0x560ae1e17540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e17540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 -- 172.21.15.33:0/1411710527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 msgr2=0x560ae1e17000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e17000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 -- 172.21.15.33:0/1411710527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae1b4e6b0 con 0x560ae1e0e710 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c259c640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc033000 tx=0x7ff6bc0314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e8fa8ef79252bc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.909+0000 7ff6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc04d020 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7ff6bc0035f0 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ae1d7f670 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc035040 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ae1e21960 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ae1d7f670 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6bc031630 con 0x560ae1e0e710 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 20 client.15291 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e23ff0 0x560ae1e25460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 4 client.15291 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.910+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ae1d794e0 con 0x560ae1e23ff0 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.911+0000 7ff6c259c640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e23ff0 0x560ae1e25460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.911+0000 7ff6c259c640 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e23ff0 0x560ae1e25460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed0528a7274efa88 server_cookie=23bef4698c454af6 in_seq=0 out_seq=0 2024-04-23T18:26:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.911+0000 7ff6a9ffb640 10 client.15291 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/1411710527 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ae1d794e0 con 0x560ae1e23ff0 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6a9ffb640 10 client.15291 handle_command_reply: tid=0 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 client.15291 shutdown 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 msgr2=0x560ae1e1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc033000 tx=0x7ff6bc0314c0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 shutdown_connections 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e23ff0 0x560ae1e25460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0e710 0x560ae1e1e580 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0dd50 0x560ae1e17540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/1411710527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0cb80 0x560ae1e17000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6a9ffb640 0 client.15291 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 >> 172.21.15.33:0/1411710527 conn(0x560ae1e08070 msgr2=0x560ae1e071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 shutdown_connections 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/1411710527 wait complete. 2024-04-23T18:26:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.912+0000 7ff6cb9c8740 20 client.15291 trim_cache size 0 max 16384 2024-04-23T18:26:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 Processor -- start 2024-04-23T18:26:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 -- start start 2024-04-23T18:26:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e237a0 con 0x560ae1e0dd50 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e23e70 con 0x560ae1e0e710 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e24540 con 0x560ae1e0cb80 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48796/0 (socket says 172.21.15.33:48796) 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/46129331 learned_addr learned my addr 172.21.15.33:0/46129331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/46129331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 msgr2=0x560ae1e14e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/46129331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 msgr2=0x560ae1e11db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.930+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/46129331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae1e0d7d0 con 0x560ae1e0dd50 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0988c0 tx=0x7ff6ac004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e959f83c7e1ff5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6abfff640 1 -- 172.21.15.33:0/46129331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac01d040 con 0x560ae1e0dd50 2024-04-23T18:26:20.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6abfff640 1 -- 172.21.15.33:0/46129331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac09cd80 con 0x560ae1e0dd50 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6abfff640 1 -- 172.21.15.33:0/46129331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac0870d0 con 0x560ae1e0dd50 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/46129331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 msgr2=0x560ae1e11870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0988c0 tx=0x7ff6ac004980 comp rx=0 tx=0).stop 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/46129331 shutdown_connections 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e11db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e11870 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/46129331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e14e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/46129331 >> 172.21.15.33:0/46129331 conn(0x560ae1e08070 msgr2=0x560ae1e030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/46129331 shutdown_connections 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.931+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/46129331 wait complete. 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 -- start start 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 Processor -- start 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e88180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e252e0 con 0x560ae1e0dd50 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e24c10 con 0x560ae1e0e710 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ae1e259b0 con 0x560ae1e0cb80 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6cb9c8740 10 client.0 _ll_register_callbacks cb 0x7ff6bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48802/0 (socket says 172.21.15.33:48802) 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/2700011903 learned_addr learned my addr 172.21.15.33:0/2700011903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c259c640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e88180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 msgr2=0x560ae1e88180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c2d9d640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e88180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/2700011903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 msgr2=0x560ae1e8cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e8cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 -- 172.21.15.33:0/2700011903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae1e8d470 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c259c640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e88180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6c1d9b640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0d3e60 tx=0x7ff6ac0d3950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c515ace40fc0008e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.932+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac09bfe0 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7ff6ac0d3bf0 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6ac09cd80 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff6ac0a52f0 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560ae1e8e770 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac09cd80 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff6ac005210 con 0x560ae1e0dd50 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 20 client.15297 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:20.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e92bd0 0x560ae1e918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 4 client.15297 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "status"}) -- 0x560ae1cf59a0 con 0x560ae1e92bd0 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.933+0000 7ff6c259c640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e92bd0 0x560ae1e918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6a9ffb640 10 client.15297 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6c259c640 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e92bd0 0x560ae1e918e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a5fd44781541ea24 server_cookie=dc38bed632125fe3 in_seq=0 out_seq=0 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6a9ffb640 1 -- 172.21.15.33:0/2700011903 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560ae1cf59a0 con 0x560ae1e92bd0 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6a9ffb640 10 client.15297 handle_command_reply: tid=0 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 client.15297 shutdown 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 msgr2=0x560ae1e8a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0d3e60 tx=0x7ff6ac0d3950 comp rx=0 tx=0).stop 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 shutdown_connections 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x560ae1e92bd0 0x560ae1e918e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ae1e0e710 0x560ae1e8cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ae1e0dd50 0x560ae1e8a6d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 --2- 172.21.15.33:0/2700011903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ae1e0cb80 0x560ae1e88180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 >> 172.21.15.33:0/2700011903 conn(0x560ae1e08070 msgr2=0x560ae1e16e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6a9ffb640 0 client.15297 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 shutdown_connections 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 1 -- 172.21.15.33:0/2700011903 wait complete. 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.934+0000 7ff6cb9c8740 20 client.15297 trim_cache size 0 max 16384 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24490, 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:20.937 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 109.612267948, 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 126.650819289 2024-04-23T18:26:20.938 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 msgr2=0x7ff6b00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 0x7ff6b00a60f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc006010 tx=0x7ff6bc009260 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 msgr2=0x7ff6c41986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00c970 tx=0x7ff6ac00ce30 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 shutdown_connections 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7ff6b00a3c30 0x7ff6b00a60f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6c4107c70 0x7ff6c419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4105340 0x7ff6c41986e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 --2- 172.21.15.33:0/4260535076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6c4102a10 0x7ff6c41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 >> 172.21.15.33:0/4260535076 conn(0x7ff6c40fc400 msgr2=0x7ff6c4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 shutdown_connections 2024-04-23T18:26:20.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:20.935+0000 7ff6c8cae640 1 -- 172.21.15.33:0/4260535076 wait complete. 2024-04-23T18:26:20.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 -- 172.21.15.33:0/3676087570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 msgr2=0x7f40dc104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 --2- 172.21.15.33:0/3676087570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc104e40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009900 tx=0x7f40c402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 -- 172.21.15.33:0/3676087570 shutdown_connections 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 --2- 172.21.15.33:0/3676087570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc107cb0 0x7f40dc10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 --2- 172.21.15.33:0/3676087570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc105380 0x7f40dc107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.052+0000 7f40e27f0640 1 --2- 172.21.15.33:0/3676087570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc104e40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 -- 172.21.15.33:0/3676087570 >> 172.21.15.33:0/3676087570 conn(0x7f40dc0fc400 msgr2=0x7f40dc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 -- 172.21.15.33:0/3676087570 shutdown_connections 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 -- 172.21.15.33:0/3676087570 wait complete. 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 Processor -- start 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 -- start start 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.053+0000 7f40e27f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc105380 0x7f40dc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 0x7f40dc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10dae0 con 0x7f40dc102a50 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10d960 con 0x7f40dc107cb0 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc10dc60 con 0x7f40dc105380 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48824/0 (socket says 172.21.15.33:48824) 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 -- 172.21.15.33:0/1031201883 learned_addr learned my addr 172.21.15.33:0/1031201883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40db7fe640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc105380 0x7f40dc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 0x7f40dc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 -- 172.21.15.33:0/1031201883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc105380 msgr2=0x7f40dc100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc105380 0x7f40dc100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 -- 172.21.15.33:0/1031201883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 msgr2=0x7f40dc100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 0x7f40dc100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 -- 172.21.15.33:0/1031201883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc1017d0 con 0x7f40dc102a50 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 0x7f40dc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40dbfff640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005940 tx=0x7f40c4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40d97fa640 1 -- 172.21.15.33:0/1031201883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c403f030 con 0x7f40dc102a50 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40d97fa640 1 -- 172.21.15.33:0/1031201883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c4003cb0 con 0x7f40dc102a50 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc1a22b0 con 0x7f40dc102a50 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40d97fa640 1 -- 172.21.15.33:0/1031201883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4035070 con 0x7f40dc102a50 2024-04-23T18:26:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.054+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc1a2850 con 0x7f40dc102a50 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40d97fa640 1 -- 172.21.15.33:0/1031201883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f40c4042030 con 0x7f40dc102a50 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 Processor -- start 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 -- start start 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb12519b80 0x55eb12519fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb1251b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb1247ec50 con 0x55eb1251b710 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb1241b830 con 0x55eb12519b80 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e55de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb1247ac10 con 0x55eb1251ad50 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48832/0 (socket says 172.21.15.33:48832) 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 -- 172.21.15.33:0/2683351240 learned_addr learned my addr 172.21.15.33:0/2683351240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb12519b80 0x55eb12519fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40dbfff640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb1251b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 -- 172.21.15.33:0/2683351240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 msgr2=0x55eb1251b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb1251b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 -- 172.21.15.33:0/2683351240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb12519b80 msgr2=0x55eb12519fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40d97fa640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 0x7f40c80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb12519b80 0x55eb12519fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40db7fe640 1 -- 172.21.15.33:0/2683351240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb121912d0 con 0x55eb1251b710 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40dbfff640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb1251b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.057+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 0x7f40c80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40d97fa640 1 -- 172.21.15.33:0/1031201883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f40c40f0ce0 con 0x7f40dc102a50 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40db7fe640 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00bc70 tx=0x7f40cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8b840644e4f351 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40d27fc640 1 -- 172.21.15.33:0/2683351240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc005220 con 0x55eb1251b710 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40d27fc640 1 -- 172.21.15.33:0/2683351240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40cc00be20 con 0x55eb1251b710 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40d27fc640 1 -- 172.21.15.33:0/2683351240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0116a0 con 0x55eb1251b710 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 -- 172.21.15.33:0/2683351240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 msgr2=0x55eb12521fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00bc70 tx=0x7f40cc0107a0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 0x7f40c80a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55eb1251ac50 tx=0x7f40d4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 -- 172.21.15.33:0/2683351240 shutdown_connections 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb1251b710 0x55eb12521fa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb1251b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 --2- 172.21.15.33:0/2683351240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb12519b80 0x55eb12519fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.058+0000 7f40e55de740 1 -- 172.21.15.33:0/2683351240 >> 172.21.15.33:0/2683351240 conn(0x55eb12515070 msgr2=0x55eb12515470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- 172.21.15.33:0/2683351240 shutdown_connections 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- 172.21.15.33:0/2683351240 wait complete. 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- start start 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 Processor -- start 2024-04-23T18:26:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12524540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb12522720 con 0x55eb12519b80 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb12419470 con 0x55eb1251b710 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb125228a0 con 0x55eb1251ad50 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40e55de740 10 client.0 _ll_register_callbacks cb 0x55eb1243ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48840/0 (socket says 172.21.15.33:48840) 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.059+0000 7f40dbfff640 1 -- 172.21.15.33:0/321714771 learned_addr learned my addr 172.21.15.33:0/321714771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40db7fe640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12524540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 -- 172.21.15.33:0/321714771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 msgr2=0x55eb12524540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12524540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 -- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 msgr2=0x55eb1252b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 -- 172.21.15.33:0/321714771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1225b6b0 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40db7fe640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12524540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40dbfff640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40c40c5700 tx=0x7f40c4032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b244bf64b5d2430 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4049040 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f40c40b9630 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40cc00be20 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40c4003e60 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40e55de740 10 client.15309 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb1252e8f0 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40cc00be20 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.060+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f40c40bf020 con 0x55eb12519b80 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:21.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 10 client.15309 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 20 client.15309 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb12531000 0x55eb12532470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 4 client.15309 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1248c670 con 0x55eb12531000 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40db7fe640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb12531000 0x55eb12532470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40db7fe640 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb12531000 0x55eb12532470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=23108644977c8491 server_cookie=b7a34d96a5eb7c35 in_seq=0 out_seq=0 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.061+0000 7f40bb7fe640 10 client.15309 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40bb7fe640 1 -- 172.21.15.33:0/321714771 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb1248c670 con 0x55eb12531000 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40bb7fe640 10 client.15309 handle_command_reply: tid=0 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 client.15309 shutdown 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 msgr2=0x55eb12524000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f40c40c5700 tx=0x7f40c4032f60 comp rx=0 tx=0).stop 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 shutdown_connections 2024-04-23T18:26:21.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb12531000 0x55eb12532470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40bb7fe640 0 client.15309 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12524540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 --2- 172.21.15.33:0/321714771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12524000 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 >> 172.21.15.33:0/321714771 conn(0x55eb12515070 msgr2=0x55eb125141d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 shutdown_connections 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 1 -- 172.21.15.33:0/321714771 wait complete. 2024-04-23T18:26:21.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.062+0000 7f40e55de740 20 client.15309 trim_cache size 0 max 16384 2024-04-23T18:26:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.080+0000 7f40e55de740 1 Processor -- start 2024-04-23T18:26:21.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.080+0000 7f40e55de740 1 -- start start 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251ad50 0x55eb1252b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251b710 0x55eb12530180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb125307b0 con 0x55eb12519b80 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb12530e80 con 0x55eb1251ad50 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40e55de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb12531550 con 0x55eb1251b710 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48844/0 (socket says 172.21.15.33:48844) 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 -- 172.21.15.33:0/3474374765 learned_addr learned my addr 172.21.15.33:0/3474374765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40db7fe640 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251ad50 0x55eb1252b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 -- 172.21.15.33:0/3474374765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251b710 msgr2=0x55eb12530180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251b710 0x55eb12530180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 -- 172.21.15.33:0/3474374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251ad50 msgr2=0x55eb1252b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251ad50 0x55eb1252b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 -- 172.21.15.33:0/3474374765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb12524dc0 con 0x55eb12519b80 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40dbfff640 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005c40 tx=0x7f40c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d42caeba327d913 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40d1ffb640 1 -- 172.21.15.33:0/3474374765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40a0070 con 0x55eb12519b80 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40d1ffb640 1 -- 172.21.15.33:0/3474374765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40f3070 con 0x55eb12519b80 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.081+0000 7f40d1ffb640 1 -- 172.21.15.33:0/3474374765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c4049070 con 0x55eb12519b80 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- 172.21.15.33:0/3474374765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 msgr2=0x55eb1252fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005c40 tx=0x7f40c402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- 172.21.15.33:0/3474374765 shutdown_connections 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251b710 0x55eb12530180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251ad50 0x55eb1252b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 --2- 172.21.15.33:0/3474374765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb1252fc40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- 172.21.15.33:0/3474374765 >> 172.21.15.33:0/3474374765 conn(0x55eb12515070 msgr2=0x55eb125100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- 172.21.15.33:0/3474374765 shutdown_connections 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- 172.21.15.33:0/3474374765 wait complete. 2024-04-23T18:26:21.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 -- start start 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.082+0000 7f40e55de740 1 Processor -- start 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12527270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb125322f0 con 0x55eb12519b80 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb12531c20 con 0x55eb1251b710 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb125329c0 con 0x55eb1251ad50 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 10 client.0 _ll_register_callbacks cb 0x7f40c4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48856/0 (socket says 172.21.15.33:48856) 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 -- 172.21.15.33:0/907224466 learned_addr learned my addr 172.21.15.33:0/907224466 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e0d8d640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40db7fe640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12527270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 -- 172.21.15.33:0/907224466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 msgr2=0x55eb12527270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12527270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 -- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 msgr2=0x55eb1252a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 -- 172.21.15.33:0/907224466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1252a880 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40dbfff640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f40c40bcf50 tx=0x7f40c4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930a8668e3d4f7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c40a0070 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f40c40c59a0 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c40f3070 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f40c4049070 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 10 client.15315 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.083+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eb1252a880 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40c40f3070 con 0x55eb12519b80 2024-04-23T18:26:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f40c40bf040 con 0x55eb12519b80 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 10 client.15315 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 10 client.15315 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 20 client.15315 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb1259b620 0x55eb1259a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 4 client.15315 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "status"}) -- 0x55eb124864e0 con 0x55eb1259b620 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40db7fe640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb1259b620 0x55eb1259a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40bb7fe640 10 client.15315 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.084+0000 7f40db7fe640 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb1259b620 0x55eb1259a330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be8e935a8aa0f9f6 server_cookie=61fea0b99fdd1c2 in_seq=0 out_seq=0 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40bb7fe640 1 -- 172.21.15.33:0/907224466 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55eb124864e0 con 0x55eb1259b620 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40bb7fe640 10 client.15315 handle_command_reply: tid=0 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 client.15315 shutdown 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 msgr2=0x55eb12526d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f40c40bcf50 tx=0x7f40c4009560 comp rx=0 tx=0).stop 2024-04-23T18:26:21.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 shutdown_connections 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eb1259b620 0x55eb1259a330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb1251b710 0x55eb1252a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb1251ad50 0x55eb12527270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40bb7fe640 0 client.15315 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 --2- 172.21.15.33:0/907224466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb12519b80 0x55eb12526d30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 >> 172.21.15.33:0/907224466 conn(0x55eb12515070 msgr2=0x55eb12521ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 shutdown_connections 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 1 -- 172.21.15.33:0/907224466 wait complete. 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e55de740 20 client.15315 trim_cache size 0 max 16384 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:21.084 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24515, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 110.769670756, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 129.57228518400001 2024-04-23T18:26:21.085 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 msgr2=0x7f40c80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.085+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 0x7f40c80a60f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55eb1251ac50 tx=0x7f40d4004a40 comp rx=0 tx=0).stop 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 msgr2=0x7f40dc1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005940 tx=0x7f40c4004430 comp rx=0 tx=0).stop 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 shutdown_connections 2024-04-23T18:26:21.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f40c80a3c30 0x7f40c80a60f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40dc107cb0 0x7f40dc100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40dc105380 0x7f40dc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 --2- 172.21.15.33:0/1031201883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40dc102a50 0x7f40dc1023c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 >> 172.21.15.33:0/1031201883 conn(0x7f40dc0fc400 msgr2=0x7f40dc103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 shutdown_connections 2024-04-23T18:26:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.086+0000 7f40e27f0640 1 -- 172.21.15.33:0/1031201883 wait complete. 2024-04-23T18:26:21.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:21 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2896697687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:21 smithi033 ceph-mon[29274]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:26:21.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:21 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2926303315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:26:21.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.199+0000 7fa2a9329640 1 -- 172.21.15.33:0/3957935804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 msgr2=0x7fa2a41023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.199+0000 7fa2a9329640 1 --2- 172.21.15.33:0/3957935804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41023e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa294009900 tx=0x7fa29402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 -- 172.21.15.33:0/3957935804 shutdown_connections 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 --2- 172.21.15.33:0/3957935804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a4103b70 0x7fa2a410a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 --2- 172.21.15.33:0/3957935804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a41031b0 0x7fa2a4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 --2- 172.21.15.33:0/3957935804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41023e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 -- 172.21.15.33:0/3957935804 >> 172.21.15.33:0/3957935804 conn(0x7fa2a40fd530 msgr2=0x7fa2a40ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 -- 172.21.15.33:0/3957935804 shutdown_connections 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 -- 172.21.15.33:0/3957935804 wait complete. 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 Processor -- start 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 -- start start 2024-04-23T18:26:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.200+0000 7fa2a9329640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a41031b0 0x7fa2a4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 0x7fa2a419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410dac0 con 0x7fa2a4101fc0 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d940 con 0x7fa2a4103b70 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410dc40 con 0x7fa2a41031b0 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48872/0 (socket says 172.21.15.33:48872) 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 -- 172.21.15.33:0/2880664676 learned_addr learned my addr 172.21.15.33:0/2880664676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a41031b0 0x7fa2a4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 0x7fa2a419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 -- 172.21.15.33:0/2880664676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a41031b0 msgr2=0x7fa2a4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a41031b0 0x7fa2a4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 -- 172.21.15.33:0/2880664676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 msgr2=0x7fa2a419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 0x7fa2a419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 -- 172.21.15.33:0/2880664676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a419fe10 con 0x7fa2a4101fc0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 0x7fa2a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa2940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa28bfff640 1 -- 172.21.15.33:0/2880664676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29403f030 con 0x7fa2a4101fc0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a41a00a0 con 0x7fa2a4101fc0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa28bfff640 1 -- 172.21.15.33:0/2880664676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa294031910 con 0x7fa2a4101fc0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa28bfff640 1 -- 172.21.15.33:0/2880664676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294035070 con 0x7fa2a4101fc0 2024-04-23T18:26:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.201+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a0590 con 0x7fa2a4101fc0 2024-04-23T18:26:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa28bfff640 1 -- 172.21.15.33:0/2880664676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa294042030 con 0x7fa2a4101fc0 2024-04-23T18:26:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 Processor -- start 2024-04-23T18:26:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 -- start start 2024-04-23T18:26:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x5567492421d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749248fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5567491a5c50 con 0x556749240b80 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556749142830 con 0x556749241d50 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2ac040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567491a1c10 con 0x556749242710 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48878/0 (socket says 172.21.15.33:48878) 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 -- 172.21.15.33:0/3979984049 learned_addr learned my addr 172.21.15.33:0/3979984049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa28bfff640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 0x7fa28c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x5567492421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749248fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 -- 172.21.15.33:0/3979984049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 msgr2=0x556749248fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749248fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 -- 172.21.15.33:0/3979984049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 msgr2=0x5567492421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.204+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x5567492421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2a37fe640 1 -- 172.21.15.33:0/3979984049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556748eb82d0 con 0x556749240b80 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749248fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 0x7fa28c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa28bfff640 1 -- 172.21.15.33:0/2880664676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fa2940f09a0 con 0x7fa2a4101fc0 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010620 tx=0x7fa29c010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e577d3d5a60d39ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2897fa640 1 -- 172.21.15.33:0/3979984049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c019070 con 0x556749240b80 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2897fa640 1 -- 172.21.15.33:0/3979984049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29c026430 con 0x556749240b80 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.205+0000 7fa2897fa640 1 -- 172.21.15.33:0/3979984049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01d650 con 0x556749240b80 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- 172.21.15.33:0/3979984049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 msgr2=0x556749240fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 0x7fa28c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa29c020000 tx=0x7fa29c01e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa29c010620 tx=0x7fa29c010ae0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- 172.21.15.33:0/3979984049 shutdown_connections 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749248fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x5567492421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- 172.21.15.33:0/3979984049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749240b80 0x556749240fa0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- 172.21.15.33:0/3979984049 >> 172.21.15.33:0/3979984049 conn(0x55674923c070 msgr2=0x55674923c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- 172.21.15.33:0/3979984049 shutdown_connections 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- 172.21.15.33:0/3979984049 wait complete. 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- start start 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 Processor -- start 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674924b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 0x5567492525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556749249720 con 0x556749241d50 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556749140470 con 0x556749242710 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567492498a0 con 0x556749240b80 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2ac040740 10 client.0 _ll_register_callbacks cb 0x556749165f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48894/0 (socket says 172.21.15.33:48894) 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.206+0000 7fa2a27fc640 1 -- 172.21.15.33:0/2929338137 learned_addr learned my addr 172.21.15.33:0/2929338137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674924b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 0x5567492525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 -- 172.21.15.33:0/2929338137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 msgr2=0x55674924b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674924b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 -- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 msgr2=0x5567492525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 0x5567492525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 -- 172.21.15.33:0/2929338137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556748f826b0 con 0x556749241d50 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 0x5567492525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2980136e0 tx=0x7fa298013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b7b7a4de15e545 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298004460 con 0x556749241d50 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fa298011960 con 0x556749241d50 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c026430 con 0x556749241d50 2024-04-23T18:26:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa298011b60 con 0x556749241d50 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2ac040740 10 client.15327 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556749255960 con 0x556749241d50 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.207+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29c026430 con 0x556749241d50 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa29801a070 con 0x556749241d50 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 20 client.15327 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556749258020 0x556749259490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 4 client.15327 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567491b3670 con 0x556749258020 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556749258020 0x556749259490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556749258020 0x556749259490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95744026b070e51f server_cookie=a6f08cc5d9947a04 in_seq=0 out_seq=0 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.208+0000 7fa286ff5640 10 client.15327 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa286ff5640 1 -- 172.21.15.33:0/2929338137 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567491b3670 con 0x556749258020 2024-04-23T18:26:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa286ff5640 10 client.15327 handle_command_reply: tid=0 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 client.15327 shutdown 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 msgr2=0x55674924b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa2980136e0 tx=0x7fa298013ba0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 shutdown_connections 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556749258020 0x556749259490 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749242710 0x5567492525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x55674924b570 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa286ff5640 0 client.15327 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 --2- 172.21.15.33:0/2929338137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674924b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 >> 172.21.15.33:0/2929338137 conn(0x55674923c070 msgr2=0x55674923b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 shutdown_connections 2024-04-23T18:26:21.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 1 -- 172.21.15.33:0/2929338137 wait complete. 2024-04-23T18:26:21.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.209+0000 7fa2ac040740 20 client.15327 trim_cache size 0 max 16384 2024-04-23T18:26:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.227+0000 7fa2ac040740 1 Processor -- start 2024-04-23T18:26:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.227+0000 7fa2ac040740 1 -- start start 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674923ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x556749252570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5567492577d0 con 0x556749242710 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556749257ea0 con 0x556749241d50 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2ac040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556749258570 con 0x556749240b80 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48906/0 (socket says 172.21.15.33:48906) 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 -- 172.21.15.33:0/208947538 learned_addr learned my addr 172.21.15.33:0/208947538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674923ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 -- 172.21.15.33:0/208947538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 msgr2=0x55674923ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674923ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 -- 172.21.15.33:0/208947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 msgr2=0x556749252570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x556749252570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 -- 172.21.15.33:0/208947538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674923cd40 con 0x556749242710 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674923ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005c40 tx=0x7fa29c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad66813f93c799e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa288ff9640 1 -- 172.21.15.33:0/208947538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c019050 con 0x556749242710 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa288ff9640 1 -- 172.21.15.33:0/208947538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29c034070 con 0x556749242710 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.228+0000 7fa288ff9640 1 -- 172.21.15.33:0/208947538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c02b040 con 0x556749242710 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- 172.21.15.33:0/208947538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 msgr2=0x55674923f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005c40 tx=0x7fa29c013000 comp rx=0 tx=0).stop 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- 172.21.15.33:0/208947538 shutdown_connections 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749242710 0x55674923f490 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749241d50 0x556749252570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- 172.21.15.33:0/208947538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749240b80 0x55674923ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- 172.21.15.33:0/208947538 >> 172.21.15.33:0/208947538 conn(0x55674923c070 msgr2=0x5567492370b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- 172.21.15.33:0/208947538 shutdown_connections 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- 172.21.15.33:0/208947538 wait complete. 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- start start 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 Processor -- start 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749240b80 0x5567492507b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749251320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556749259310 con 0x556749241d50 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.229+0000 7fa2ac040740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556749258c40 con 0x556749240b80 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2ac040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567492599e0 con 0x556749242710 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2ac040740 10 client.0 _ll_register_callbacks cb 0x7fa2980097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48920/0 (socket says 172.21.15.33:48920) 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 -- 172.21.15.33:0/4186475909 learned_addr learned my addr 172.21.15.33:0/4186475909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749240b80 0x5567492507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749251320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 -- 172.21.15.33:0/4186475909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 msgr2=0x556749251320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749251320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 -- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749240b80 msgr2=0x5567492507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749240b80 0x5567492507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 -- 172.21.15.33:0/4186475909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556749251b50 con 0x556749241d50 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a37fe640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749251320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2a27fc640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29801c550 tx=0x7fa2980096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbac7b97431db21d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa286ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29801c9d0 con 0x556749241d50 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fa29801cb90 con 0x556749241d50 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29c034070 con 0x556749241d50 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa2980119b0 con 0x556749241d50 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2ac040740 10 client.15333 fetch_fsmap learned FSMap version 27 2024-04-23T18:26:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.230+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5567492be310 con 0x556749241d50 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa29c034070 con 0x556749241d50 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa29801a070 con 0x556749241d50 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 10 client.15333 fetch_fsmap finished waiting for FSMap version 27 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 20 client.15333 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5567492c2f40 0x5567492c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 4 client.15333 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "status"}) -- 0x5567491ad4e0 con 0x5567492c2f40 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5567492c2f40 0x5567492c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa286ff5640 10 client.15333 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.231+0000 7fa2a2ffd640 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5567492c2f40 0x5567492c1c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892ec11f945949dd server_cookie=12ad2fbc857d96cc in_seq=0 out_seq=0 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa286ff5640 1 -- 172.21.15.33:0/4186475909 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5567491ad4e0 con 0x5567492c2f40 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa286ff5640 10 client.15333 handle_command_reply: tid=0 2024-04-23T18:26:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 client.15333 shutdown 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 msgr2=0x556749250cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa29801c550 tx=0x7fa2980096f0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 shutdown_connections 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5567492c2f40 0x5567492c1c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556749242710 0x556749251320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa286ff5640 0 client.15333 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556749241d50 0x556749250cf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 --2- 172.21.15.33:0/4186475909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556749240b80 0x5567492507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 >> 172.21.15.33:0/4186475909 conn(0x55674923c070 msgr2=0x55674923a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 shutdown_connections 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 1 -- 172.21.15.33:0/4186475909 wait complete. 2024-04-23T18:26:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.232+0000 7fa2ac040740 20 client.15333 trim_cache size 0 max 16384 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 113.950468062, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 87, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 85, 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 118.07958427 2024-04-23T18:26:21.232 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 msgr2=0x7fa28c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 0x7fa28c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa29c020000 tx=0x7fa29c01e000 comp rx=0 tx=0).stop 2024-04-23T18:26:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 msgr2=0x7fa2a41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa294006010 tx=0x7fa2940042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 shutdown_connections 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fa28c0a3bb0 0x7fa28c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2a4103b70 0x7fa2a419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2a41031b0 0x7fa2a4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 --2- 172.21.15.33:0/2880664676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a4101fc0 0x7fa2a41981f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 >> 172.21.15.33:0/2880664676 conn(0x7fa2a40fd530 msgr2=0x7fa2a4106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 shutdown_connections 2024-04-23T18:26:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.233+0000 7fa2a9329640 1 -- 172.21.15.33:0/2880664676 wait complete. 2024-04-23T18:26:21.236 INFO:teuthology.run_tasks:Running task sequential... 2024-04-23T18:26:21.248 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-23T18:26:21.248 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-23T18:26:21.248 INFO:teuthology.task.print:Enabling mgr modules 2024-04-23T18:26:21.248 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-23T18:26:21.249 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:26:21.249 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:26:21.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-23T18:26:21.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 -- 172.21.15.33:0/1220898677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100103b90 msgr2=0x7fd10010a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 --2- 172.21.15.33:0/1220898677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100103b90 0x7fd10010a420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009ca0 tx=0x7fd0f802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 -- 172.21.15.33:0/1220898677 shutdown_connections 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 --2- 172.21.15.33:0/1220898677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100103b90 0x7fd10010a420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 --2- 172.21.15.33:0/1220898677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1001031d0 0x7fd100103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 --2- 172.21.15.33:0/1220898677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd100101fe0 0x7fd100102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 -- 172.21.15.33:0/1220898677 >> 172.21.15.33:0/1220898677 conn(0x7fd1000fd570 msgr2=0x7fd1000ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 -- 172.21.15.33:0/1220898677 shutdown_connections 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.358+0000 7fd105b43640 1 -- 172.21.15.33:0/1220898677 wait complete. 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 Processor -- start 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 -- start start 2024-04-23T18:26:21.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1001031d0 0x7fd100198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 0x7fd10019f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd10010db20 con 0x7fd100101fe0 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd10010d9a0 con 0x7fd1001031d0 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd105b43640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd10010dca0 con 0x7fd100103b90 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48924/0 (socket says 172.21.15.33:48924) 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 -- 172.21.15.33:0/4059638017 learned_addr learned my addr 172.21.15.33:0/4059638017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0feffd640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1001031d0 0x7fd100198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0fffff640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 0x7fd10019f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 -- 172.21.15.33:0/4059638017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 msgr2=0x7fd10019f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 0x7fd10019f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 -- 172.21.15.33:0/4059638017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1001031d0 msgr2=0x7fd100198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1001031d0 0x7fd100198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 -- 172.21.15.33:0/4059638017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10019fcf0 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0fffff640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 0x7fd10019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.359+0000 7fd0ff7fe640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c970 tx=0x7fd0f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.360+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4004220 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.360+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f40043c0 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.360+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10019ffe0 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.360+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f400f450 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.360+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1001a0440 con 0x7fd100101fe0 2024-04-23T18:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.361+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd100109300 con 0x7fd100101fe0 2024-04-23T18:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.362+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd0f4003610 con 0x7fd100101fe0 2024-04-23T18:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.362+0000 7fd0fcff9640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 0x7fd0e80a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.363+0000 7fd0feffd640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 0x7fd0e80a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.363+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7fd0f40d0680 con 0x7fd100101fe0 2024-04-23T18:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.363+0000 7fd0feffd640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 0x7fd0e80a6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1001995e0 tx=0x7fd0f000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:21.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.366+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd0f4014030 con 0x7fd100101fe0 2024-04-23T18:26:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:21 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2896697687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:21 smithi106 ceph-mon[32716]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:26:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:21 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2926303315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:21 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2896697687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:21 smithi122 ceph-mon[32546]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-04-23T18:26:21.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:21 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2926303315' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:21.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:21.606+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd10002d870 con 0x7fd100101fe0 2024-04-23T18:26:22.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.052+0000 7fd0fcff9640 1 -- 172.21.15.33:0/4059638017 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd0f409d060 con 0x7fd100101fe0 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 msgr2=0x7fd0e80a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 0x7fd0e80a6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1001995e0 tx=0x7fd0f000b590 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 msgr2=0x7fd1001980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c970 tx=0x7fd0f400ce30 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 shutdown_connections 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7fd0e80a3c50 0x7fd0e80a6110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd100103b90 0x7fd10019f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1001031d0 0x7fd100198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 --2- 172.21.15.33:0/4059638017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd100101fe0 0x7fd1001980d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.055+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 >> 172.21.15.33:0/4059638017 conn(0x7fd1000fd570 msgr2=0x7fd1001064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.057+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 shutdown_connections 2024-04-23T18:26:22.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.057+0000 7fd105b43640 1 -- 172.21.15.33:0/4059638017 wait complete. 2024-04-23T18:26:22.067 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 -- 172.21.15.33:0/555718486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c105380 msgr2=0x7f498c107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 --2- 172.21.15.33:0/555718486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c105380 0x7f498c107770 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f497c009900 tx=0x7f497c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 -- 172.21.15.33:0/555718486 shutdown_connections 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 --2- 172.21.15.33:0/555718486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c107cb0 0x7f498c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 --2- 172.21.15.33:0/555718486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c105380 0x7f498c107770 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 --2- 172.21.15.33:0/555718486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c102a50 0x7f498c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 -- 172.21.15.33:0/555718486 >> 172.21.15.33:0/555718486 conn(0x7f498c0fc440 msgr2=0x7f498c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 -- 172.21.15.33:0/555718486 shutdown_connections 2024-04-23T18:26:22.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.184+0000 7f4990810640 1 -- 172.21.15.33:0/555718486 wait complete. 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 Processor -- start 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 -- start start 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c102a50 0x7f498c198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c105380 0x7f498c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c10db10 con 0x7f498c107cb0 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f498c10d990 con 0x7f498c105380 2024-04-23T18:26:22.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4990810640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f498c10dc90 con 0x7f498c102a50 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48974/0 (socket says 172.21.15.33:48974) 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 -- 172.21.15.33:0/4049594637 learned_addr learned my addr 172.21.15.33:0/4049594637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498a59c640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c102a50 0x7f498c198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f4989d9b640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c105380 0x7f498c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 -- 172.21.15.33:0/4049594637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c102a50 msgr2=0x7f498c198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c102a50 0x7f498c198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 -- 172.21.15.33:0/4049594637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c105380 msgr2=0x7f498c1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c105380 0x7f498c1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.185+0000 7f498ad9d640 1 -- 172.21.15.33:0/4049594637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c19fd80 con 0x7f498c107cb0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f498400eaa0 tx=0x7f498400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400cdb0 con 0x7f498c107cb0 2024-04-23T18:26:22.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984005020 con 0x7f498c107cb0 2024-04-23T18:26:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1a0070 con 0x7f498c107cb0 2024-04-23T18:26:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840116d0 con 0x7f498c107cb0 2024-04-23T18:26:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.186+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a05e0 con 0x7f498c107cb0 2024-04-23T18:26:22.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.187+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c0fdf30 con 0x7f498c107cb0 2024-04-23T18:26:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.188+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f49840051d0 con 0x7f498c107cb0 2024-04-23T18:26:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.188+0000 7f497b7fe640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 0x7f49740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.189+0000 7f498a59c640 1 -- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 msgr2=0x7f49740a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.189+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f49840d0440 con 0x7f498c107cb0 2024-04-23T18:26:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.189+0000 7f498a59c640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 0x7f49740a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:22.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.193+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4984016030 con 0x7f498c107cb0 2024-04-23T18:26:22.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:22.370 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:22.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:22.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-mgr[34113]: -- 172.21.15.106:0/3222966161 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55559f4da5a0 con 0x55559f4aec00 2024-04-23T18:26:22.371 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:22.233+0000 7f4a0bf00200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.383+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f498c062b50 con 0x7f498c107cb0 2024-04-23T18:26:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.387+0000 7f497b7fe640 1 -- 172.21.15.33:0/4049594637 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f498409c7d0 con 0x7f498c107cb0 2024-04-23T18:26:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 msgr2=0x7f49740a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:22.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 0x7f49740a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 msgr2=0x7f498c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f498400eaa0 tx=0x7f498400ef60 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 shutdown_connections 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f49740a3d20 0x7f49740a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c107cb0 0x7f498c19f710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c105380 0x7f498c1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 --2- 172.21.15.33:0/4049594637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c102a50 0x7f498c198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 >> 172.21.15.33:0/4049594637 conn(0x7f498c0fc440 msgr2=0x7f498c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 shutdown_connections 2024-04-23T18:26:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.389+0000 7f4990810640 1 -- 172.21.15.33:0/4049594637 wait complete. 2024-04-23T18:26:22.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:22.393 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:22.393 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:22.393 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:22.244+0000 7f05ef335200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:22.394 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-23T18:26:22.404 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-23T18:26:22.404 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: ignoring --setuser ceph since I am not root 2024-04-23T18:26:22.404 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: ignoring --setgroup ceph since I am not root 2024-04-23T18:26:22.404 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-mgr[34409]: -- 172.21.15.122:0/2592287225 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5f21145a0 con 0x55c5f20e9000 2024-04-23T18:26:22.405 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:22.229+0000 7fc6ce47f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-23T18:26:22.405 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:22.411+0000 7fc6ce47f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 -- 172.21.15.33:0/986454491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350105360 msgr2=0x7f0350107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 --2- 172.21.15.33:0/986454491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350105360 0x7f0350107750 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f034c009900 tx=0x7f034c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 -- 172.21.15.33:0/986454491 shutdown_connections 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 --2- 172.21.15.33:0/986454491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350107c90 0x7f035010a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 --2- 172.21.15.33:0/986454491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350105360 0x7f0350107750 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 --2- 172.21.15.33:0/986454491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350102a30 0x7f0350104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 -- 172.21.15.33:0/986454491 >> 172.21.15.33:0/986454491 conn(0x7f03500fc420 msgr2=0x7f03500fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 -- 172.21.15.33:0/986454491 shutdown_connections 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.505+0000 7f0357f4f640 1 -- 172.21.15.33:0/986454491 wait complete. 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 Processor -- start 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 -- start start 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350105360 0x7f03501009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350107c90 0x7f0350100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f035010daf0 con 0x7f0350102a30 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f035010d970 con 0x7f0350107c90 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0357f4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f035010dc70 con 0x7f0350105360 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48994/0 (socket says 172.21.15.33:48994) 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 -- 172.21.15.33:0/1556422919 learned_addr learned my addr 172.21.15.33:0/1556422919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f03554ea640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350105360 0x7f03501009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f03564ec640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350107c90 0x7f0350100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 -- 172.21.15.33:0/1556422919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350105360 msgr2=0x7f03501009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350105360 0x7f03501009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 -- 172.21.15.33:0/1556422919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350107c90 msgr2=0x7f0350100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350107c90 0x7f0350100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.506+0000 7f0355ceb640 1 -- 172.21.15.33:0/1556422919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03501017b0 con 0x7f0350102a30 2024-04-23T18:26:22.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0355ceb640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340017030 con 0x7f0350102a30 2024-04-23T18:26:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0340004020 con 0x7f0350102a30 2024-04-23T18:26:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400047a0 con 0x7f0350102a30 2024-04-23T18:26:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03501a2300 con 0x7f0350102a30 2024-04-23T18:26:22.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.507+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03501a2820 con 0x7f0350102a30 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.508+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035004fce0 con 0x7f0350102a30 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.510+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0340009a30 con 0x7f0350102a30 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.511+0000 7f0346ffd640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 0x7f033c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.511+0000 7f03554ea640 1 -- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 msgr2=0x7f033c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.511+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f03400cff00 con 0x7f0350102a30 2024-04-23T18:26:22.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.511+0000 7f03554ea640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 0x7f033c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.516+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f034009c8a0 con 0x7f0350102a30 2024-04-23T18:26:22.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.703+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f03500fdf10 con 0x7f0350102a30 2024-04-23T18:26:22.698 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:22.513+0000 7fc6ce47f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.708+0000 7f0346ffd640 1 -- 172.21.15.33:0/1556422919 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f034009c1a0 con 0x7f0350102a30 2024-04-23T18:26:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 msgr2=0x7f033c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 0x7f033c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 msgr2=0x7f03501023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f034000d950 tx=0x7f034000de10 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 shutdown_connections 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f033c0a3ca0 0x7f033c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0350107c90 0x7f0350100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0350105360 0x7f03501009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 --2- 172.21.15.33:0/1556422919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0350102a30 0x7f03501023a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.710+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 >> 172.21.15.33:0/1556422919 conn(0x7f03500fc420 msgr2=0x7f0350103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.711+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 shutdown_connections 2024-04-23T18:26:22.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:22.711+0000 7f0357f4f640 1 -- 172.21.15.33:0/1556422919 wait complete. 2024-04-23T18:26:22.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:22.443+0000 7f05ef335200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:22.714 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:22.556+0000 7f05ef335200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:22.715 INFO:teuthology.run_tasks:Running task kclient... 2024-04-23T18:26:22.738 INFO:tasks.kclient:Mounting kernel clients... 2024-04-23T18:26:22.738 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-23T18:26:22.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:22.739 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-23T18:26:22.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:22.755 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-23T18:26:22.821 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-04-23T18:26:22.822 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:22.419+0000 7f4a0bf00200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:22.822 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:22.521+0000 7f4a0bf00200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-23T18:26:22.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:26:22.823 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-23T18:26:22.823 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-23T18:26:22.823 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:22.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-23T18:26:22.971 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:22.886+0000 7f05ef335200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:22.971 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:22.978+0000 7f05ef335200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:23 smithi106 ceph-mon[32716]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:26:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4049594637' entity='client.admin' 2024-04-23T18:26:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1556422919' entity='client.admin' 2024-04-23T18:26:23.121 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:22.829+0000 7f4a0bf00200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:23.121 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:22.913+0000 7f4a0bf00200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:23.125 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T18:26:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:23 smithi122 ceph-mon[32546]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:26:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4049594637' entity='client.admin' 2024-04-23T18:26:23.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1556422919' entity='client.admin' 2024-04-23T18:26:23.199 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:22.817+0000 7fc6ce47f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-23T18:26:23.199 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:22.900+0000 7fc6ce47f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-23T18:26:23.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4059638017' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-23T18:26:23.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:23 smithi033 ceph-mon[29274]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-04-23T18:26:23.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4049594637' entity='client.admin' 2024-04-23T18:26:23.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1556422919' entity='client.admin' 2024-04-23T18:26:24.370 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:24.086+0000 7f4a0bf00200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:24.370 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:24.255+0000 7f4a0bf00200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:24.385 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.057+0000 7fc6ce47f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:24.385 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.224+0000 7fc6ce47f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:24.536 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:24.262+0000 7f05ef335200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-23T18:26:24.536 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:24.446+0000 7f05ef335200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:24.671 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.392+0000 7fc6ce47f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:24.706 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:24.425+0000 7f4a0bf00200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:24.823 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:24.631+0000 7f05ef335200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-23T18:26:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 -- 172.21.15.33:0/1865507523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454108ab0 msgr2=0x7f2454108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 --2- 172.21.15.33:0/1865507523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454108ab0 0x7f2454108e90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2444009990 tx=0x7f2444009ca0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 -- 172.21.15.33:0/1865507523 shutdown_connections 2024-04-23T18:26:24.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 --2- 172.21.15.33:0/1865507523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454103480 0x7f245410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 --2- 172.21.15.33:0/1865507523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454108ab0 0x7f2454108e90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 --2- 172.21.15.33:0/1865507523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454102ac0 0x7f2454102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 -- 172.21.15.33:0/1865507523 >> 172.21.15.33:0/1865507523 conn(0x7f2454079bf0 msgr2=0x7f245407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 -- 172.21.15.33:0/1865507523 shutdown_connections 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.936+0000 7f245a102700 1 -- 172.21.15.33:0/1865507523 wait complete. 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 Processor -- start 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 -- start start 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 0x7f24541996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 0x7f245419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2454112ca0 con 0x7f2454102ac0 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2454112b20 con 0x7f2454108ab0 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f245a102700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2454112e20 con 0x7f2454103480 2024-04-23T18:26:24.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f24537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f24537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49014/0 (socket says 172.21.15.33:49014) 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f24537fe700 1 -- 172.21.15.33:0/2067598681 learned_addr learned my addr 172.21.15.33:0/2067598681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f244affd700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 0x7f24541996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.937+0000 7f2453fff700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 0x7f245419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 msgr2=0x7f24541996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 0x7f24541996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 msgr2=0x7f245419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 0x7f245419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 -- 172.21.15.33:0/2067598681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245419e180 con 0x7f2454102ac0 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f244affd700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 0x7f24541996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f2453fff700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 0x7f245419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24537fe700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f244400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:24.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444027070 con 0x7f2454102ac0 2024-04-23T18:26:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245419e410 con 0x7f2454102ac0 2024-04-23T18:26:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2444022470 con 0x7f2454102ac0 2024-04-23T18:26:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24541a5cf0 con 0x7f2454102ac0 2024-04-23T18:26:24.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.938+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2444017650 con 0x7f2454102ac0 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.939+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245410b040 con 0x7f2454102ac0 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.940+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f244402a070 con 0x7f2454102ac0 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.941+0000 7f24517fa700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 0x7f24400a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.941+0000 7f244affd700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 msgr2=0x7f24400a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.941+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f24440d14b0 con 0x7f2454102ac0 2024-04-23T18:26:24.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.941+0000 7f244affd700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 0x7f24400a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:24.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:24.945+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2444013070 con 0x7f2454102ac0 2024-04-23T18:26:24.948 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.678+0000 7fc6ce47f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:24.948 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.769+0000 7fc6ce47f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:24.998 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:24.713+0000 7f4a0bf00200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:24.998 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:24.804+0000 7f4a0bf00200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:24.998 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:25.005+0000 7f4a0bf00200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:25.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.142+0000 7f244affd700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 msgr2=0x7f24400a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:25.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.142+0000 7f244affd700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 0x7f24400a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:25.135 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:24.945+0000 7f05ef335200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-23T18:26:25.135 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:25.044+0000 7f05ef335200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-23T18:26:25.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.200+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f245419a4d0 con 0x7f2454102ac0 2024-04-23T18:26:25.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.200+0000 7f24517fa700 1 -- 172.21.15.33:0/2067598681 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f244409df20 con 0x7f2454102ac0 2024-04-23T18:26:25.193 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-23T18:26:25.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 msgr2=0x7f24400a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 0x7f24400a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 msgr2=0x7f2454199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2444000c00 tx=0x7f244400bfd0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 shutdown_connections 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f24400a3d10 0x7f24400a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2454108ab0 0x7f245419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2454102ac0 0x7f2454199170 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 --2- 172.21.15.33:0/2067598681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2454103480 0x7f24541996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 >> 172.21.15.33:0/2067598681 conn(0x7f2454079bf0 msgr2=0x7f2454100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 shutdown_connections 2024-04-23T18:26:25.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:25.203+0000 7f245a102700 1 -- 172.21.15.33:0/2067598681 wait complete. 2024-04-23T18:26:25.245 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:24.969+0000 7fc6ce47f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:25.245 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:25.159+0000 7fc6ce47f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:25.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:25 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2067598681' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:25.370 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:25.196+0000 7f4a0bf00200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:25.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:25 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2067598681' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:25.460 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:25.262+0000 7f05ef335200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-23T18:26:25.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:25 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2067598681' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-23T18:26:25.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:25.466+0000 7f05ef335200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-23T18:26:25.994 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:26:25.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:25.995 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42080sec preferred_lft 42080sec 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-23T18:26:26.013 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-23T18:26:26.014 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-23T18:26:26.014 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-23T18:26:26.014 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-23T18:26:26.014 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:26.014 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-23T18:26:26.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-23T18:26:26.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-23T18:26:26.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-23T18:26:26.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-23T18:26:26.015 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-23T18:26:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:26.120 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:25.794+0000 7f4a0bf00200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:26.154 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:25.749+0000 7fc6ce47f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:26.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:26.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:26.204 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-23T18:26:26.215 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:26.108+0000 7f05ef335200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-23T18:26:26.229 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-23T18:26:26.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:26.231 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-23T18:26:26.296 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-23T18:26:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-23T18:26:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-23T18:26:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-23T18:26:26.297 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-23T18:26:26.297 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:26.297 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-23T18:26:26.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-23T18:26:26.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-23T18:26:26.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-23T18:26:26.298 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-23T18:26:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:26.448 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:26.162+0000 7fc6ce47f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:26.448 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:26.256+0000 7fc6ce47f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:26.501 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:26.212+0000 7f4a0bf00200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:26.501 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:26.309+0000 7f4a0bf00200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:26.501 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:26.508+0000 7f4a0bf00200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:26.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:26.581 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-23T18:26:26.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:26.597 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-23T18:26:26.653 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:26.653 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-23T18:26:26.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T18:26:26.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-23T18:26:26.653 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-23T18:26:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:26.750 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:26.583+0000 7f05ef335200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-23T18:26:26.750 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:26.688+0000 7f05ef335200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-23T18:26:26.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:26.773 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-23T18:26:26.773 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-23T18:26:26.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:26.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:26.910 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-23T18:26:26.910 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-23T18:26:26.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-23T18:26:26.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-23T18:26:26.910 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-23T18:26:26.948 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:26.458+0000 7fc6ce47f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:26.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:26.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:26:26.990 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:27.006 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-23T18:26:27.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-23T18:26:27.006 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:27.062 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-23T18:26:27.062 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-23T18:26:27.062 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/a5f2bf8e-f395-4df6-b53a-df0570b88564 2024-04-23T18:26:27.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:27.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/a5f2bf8e-f395-4df6-b53a-df0570b88564 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:27.123 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:26.921+0000 7f05ef335200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-23T18:26:27.177 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:27.177 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T18:26:27.290 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-23T18:26:27.291 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-23T18:26:27.291 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.106:3300,172.21.15.122:3300" 2024-04-23T18:26:27.291 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with new device syntax: 0@766ecab6-019d-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/a5f2bf8e-f395-4df6-b53a-df0570b88564 2024-04-23T18:26:27.291 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.33:3300/172.21.15.106:3300/172.21.15.122:3300" will pass to kernel 2024-04-23T18:26:27.292 INFO:teuthology.orchestra.run.smithi033.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-23T18:26:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: You just mounted a file system that supports labels which does not 2024-04-23T18:26:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-23T18:26:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: applications will generate AVC messages and not be allowed access to 2024-04-23T18:26:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-23T18:26:27.294 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-23T18:26:27.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T18:26:27.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:27.299 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:26.978+0000 7fc6ce47f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:27.299 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.068+0000 7fc6ce47f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:27.356 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.028+0000 7f4a0bf00200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:27.356 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.119+0000 7f4a0bf00200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:27.362 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:27.461 INFO:teuthology.orchestra.run.smithi033.stdout:client25031 2024-04-23T18:26:27.461 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:26:27.461 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T18:26:27.478 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:27.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:27.578 INFO:teuthology.orchestra.run.smithi033.stdout:client25031 2024-04-23T18:26:27.579 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-23T18:26:27.620 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.363+0000 7f4a0bf00200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:27.620 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.495+0000 7f4a0bf00200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:27.620 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.590+0000 7f4a0bf00200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:27.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-23T18:26:27.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:27.637 INFO:teuthology.orchestra.run.smithi033.stdout:client25031 2024-04-23T18:26:27.637 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/766ecab6-019d-11ef-bc93-c7b262605968.client25031/status') 2024-04-23T18:26:27.638 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.306+0000 7fc6ce47f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:27.638 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.438+0000 7fc6ce47f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:27.638 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.531+0000 7fc6ce47f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:27.638 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.645+0000 7fc6ce47f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-23T18:26:27.696 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:27.499+0000 7f05ef335200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-23T18:26:27.696 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:27.605+0000 7f05ef335200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-23T18:26:27.698 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.25031 (3)172.21.15.33:0/504568517 2024-04-23T18:26:27.699 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-04-23T18:26:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-04-23T18:26:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-04-23T18:26:27.699 INFO:teuthology.orchestra.run.smithi033.stderr:70 bytes copied, 2.3824e-05 s, 2.9 MB/s 2024-04-23T18:26:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-23T18:26:27.706 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:26:27.717 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:26:27.717 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:26:27.717 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-23T18:26:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 -- 172.21.15.33:0/3798916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 msgr2=0x7f066410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 --2- 172.21.15.33:0/3798916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066410a0e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f065c009ca0 tx=0x7f065c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 -- 172.21.15.33:0/3798916287 shutdown_connections 2024-04-23T18:26:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 --2- 172.21.15.33:0/3798916287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066410a0e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 --2- 172.21.15.33:0/3798916287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 --2- 172.21.15.33:0/3798916287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 0x7f0664104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 -- 172.21.15.33:0/3798916287 >> 172.21.15.33:0/3798916287 conn(0x7f06640fc420 msgr2=0x7f06640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 -- 172.21.15.33:0/3798916287 shutdown_connections 2024-04-23T18:26:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.831+0000 7f066906b640 1 -- 172.21.15.33:0/3798916287 wait complete. 2024-04-23T18:26:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.832+0000 7f066906b640 1 Processor -- start 2024-04-23T18:26:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.832+0000 7f066906b640 1 -- start start 2024-04-23T18:26:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 0x7f0664198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f066410daf0 con 0x7f0664107c90 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f066410d970 con 0x7f0664105360 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066906b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f066410dc70 con 0x7f0664102a30 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43840/0 (socket says 172.21.15.33:43840) 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066359e640 1 -- 172.21.15.33:0/3691386474 learned_addr learned my addr 172.21.15.33:0/3691386474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:59454/0 (socket says 172.21.15.33:59454) 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f0662d9d640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 0x7f0664198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 msgr2=0x7f0664198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 0x7f0664198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 msgr2=0x7f066419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066259c640 1 -- 172.21.15.33:0/3691386474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066419fd20 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.833+0000 7f066359e640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f066259c640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f065800e9c0 tx=0x7f065800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065800ccc0 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06580108e0 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066419ffb0 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06580187c0 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.834+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06641a04d0 con 0x7f0664105360 2024-04-23T18:26:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.835+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06640fdf10 con 0x7f0664105360 2024-04-23T18:26:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.836+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0658010430 con 0x7f0664105360 2024-04-23T18:26:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.837+0000 7f064bfff640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 0x7f064c0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.837+0000 7f0662d9d640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 msgr2=0x7f064c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.837+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f06580d0bf0 con 0x7f0664105360 2024-04-23T18:26:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.837+0000 7f0662d9d640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 0x7f064c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:27.840+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f065809cf80 con 0x7f0664105360 2024-04-23T18:26:27.980 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:27.880+0000 7f05ef335200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-23T18:26:27.994 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:27.705+0000 7f4a0bf00200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:28.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.038+0000 7f0662d9d640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 msgr2=0x7f064c0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:28.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.038+0000 7f0662d9d640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 0x7f064c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.069+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f066402d7c0 con 0x7f0664105360 2024-04-23T18:26:28.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.124+0000 7f064bfff640 1 -- 172.21.15.33:0/3691386474 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f06580a1280 con 0x7f0664105360 2024-04-23T18:26:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 msgr2=0x7f064c0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 0x7f064c0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 msgr2=0x7f0664198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f065800e9c0 tx=0x7f065800ee80 comp rx=0 tx=0).stop 2024-04-23T18:26:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 shutdown_connections 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f064c0a3ca0 0x7f064c0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0664107c90 0x7f066419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0664105360 0x7f0664198640 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 --2- 172.21.15.33:0/3691386474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0664102a30 0x7f0664198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 >> 172.21.15.33:0/3691386474 conn(0x7f06640fc420 msgr2=0x7f0664103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 shutdown_connections 2024-04-23T18:26:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.126+0000 7f066906b640 1 -- 172.21.15.33:0/3691386474 wait complete. 2024-04-23T18:26:28.132 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-23T18:26:28.198 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:27.942+0000 7fc6ce47f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:28.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3691386474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:28 smithi122 ceph-mon[32546]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3363911552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102a00 msgr2=0x7f58a4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3363911552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102a00 0x7f58a4109320 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f589c009ca0 tx=0x7f589c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3363911552 shutdown_connections 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3363911552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102a00 0x7f58a4109320 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3363911552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58a4102040 0x7f58a41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3363911552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a4100e60 0x7f58a4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3363911552 >> 172.21.15.33:0/3363911552 conn(0x7f58a40fc420 msgr2=0x7f58a40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3363911552 shutdown_connections 2024-04-23T18:26:28.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.247+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3363911552 wait complete. 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 Processor -- start 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 -- start start 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58a4100e60 0x7f58a4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 0x7f58a4198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a4102a00 0x7f58a419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410db30 con 0x7f58a4102040 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410d9b0 con 0x7f58a4102a00 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58ab5a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58a410dcb0 con 0x7f58a4100e60 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58a8b3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 0x7f58a4198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58a8b3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 0x7f58a4198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43870/0 (socket says 172.21.15.33:43870) 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58a8b3d640 1 -- 172.21.15.33:0/3807264583 learned_addr learned my addr 172.21.15.33:0/3807264583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58a4100e60 0x7f58a4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.248+0000 7f58a9b3f640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a4102a00 0x7f58a419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58a4100e60 msgr2=0x7f58a4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58a4100e60 0x7f58a4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a4102a00 msgr2=0x7f58a419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58a4102a00 0x7f58a419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a419fd20 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58a8b3d640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 0x7f58a4198640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f589400ea60 tx=0x7f589400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589400cde0 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a419ffb0 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58940035f0 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5894010690 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.249+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a41a04d0 con 0x7f58a4102040 2024-04-23T18:26:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.250+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a4108200 con 0x7f58a4102040 2024-04-23T18:26:28.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.252+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f58940037a0 con 0x7f58a4102040 2024-04-23T18:26:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.253+0000 7f589a7fc640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 0x7f58900a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.253+0000 7f58a933e640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 msgr2=0x7f58900a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.253+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f58940d0700 con 0x7f58a4102040 2024-04-23T18:26:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.253+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 0x7f58900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-23T18:26:28.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.258+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f589409d0e0 con 0x7f58a4102040 2024-04-23T18:26:28.370 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:28.001+0000 7f4a0bf00200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:28.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3691386474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:28 smithi106 ceph-mon[32716]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.454+0000 7f58a933e640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 msgr2=0x7f58900a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:28.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.454+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 0x7f58900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-23T18:26:28.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3691386474' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-mon[29274]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-23T18:26:28.461 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:28.025+0000 7f05ef335200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-23T18:26:28.461 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:28.129+0000 7f05ef335200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-23T18:26:28.461 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:28.261+0000 7f05ef335200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-23T18:26:28.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.529+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f58a4062430 con 0x7f58900a3ca0 2024-04-23T18:26:28.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:28.581+0000 7f05ef335200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-23T18:26:28.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.855+0000 7f58a933e640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 msgr2=0x7f58900a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2158874591 2024-04-23T18:26:28.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:28.855+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 0x7f58900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-23T18:26:28.870 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:28.605+0000 7f4a0bf00200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:28.870 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 18:26:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y[34109]: 2024-04-23T18:26:28.819+0000 7f4a0bf00200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:28.948 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:28.538+0000 7fc6ce47f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:28.948 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 18:26:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z[34405]: 2024-04-23T18:26:28.752+0000 7fc6ce47f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:29.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:29.141+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f5894004000 con 0x7f58a4102040 2024-04-23T18:26:29.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:29.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: Standby manager daemon z restarted 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: Standby manager daemon z started 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: Standby manager daemon y restarted 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: Standby manager daemon y started 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:29.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:29 smithi122 ceph-mon[32546]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.453 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.229+0000 7f05ef335200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-23T18:26:29.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:29.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:29.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: Standby manager daemon z restarted 2024-04-23T18:26:29.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: Standby manager daemon z started 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: Standby manager daemon y restarted 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: Standby manager daemon y started 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:29.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-mon[29274]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:29.471+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f5894099050 con 0x7f58a4102040 2024-04-23T18:26:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:29.472+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 msgr2=0x7f58900a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-23T18:26:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:29.472+0000 7f589a7fc640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/2158874591,v1:172.21.15.33:6801/2158874591] conn(0x7f58900a3ca0 0x7f58900a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:29.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-23T18:26:29.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:26:29.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: Standby manager daemon z restarted 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: Standby manager daemon z started 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.122:0/2817799996' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: Standby manager daemon y restarted 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: Standby manager daemon y started 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-23T18:26:29.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:29 smithi106 ceph-mon[32716]: from='mgr.? 172.21.15.106:0/3089864902' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-23T18:26:29.712 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.460+0000 7f05ef335200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-23T18:26:29.712 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.708+0000 7f0551caf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.708+0000 7f0551caf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.708+0000 7f0551caf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.708+0000 7f0551caf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.708+0000 7f0551caf700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.709+0000 7f054ec69700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.709+0000 7f054ec69700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.709+0000 7f054ec69700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.709+0000 7f054ec69700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:29.713 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:29.709+0000 7f054ec69700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: Active manager daemon x restarted 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: Activating manager daemon x 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: mgrmap e21: x(active, starting, since 0.0115755s), standbys: z, y 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:30.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:30.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:30.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:30.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:30.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: Manager daemon x is now available 2024-04-23T18:26:30.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:30.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:30.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:30.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.484+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mon.0 v2:172.21.15.33:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f5894078540 con 0x7f58a4102040 2024-04-23T18:26:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.485+0000 7f589a7fc640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5890035890 0x7f5890035c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.485+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5894021e50 con 0x7f5890035890 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: Active manager daemon x restarted 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: Activating manager daemon x 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: mgrmap e21: x(active, starting, since 0.0115755s), standbys: z, y 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:30.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:30.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:30.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: Manager daemon x is now available 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:30.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:30.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.489+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5890035890 0x7f5890035c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.491+0000 7f58a933e640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5890035890 0x7f5890035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101bb0 tx=0x7f58a0005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.511+0000 7f589a7fc640 1 -- 172.21.15.33:0/3807264583 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5894021e50 con 0x7f5890035890 2024-04-23T18:26:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:{} 2024-04-23T18:26:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.513+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5890035890 msgr2=0x7f5890035c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.513+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5890035890 0x7f5890035c70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58a4101bb0 tx=0x7f58a0005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.513+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 msgr2=0x7f58a4198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.513+0000 7f58ab5a2640 1 --2- 172.21.15.33:0/3807264583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58a4102040 0x7f58a4198640 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f589400ea60 tx=0x7f589400ef20 comp rx=0 tx=0).stop 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.513+0000 7f58a9b3f640 1 -- 172.21.15.33:0/3807264583 reap_dead start 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.514+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 shutdown_connections 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.514+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 >> 172.21.15.33:0/3807264583 conn(0x7f58a40fc420 msgr2=0x7f58a41053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.514+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 shutdown_connections 2024-04-23T18:26:30.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.514+0000 7f58ab5a2640 1 -- 172.21.15.33:0/3807264583 wait complete. 2024-04-23T18:26:30.520 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-23T18:26:30.532 INFO:teuthology.run_tasks:Running task exec... 2024-04-23T18:26:30.612 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:26:30.612 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:26:30.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-23T18:26:30.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: Active manager daemon x restarted 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: Activating manager daemon x 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: osdmap e88: 12 total, 12 up, 12 in 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: mgrmap e21: x(active, starting, since 0.0115755s), standbys: z, y 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T18:26:30.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-23T18:26:30.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: Manager daemon x is now available 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-23T18:26:30.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-23T18:26:30.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.725+0000 7ff51b9c6640 1 -- 172.21.15.33:0/71475278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514107c90 msgr2=0x7ff51410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.725+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/71475278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514107c90 0x7ff51410a0e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- 172.21.15.33:0/71475278 shutdown_connections 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/71475278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514107c90 0x7ff51410a0e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/71475278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 0x7ff514107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/71475278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514102a30 0x7ff514104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- 172.21.15.33:0/71475278 >> 172.21.15.33:0/71475278 conn(0x7ff5140fc420 msgr2=0x7ff5140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- 172.21.15.33:0/71475278 shutdown_connections 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- 172.21.15.33:0/71475278 wait complete. 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 Processor -- start 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- start start 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 0x7ff5141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514107c90 0x7ff51419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410daf0 con 0x7ff514102a30 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410d970 con 0x7ff514105360 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.726+0000 7ff51b9c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff51410dc70 con 0x7ff514107c90 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44026/0 (socket says 172.21.15.33:44026) 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 -- 172.21.15.33:0/2042168745 learned_addr learned my addr 172.21.15.33:0/2042168745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519f63640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514107c90 0x7ff51419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff518f61640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 0x7ff5141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 -- 172.21.15.33:0/2042168745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514107c90 msgr2=0x7ff51419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514107c90 0x7ff51419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 -- 172.21.15.33:0/2042168745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 msgr2=0x7ff5141986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 0x7ff5141986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 -- 172.21.15.33:0/2042168745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51419fd40 con 0x7ff514102a30 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff519762640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50400cd80 con 0x7ff514102a30 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5040108e0 con 0x7ff514102a30 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51419ffd0 con 0x7ff514102a30 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504018980 con 0x7ff514102a30 2024-04-23T18:26:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.727+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a0490 con 0x7ff514102a30 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.728+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5140fdf10 con 0x7ff514102a30 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.729+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ff504010430 con 0x7ff514102a30 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.730+0000 7ff50a7fc640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 0x7ff5000a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.730+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7ff5040d03c0 con 0x7ff514102a30 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.730+0000 7ff518f61640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 0x7ff5000a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.730+0000 7ff518f61640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 0x7ff5000a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff5100095f0 tx=0x7ff510005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.735+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff50409c930 con 0x7ff514102a30 2024-04-23T18:26:30.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:30.497+0000 7f054bc63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:30.497+0000 7f054bc63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.764 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:30.497+0000 7f054bc63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.765 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:30.497+0000 7f054bc63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.765 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:26:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:26:30.497+0000 7f054bc63700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:26:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:30.955+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff514062390 con 0x7ff5000a3a60 2024-04-23T18:26:31.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.015+0000 7ff50a7fc640 1 -- 172.21.15.33:0/2042168745 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff514062390 con 0x7ff5000a3a60 2024-04-23T18:26:31.008 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path / 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 msgr2=0x7ff5000a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 0x7ff5000a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff5100095f0 tx=0x7ff510005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 msgr2=0x7ff514198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff50400e9f0 tx=0x7ff50400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 shutdown_connections 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5000a3a60 0x7ff5000a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff514107c90 0x7ff51419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff514105360 0x7ff5141986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 --2- 172.21.15.33:0/2042168745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff514102a30 0x7ff514198170 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 >> 172.21.15.33:0/2042168745 conn(0x7ff5140fc420 msgr2=0x7ff514103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 shutdown_connections 2024-04-23T18:26:31.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.018+0000 7ff51b9c6640 1 -- 172.21.15.33:0/2042168745 wait complete. 2024-04-23T18:26:31.022 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 -- 172.21.15.33:0/363028952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c103b60 msgr2=0x7f3d0c10a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 --2- 172.21.15.33:0/363028952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c103b60 0x7f3d0c10a3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009be0 tx=0x7f3d0402a6c0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 -- 172.21.15.33:0/363028952 shutdown_connections 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 --2- 172.21.15.33:0/363028952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c103b60 0x7f3d0c10a3f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 --2- 172.21.15.33:0/363028952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 --2- 172.21.15.33:0/363028952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c101fb0 0x7f3d0c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 -- 172.21.15.33:0/363028952 >> 172.21.15.33:0/363028952 conn(0x7f3d0c0fd540 msgr2=0x7f3d0c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 -- 172.21.15.33:0/363028952 shutdown_connections 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.137+0000 7f3d13e60640 1 -- 172.21.15.33:0/363028952 wait complete. 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 Processor -- start 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 -- start start 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c101fb0 0x7f3d0c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c103b60 0x7f3d0c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10daf0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d970 con 0x7f3d0c101fb0 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d13e60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10dc70 con 0x7f3d0c103b60 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44040/0 (socket says 172.21.15.33:44040) 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 -- 172.21.15.33:0/3047743302 learned_addr learned my addr 172.21.15.33:0/3047743302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:31.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d123fd640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c103b60 0x7f3d0c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d11bfc640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c101fb0 0x7f3d0c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 -- 172.21.15.33:0/3047743302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c103b60 msgr2=0x7f3d0c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c103b60 0x7f3d0c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 -- 172.21.15.33:0/3047743302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c101fb0 msgr2=0x7f3d0c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c101fb0 0x7f3d0c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.138+0000 7f3d113fb640 1 -- 172.21.15.33:0/3047743302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c19fcd0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d113fb640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9c0 tx=0x7f3cfc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc00cde0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c19ffc0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cfc0035f0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc010690 con 0x7f3d0c1031a0 2024-04-23T18:26:31.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.139+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a04d0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.140+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c1092d0 con 0x7f3d0c1031a0 2024-04-23T18:26:31.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.141+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f3cfc004000 con 0x7f3d0c1031a0 2024-04-23T18:26:31.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.142+0000 7f3d02ffd640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 0x7f3cf80a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.142+0000 7f3d11bfc640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 0x7f3cf80a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.142+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3cfc0d0550 con 0x7f3d0c1031a0 2024-04-23T18:26:31.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.142+0000 7f3d11bfc640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 0x7f3cf80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c103000 tx=0x7f3d08005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.147+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cfc09d110 con 0x7f3d0c1031a0 2024-04-23T18:26:31.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.378+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f3d0c062430 con 0x7f3cf80a3a60 2024-04-23T18:26:31.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.438+0000 7f3d02ffd640 1 -- 172.21.15.33:0/3047743302 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f3d0c062430 con 0x7f3cf80a3a60 2024-04-23T18:26:31.431 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path / 2024-04-23T18:26:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.440+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 msgr2=0x7f3cf80a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.440+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 0x7f3cf80a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c103000 tx=0x7f3d08005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.440+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 msgr2=0x7f3d0c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.440+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00e9c0 tx=0x7f3cfc00ee80 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.440+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 shutdown_connections 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3cf80a3a60 0x7f3cf80a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d0c103b60 0x7f3d0c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d0c1031a0 0x7f3d0c1985f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 --2- 172.21.15.33:0/3047743302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d0c101fb0 0x7f3d0c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 >> 172.21.15.33:0/3047743302 conn(0x7f3d0c0fd540 msgr2=0x7f3d0c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 shutdown_connections 2024-04-23T18:26:31.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.441+0000 7f3d13e60640 1 -- 172.21.15.33:0/3047743302 wait complete. 2024-04-23T18:26:31.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:26:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 -- 172.21.15.33:0/4037580505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 msgr2=0x7faad0100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 --2- 172.21.15.33:0/4037580505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad0100ca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faacc009900 tx=0x7faacc02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 -- 172.21.15.33:0/4037580505 shutdown_connections 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 --2- 172.21.15.33:0/4037580505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 --2- 172.21.15.33:0/4037580505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 0x7faad0101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 --2- 172.21.15.33:0/4037580505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad0100ca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.571+0000 7faad7c20640 1 -- 172.21.15.33:0/4037580505 >> 172.21.15.33:0/4037580505 conn(0x7faad00fc420 msgr2=0x7faad00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 -- 172.21.15.33:0/4037580505 shutdown_connections 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 -- 172.21.15.33:0/4037580505 wait complete. 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 Processor -- start 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 -- start start 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 0x7faad019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.572+0000 7faad7c20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad01a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad7c20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faad010fd00 con 0x7faad0101b80 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad7c20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faad010fb80 con 0x7faad0100880 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad7c20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faad010fe80 con 0x7faad01011e0 2024-04-23T18:26:31.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad61bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad01a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:59702/0 (socket says 172.21.15.33:59702) 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 -- 172.21.15.33:0/4195764938 learned_addr learned my addr 172.21.15.33:0/4195764938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad51bb640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 0x7faad019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 -- 172.21.15.33:0/4195764938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 msgr2=0x7faad019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 0x7faad019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 -- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 msgr2=0x7faad01a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad01a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 -- 172.21.15.33:0/4195764938 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad01a1fd0 con 0x7faad0100880 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad61bd640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faad59bc640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faacc005940 tx=0x7faacc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc03f030 con 0x7faad0100880 2024-04-23T18:26:31.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.573+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faacc003cb0 con 0x7faad0100880 2024-04-23T18:26:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.574+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad01a2260 con 0x7faad0100880 2024-04-23T18:26:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.574+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc035070 con 0x7faad0100880 2024-04-23T18:26:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.574+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a27a0 con 0x7faad0100880 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.575+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad010b5d0 con 0x7faad0100880 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.576+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7faacc042030 con 0x7faad0100880 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.577+0000 7faac6ffd640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.577+0000 7faad51bb640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.577+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7faacc0f0ce0 con 0x7faad0100880 2024-04-23T18:26:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.578+0000 7faad51bb640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 0x7faabc0a63a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faad019b8c0 tx=0x7faac0005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.582+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faacc0bd220 con 0x7faad0100880 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: mgrmap e22: x(active, since 1.02214s), standbys: z, y 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: [23/Apr/2024:18:26:30] ENGINE Bus STARTING 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: [23/Apr/2024:18:26:30] ENGINE Serving on https://172.21.15.33:7150 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: [23/Apr/2024:18:26:30] ENGINE Serving on http://172.21.15.33:8765 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: [23/Apr/2024:18:26:30] ENGINE Bus STARTED 2024-04-23T18:26:31.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:31 smithi033 ceph-mon[29274]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:31.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.805+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faad0062390 con 0x7faabc0a3ee0 2024-04-23T18:26:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.814+0000 7faac6ffd640 1 -- 172.21.15.33:0/4195764938 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7faad0062390 con 0x7faabc0a3ee0 2024-04-23T18:26:31.807 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:26:30", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 msgr2=0x7faabc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 0x7faabc0a63a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faad019b8c0 tx=0x7faac0005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 msgr2=0x7faad019a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faacc005940 tx=0x7faacc0042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 shutdown_connections 2024-04-23T18:26:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7faabc0a3ee0 0x7faabc0a63a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faad0101b80 0x7faad01a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faad01011e0 0x7faad019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 --2- 172.21.15.33:0/4195764938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faad0100880 0x7faad019a3b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 >> 172.21.15.33:0/4195764938 conn(0x7faad00fc420 msgr2=0x7faad00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 shutdown_connections 2024-04-23T18:26:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.816+0000 7faad7c20640 1 -- 172.21.15.33:0/4195764938 wait complete. 2024-04-23T18:26:31.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: mgrmap e22: x(active, since 1.02214s), standbys: z, y 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: [23/Apr/2024:18:26:30] ENGINE Bus STARTING 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: [23/Apr/2024:18:26:30] ENGINE Serving on https://172.21.15.33:7150 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: [23/Apr/2024:18:26:30] ENGINE Serving on http://172.21.15.33:8765 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: [23/Apr/2024:18:26:30] ENGINE Bus STARTED 2024-04-23T18:26:31.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:31 smithi106 ceph-mon[32716]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:31.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.932+0000 7f3e0ef07640 1 -- 172.21.15.33:0/648687287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102a00 msgr2=0x7f3e08109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.932+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/648687287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102a00 0x7f3e08109320 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009ca0 tx=0x7f3e0002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- 172.21.15.33:0/648687287 shutdown_connections 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/648687287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102a00 0x7f3e08109320 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/648687287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08102040 0x7f3e081024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/648687287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08100e60 0x7f3e08101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- 172.21.15.33:0/648687287 >> 172.21.15.33:0/648687287 conn(0x7f3e080fc420 msgr2=0x7f3e080fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- 172.21.15.33:0/648687287 shutdown_connections 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- 172.21.15.33:0/648687287 wait complete. 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 Processor -- start 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- start start 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 0x7f3e0819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08102a00 0x7f3e081a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810ca10 con 0x7f3e08102040 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810c890 con 0x7f3e08100e60 2024-04-23T18:26:31.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0ef07640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0810cb90 con 0x7f3e08102a00 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e0cca3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.933+0000 7f3e07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 0x7f3e0819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0cca3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:59736/0 (socket says 172.21.15.33:59736) 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0cca3640 1 -- 172.21.15.33:0/1973499846 learned_addr learned my addr 172.21.15.33:0/1973499846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0d4a4640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08102a00 0x7f3e081a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 -- 172.21.15.33:0/1973499846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08102a00 msgr2=0x7f3e081a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08102a00 0x7f3e081a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 -- 172.21.15.33:0/1973499846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 msgr2=0x7f3e0819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 -- 172.21.15.33:0/1973499846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e081a1f10 con 0x7f3e08102040 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0cca3640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e07fff640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 0x7f3e0819a880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9f0 tx=0x7f3df400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df400ccc0 con 0x7f3e08102040 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df40108e0 con 0x7f3e08102040 2024-04-23T18:26:31.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e081a2200 con 0x7f3e08102040 2024-04-23T18:26:31.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4018880 con 0x7f3e08102040 2024-04-23T18:26:31.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.934+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e081a2750 con 0x7f3e08102040 2024-04-23T18:26:31.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.936+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e081a29b0 con 0x7f3e08102040 2024-04-23T18:26:31.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.937+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f3df4010430 con 0x7f3e08102040 2024-04-23T18:26:31.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.938+0000 7f3e05ffb640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 0x7f3df80a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:31.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.938+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3df40d0cf0 con 0x7f3e08102040 2024-04-23T18:26:31.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.938+0000 7f3e0cca3640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 0x7f3df80a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:31.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.939+0000 7f3e0cca3640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 0x7f3df80a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0058c0 tx=0x7f3dfc005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:31.943+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3df409cb80 con 0x7f3e08102040 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: mgrmap e22: x(active, since 1.02214s), standbys: z, y 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: [23/Apr/2024:18:26:30] ENGINE Bus STARTING 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: [23/Apr/2024:18:26:30] ENGINE Serving on https://172.21.15.33:7150 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: [23/Apr/2024:18:26:30] ENGINE Serving on http://172.21.15.33:8765 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: [23/Apr/2024:18:26:30] ENGINE Bus STARTED 2024-04-23T18:26:31.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:31 smithi122 ceph-mon[32546]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.154+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3e08069d10 con 0x7f3df80a3e60 2024-04-23T18:26:32.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.181+0000 7f3e05ffb640 1 -- 172.21.15.33:0/1973499846 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3e08069d10 con 0x7f3df80a3e60 2024-04-23T18:26:32.174 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-04-23T18:26:32.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 msgr2=0x7f3df80a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 0x7f3df80a6320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0058c0 tx=0x7f3dfc005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 msgr2=0x7f3e0819a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 0x7f3e0819a880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9f0 tx=0x7f3df400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 shutdown_connections 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3df80a3e60 0x7f3df80a6320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e08102a00 0x7f3e081a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e08102040 0x7f3e0819a880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 --2- 172.21.15.33:0/1973499846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e08100e60 0x7f3e0819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.184+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 >> 172.21.15.33:0/1973499846 conn(0x7f3e080fc420 msgr2=0x7f3e08069600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.185+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 shutdown_connections 2024-04-23T18:26:32.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.185+0000 7f3e0ef07640 1 -- 172.21.15.33:0/1973499846 wait complete. 2024-04-23T18:26:32.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-23T18:26:32.217 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-23T18:26:32.230 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-23T18:26:32.259 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:26:32.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-23T18:26:32.262 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-23T18:26:32.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 -- 172.21.15.33:0/1814029053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c4107cd0 msgr2=0x7fd9c410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 --2- 172.21.15.33:0/1814029053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c4107cd0 0x7fd9c410a0c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009be0 tx=0x7fd9bc02a6c0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 -- 172.21.15.33:0/1814029053 shutdown_connections 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 --2- 172.21.15.33:0/1814029053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c4107cd0 0x7fd9c410a0c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 --2- 172.21.15.33:0/1814029053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c41053a0 0x7fd9c4107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 --2- 172.21.15.33:0/1814029053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 -- 172.21.15.33:0/1814029053 >> 172.21.15.33:0/1814029053 conn(0x7fd9c40fc400 msgr2=0x7fd9c40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.407+0000 7fd9ca613640 1 -- 172.21.15.33:0/1814029053 shutdown_connections 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 -- 172.21.15.33:0/1814029053 wait complete. 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 Processor -- start 2024-04-23T18:26:32.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 -- start start 2024-04-23T18:26:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c41053a0 0x7fd9c4100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 0x7fd9c4100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c410dac0 con 0x7fd9c4107cd0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c410d940 con 0x7fd9c41053a0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.408+0000 7fd9ca613640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c410dc40 con 0x7fd9c4102a70 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c8bb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 0x7fd9c4100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c41053a0 0x7fd9c4100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c8bb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 0x7fd9c4100f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44118/0 (socket says 172.21.15.33:44118) 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c8bb0640 1 -- 172.21.15.33:0/3857739716 learned_addr learned my addr 172.21.15.33:0/3857739716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 -- 172.21.15.33:0/3857739716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c41053a0 msgr2=0x7fd9c4100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c41053a0 0x7fd9c4100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 -- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 msgr2=0x7fd9c4100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 0x7fd9c4100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 -- 172.21.15.33:0/3857739716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c4101820 con 0x7fd9c4102a70 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c3fff640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4102390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cd80 con 0x7fd9c4102a70 2024-04-23T18:26:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c41a22f0 con 0x7fd9c4102a70 2024-04-23T18:26:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9b40035f0 con 0x7fd9c4102a70 2024-04-23T18:26:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4010690 con 0x7fd9c4102a70 2024-04-23T18:26:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.409+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c41a2860 con 0x7fd9c4102a70 2024-04-23T18:26:32.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.410+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c40fdef0 con 0x7fd9c4102a70 2024-04-23T18:26:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.411+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fd9b40037a0 con 0x7fd9c4102a70 2024-04-23T18:26:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.412+0000 7fd9c17fa640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 0x7fd9ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.412+0000 7fd9c37fe640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 0x7fd9ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.412+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd9b40d0c30 con 0x7fd9c4102a70 2024-04-23T18:26:32.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.412+0000 7fd9c37fe640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 0x7fd9ac0a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4102100 tx=0x7fd9b8009260 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.417+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9b409d110 con 0x7fd9c4102a70 2024-04-23T18:26:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.608+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9c402d830 con 0x7fd9c4102a70 2024-04-23T18:26:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.609+0000 7fd9c17fa640 1 -- 172.21.15.33:0/3857739716 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23101 (secure 0 0 0) 0x7fd9b409ca10 con 0x7fd9c4102a70 2024-04-23T18:26:32.603 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:32.603 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:26:29.462863+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.611+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 msgr2=0x7fd9ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.611+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 0x7fd9ac0a6270 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4102100 tx=0x7fd9b8009260 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.611+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 msgr2=0x7fd9c4102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.611+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4102390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9b400e9f0 tx=0x7fd9b400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 shutdown_connections 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd9ac0a3db0 0x7fd9ac0a6270 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9c4107cd0 0x7fd9c4100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9c41053a0 0x7fd9c4100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 --2- 172.21.15.33:0/3857739716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9c4102a70 0x7fd9c4102390 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 >> 172.21.15.33:0/3857739716 conn(0x7fd9c40fc400 msgr2=0x7fd9c41033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 shutdown_connections 2024-04-23T18:26:32.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.612+0000 7fd9ca613640 1 -- 172.21.15.33:0/3857739716 wait complete. 2024-04-23T18:26:32.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:32.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 -- 172.21.15.33:0/722270974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4104107cd0 msgr2=0x7f410410a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 --2- 172.21.15.33:0/722270974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4104107cd0 0x7f410410a0c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f40fc009900 tx=0x7f40fc02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 -- 172.21.15.33:0/722270974 shutdown_connections 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 --2- 172.21.15.33:0/722270974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4104107cd0 0x7f410410a0c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 --2- 172.21.15.33:0/722270974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41041053a0 0x7f4104107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 --2- 172.21.15.33:0/722270974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 0x7f4104104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 -- 172.21.15.33:0/722270974 >> 172.21.15.33:0/722270974 conn(0x7f41040fc400 msgr2=0x7f41040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 -- 172.21.15.33:0/722270974 shutdown_connections 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.733+0000 7f41095a8640 1 -- 172.21.15.33:0/722270974 wait complete. 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 Processor -- start 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 -- start start 2024-04-23T18:26:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 0x7f4104102390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4104107cd0 0x7f4104100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f410410dae0 con 0x7f41041053a0 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f410410d960 con 0x7f4104107cd0 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41095a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f410410dc60 con 0x7f4104102a70 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44134/0 (socket says 172.21.15.33:44134) 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 -- 172.21.15.33:0/1047453478 learned_addr learned my addr 172.21.15.33:0/1047453478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f4102ffd640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 0x7f4104102390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41037fe640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4104107cd0 0x7f4104100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 -- 172.21.15.33:0/1047453478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 msgr2=0x7f4104102390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 0x7f4104102390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 -- 172.21.15.33:0/1047453478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4104107cd0 msgr2=0x7f4104100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4104107cd0 0x7f4104100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.734+0000 7f41027fc640 1 -- 172.21.15.33:0/1047453478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4104101820 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f41027fc640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40f800e9c0 tx=0x7f40f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f800cd80 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f80054d0 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41041a2380 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8010430 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.735+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41041a28a0 con 0x7f41041053a0 2024-04-23T18:26:32.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.736+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41040fdef0 con 0x7f41041053a0 2024-04-23T18:26:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.737+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f40f8004fb0 con 0x7f41041053a0 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.731 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:32 smithi033 ceph-mon[29274]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.738+0000 7f40ebfff640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 0x7f40ec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.738+0000 7f4102ffd640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 0x7f40ec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.738+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f40f80d0f80 con 0x7f41041053a0 2024-04-23T18:26:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.738+0000 7f4102ffd640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 0x7f40ec0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005fd0 tx=0x7f40f4009680 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.742+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40f8053ff0 con 0x7f41041053a0 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:32 smithi106 ceph-mon[32716]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.932+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f410402d810 con 0x7f41041053a0 2024-04-23T18:26:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.933+0000 7f40ebfff640 1 -- 172.21.15.33:0/1047453478 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f40f809d4c0 con 0x7f41041053a0 2024-04-23T18:26:32.926 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 msgr2=0x7f40ec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 0x7f40ec0a6270 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005fd0 tx=0x7f40f4009680 comp rx=0 tx=0).stop 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 msgr2=0x7f4104100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f40f800e9c0 tx=0x7f40f800ee80 comp rx=0 tx=0).stop 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 shutdown_connections 2024-04-23T18:26:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40ec0a3db0 0x7f40ec0a6270 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4104107cd0 0x7f4104100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41041053a0 0x7f4104100a20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 --2- 172.21.15.33:0/1047453478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4104102a70 0x7f4104102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 >> 172.21.15.33:0/1047453478 conn(0x7f41040fc400 msgr2=0x7f41041034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 shutdown_connections 2024-04-23T18:26:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:32.935+0000 7f41095a8640 1 -- 172.21.15.33:0/1047453478 wait complete. 2024-04-23T18:26:32.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:32.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:32 smithi122 ceph-mon[32546]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 -- 172.21.15.33:0/2422326304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4107c50 msgr2=0x7f01e410a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/2422326304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4107c50 0x7f01e410a0a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009ca0 tx=0x7f01dc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 -- 172.21.15.33:0/2422326304 shutdown_connections 2024-04-23T18:26:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/2422326304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4107c50 0x7f01e410a0a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/2422326304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4105320 0x7f01e4107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/2422326304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 -- 172.21.15.33:0/2422326304 >> 172.21.15.33:0/2422326304 conn(0x7f01e40fc3e0 msgr2=0x7f01e40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 -- 172.21.15.33:0/2422326304 shutdown_connections 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.053+0000 7f01e8e7c640 1 -- 172.21.15.33:0/2422326304 wait complete. 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 Processor -- start 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 -- start start 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 0x7f01e4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4107c50 0x7f01e419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410daa0 con 0x7f01e4105320 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410d920 con 0x7f01e4107c50 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e8e7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01e410dc20 con 0x7f01e41029f0 2024-04-23T18:26:33.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 0x7f01e4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 0x7f01e4198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44154/0 (socket says 172.21.15.33:44154) 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.054+0000 7f01e1d9b640 1 -- 172.21.15.33:0/1802535040 learned_addr learned my addr 172.21.15.33:0/1802535040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e2d9d640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4107c50 0x7f01e419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 -- 172.21.15.33:0/1802535040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4107c50 msgr2=0x7f01e419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4107c50 0x7f01e419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 -- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 msgr2=0x7f01e4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 0x7f01e4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 -- 172.21.15.33:0/1802535040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e419fd70 con 0x7f01e41029f0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e259c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4198150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e9f0 tx=0x7f01cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc00ccc0 con 0x7f01e41029f0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41a0060 con 0x7f01e41029f0 2024-04-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01cc0108e0 con 0x7f01e41029f0 2024-04-23T18:26:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc018880 con 0x7f01e41029f0 2024-04-23T18:26:33.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.055+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41a05b0 con 0x7f01e41029f0 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.056+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40fded0 con 0x7f01e41029f0 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.057+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f01cc010430 con 0x7f01e41029f0 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.058+0000 7f01db7fe640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 0x7f01d00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.058+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f01cc0d0dc0 con 0x7f01e41029f0 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.058+0000 7f01e1d9b640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 0x7f01d00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.059+0000 7f01e1d9b640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 0x7f01d00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d40095f0 tx=0x7f01d4005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.062+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01cc09cce0 con 0x7f01e41029f0 2024-04-23T18:26:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.281+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f01e402d770 con 0x7f01e41029f0 2024-04-23T18:26:33.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.282+0000 7f01db7fe640 1 -- 172.21.15.33:0/1802535040 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f01cc0a0fe0 con 0x7f01e41029f0 2024-04-23T18:26:33.275 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 msgr2=0x7f01d00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 0x7f01d00a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f01d40095f0 tx=0x7f01d4005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 msgr2=0x7f01e4198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4198150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00e9f0 tx=0x7f01cc00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 shutdown_connections 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f01d00a3db0 0x7f01d00a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e4107c50 0x7f01e419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01e4105320 0x7f01e4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 --2- 172.21.15.33:0/1802535040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01e41029f0 0x7f01e4198150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 >> 172.21.15.33:0/1802535040 conn(0x7f01e40fc3e0 msgr2=0x7f01e4103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 shutdown_connections 2024-04-23T18:26:33.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.284+0000 7f01e8e7c640 1 -- 172.21.15.33:0/1802535040 wait complete. 2024-04-23T18:26:33.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- 172.21.15.33:0/4283342094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21301053a0 msgr2=0x7f2130107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 --2- 172.21.15.33:0/4283342094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21301053a0 0x7f2130107790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2124009860 tx=0x7f212402a340 comp rx=0 tx=0).stop 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- 172.21.15.33:0/4283342094 shutdown_connections 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 --2- 172.21.15.33:0/4283342094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130107cd0 0x7f213010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 --2- 172.21.15.33:0/4283342094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21301053a0 0x7f2130107790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 --2- 172.21.15.33:0/4283342094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2130102a70 0x7f2130104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- 172.21.15.33:0/4283342094 >> 172.21.15.33:0/4283342094 conn(0x7f21300fc440 msgr2=0x7f21300fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- 172.21.15.33:0/4283342094 shutdown_connections 2024-04-23T18:26:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- 172.21.15.33:0/4283342094 wait complete. 2024-04-23T18:26:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 Processor -- start 2024-04-23T18:26:33.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.403+0000 7f2136751640 1 -- start start 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 0x7f2130100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2130107cd0 0x7f2130100f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f213010daa0 con 0x7f2130102a70 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f213010d920 con 0x7f2130107cd0 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2136751640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f213010dc20 con 0x7f21301053a0 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44170/0 (socket says 172.21.15.33:44170) 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 -- 172.21.15.33:0/4216746761 learned_addr learned my addr 172.21.15.33:0/4216746761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f2134cee640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2130107cd0 0x7f2130100f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212f7fe640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 0x7f2130100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 -- 172.21.15.33:0/4216746761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 msgr2=0x7f2130100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 0x7f2130100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 -- 172.21.15.33:0/4216746761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2130107cd0 msgr2=0x7f2130100f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2130107cd0 0x7f2130100f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212ffff640 1 -- 172.21.15.33:0/4216746761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21301017f0 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.404+0000 7f212f7fe640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 0x7f2130100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f212ffff640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9c0 tx=0x7f212000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212000cd80 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21200035f0 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21301a2340 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120010690 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.405+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21301a28b0 con 0x7f2130102a70 2024-04-23T18:26:33.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.406+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21300fdf30 con 0x7f2130102a70 2024-04-23T18:26:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.407+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7f2120004000 con 0x7f2130102a70 2024-04-23T18:26:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.408+0000 7f212d7fa640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 0x7f21180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.408+0000 7f212f7fe640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 0x7f21180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.408+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f21200d0c30 con 0x7f2130102a70 2024-04-23T18:26:33.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.408+0000 7f212f7fe640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 0x7f21180a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f212402a930 tx=0x7f2124030470 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.413+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f212009d110 con 0x7f2130102a70 2024-04-23T18:26:33.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.618+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2130062b50 con 0x7f2130102a70 2024-04-23T18:26:33.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.618+0000 7f212d7fa640 1 -- 172.21.15.33:0/4216746761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f212009ca10 con 0x7f2130102a70 2024-04-23T18:26:33.611 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:26:33.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.620+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 msgr2=0x7f21180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.620+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 0x7f21180a62f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f212402a930 tx=0x7f2124030470 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.620+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 msgr2=0x7f21301023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.620+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f212000e9c0 tx=0x7f212000edc0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 shutdown_connections 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f21180a3e30 0x7f21180a62f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2130107cd0 0x7f2130100f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21301053a0 0x7f2130100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 --2- 172.21.15.33:0/4216746761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2130102a70 0x7f21301023e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 >> 172.21.15.33:0/4216746761 conn(0x7f21300fc440 msgr2=0x7f2130103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 shutdown_connections 2024-04-23T18:26:33.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.621+0000 7f2136751640 1 -- 172.21.15.33:0/4216746761 wait complete. 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3857739716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1047453478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1802535040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.628 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 -- 172.21.15.33:0/96047348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70101a50 msgr2=0x7fcb70105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/96047348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70101a50 0x7fcb70105a90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009900 tx=0x7fcb5802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 -- 172.21.15.33:0/96047348 shutdown_connections 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/96047348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb70106170 0x7fcb7010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/96047348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70101a50 0x7fcb70105a90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/96047348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 -- 172.21.15.33:0/96047348 >> 172.21.15.33:0/96047348 conn(0x7fcb70077f50 msgr2=0x7fcb70078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 -- 172.21.15.33:0/96047348 shutdown_connections 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.739+0000 7fcb74bc6640 1 -- 172.21.15.33:0/96047348 wait complete. 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 Processor -- start 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 -- start start 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb7019d000 0x7fcb7019d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb701110b0 con 0x7fcb70106170 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70110f30 con 0x7fcb701010a0 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb74bc6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70111230 con 0x7fcb7019d000 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44182/0 (socket says 172.21.15.33:44182) 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6dd9b640 1 -- 172.21.15.33:0/2141983198 learned_addr learned my addr 172.21.15.33:0/2141983198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6ed9d640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb7019d000 0x7fcb7019d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6e59c640 1 -- 172.21.15.33:0/2141983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb7019d000 msgr2=0x7fcb7019d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.740+0000 7fcb6e59c640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb7019d000 0x7fcb7019d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb6e59c640 1 -- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 msgr2=0x7fcb70198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb6e59c640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb6e59c640 1 -- 172.21.15.33:0/2141983198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7019daf0 con 0x7fcb701010a0 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb6dd9b640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb6e59c640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70198740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9f0 tx=0x7fcb6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800cd80 con 0x7fcb701010a0 2024-04-23T18:26:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb680035f0 con 0x7fcb701010a0 2024-04-23T18:26:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7019dde0 con 0x7fcb701010a0 2024-04-23T18:26:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68010670 con 0x7fcb701010a0 2024-04-23T18:26:33.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.741+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb70102d80 con 0x7fcb701010a0 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.742+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb70068d70 con 0x7fcb701010a0 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.743+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fcb680037a0 con 0x7fcb701010a0 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.744+0000 7fcb677fe640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 0x7fcb5c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.744+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fcb680d0c70 con 0x7fcb701010a0 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.744+0000 7fcb6dd9b640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 0x7fcb5c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:33.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.744+0000 7fcb6dd9b640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 0x7fcb5c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcb58005c40 tx=0x7fcb58005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:33.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.749+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb6809d150 con 0x7fcb701010a0 2024-04-23T18:26:33.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.872+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcb68009310 con 0x7fcb701010a0 2024-04-23T18:26:33.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:33.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3857739716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:26:33.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:26:33.873 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1047453478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.873 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1802535040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3857739716' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1047453478' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1802535040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.969+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb7010a3e0 con 0x7fcb701010a0 2024-04-23T18:26:33.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.970+0000 7fcb677fe640 1 -- 172.21.15.33:0/2141983198 <== mon.2 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23101 (secure 0 0 0) 0x7fcb68003ac0 con 0x7fcb701010a0 2024-04-23T18:26:33.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:33.964 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:26:29.462863+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.972+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 msgr2=0x7fcb5c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.972+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 0x7fcb5c0a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcb58005c40 tx=0x7fcb58005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 msgr2=0x7fcb70198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70198740 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800e9f0 tx=0x7fcb6800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 shutdown_connections 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb5c0a3db0 0x7fcb5c0a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb7019d000 0x7fcb7019d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb70106170 0x7fcb70198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 --2- 172.21.15.33:0/2141983198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb701010a0 0x7fcb70198740 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 >> 172.21.15.33:0/2141983198 conn(0x7fcb70077f50 msgr2=0x7fcb701001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 shutdown_connections 2024-04-23T18:26:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:33.973+0000 7fcb74bc6640 1 -- 172.21.15.33:0/2141983198 wait complete. 2024-04-23T18:26:33.979 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:33.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 -- 172.21.15.33:0/3533917277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84106160 msgr2=0x7f4b8410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/3533917277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84106160 0x7f4b8410dd30 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009ca0 tx=0x7f4b7c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 -- 172.21.15.33:0/3533917277 shutdown_connections 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/3533917277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b84106160 0x7f4b8410dd30 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/3533917277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84101a40 0x7f4b84105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/3533917277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 0x7f4b84101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 -- 172.21.15.33:0/3533917277 >> 172.21.15.33:0/3533917277 conn(0x7f4b84077f50 msgr2=0x7f4b84078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 -- 172.21.15.33:0/3533917277 shutdown_connections 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.093+0000 7f4b8ba74640 1 -- 172.21.15.33:0/3533917277 wait complete. 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 Processor -- start 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 -- start start 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 0x7f4b84197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84106160 0x7f4b84197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b841110d0 con 0x7f4b8419bec0 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84110f50 con 0x7f4b84106160 2024-04-23T18:26:34.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8ba74640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b84111250 con 0x7f4b84101090 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44202/0 (socket says 172.21.15.33:44202) 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 -- 172.21.15.33:0/401512533 learned_addr learned my addr 172.21.15.33:0/401512533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b89810640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 0x7f4b84197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8900f640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84106160 0x7f4b84197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 -- 172.21.15.33:0/401512533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 msgr2=0x7f4b84197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 0x7f4b84197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 -- 172.21.15.33:0/401512533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84106160 msgr2=0x7f4b84197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84106160 0x7f4b84197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.094+0000 7f4b8a011640 1 -- 172.21.15.33:0/401512533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8419c9b0 con 0x7f4b8419bec0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b8a011640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005cc0 tx=0x7f4b7c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c003b70 con 0x7f4b8419bec0 2024-04-23T18:26:34.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8419cc40 con 0x7f4b8419bec0 2024-04-23T18:26:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b7c0318f0 con 0x7f4b8419bec0 2024-04-23T18:26:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c02f950 con 0x7f4b8419bec0 2024-04-23T18:26:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.095+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b841a4490 con 0x7f4b8419bec0 2024-04-23T18:26:34.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.096+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b84068d70 con 0x7f4b8419bec0 2024-04-23T18:26:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.097+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f4b7c02fb10 con 0x7f4b8419bec0 2024-04-23T18:26:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.097+0000 7f4b7affd640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 0x7f4b700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.098+0000 7f4b89810640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 0x7f4b700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.098+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f4b7c0f1f20 con 0x7f4b8419bec0 2024-04-23T18:26:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.098+0000 7f4b89810640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 0x7f4b700a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009bd0 tx=0x7f4b80005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.102+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b7c0be490 con 0x7f4b8419bec0 2024-04-23T18:26:34.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.299+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b841093f0 con 0x7f4b8419bec0 2024-04-23T18:26:34.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.299+0000 7f4b7affd640 1 -- 172.21.15.33:0/401512533 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f4b7c0bdd90 con 0x7f4b8419bec0 2024-04-23T18:26:34.292 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 msgr2=0x7f4b700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 0x7f4b700a6300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009bd0 tx=0x7f4b80005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 msgr2=0x7f4b8419c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c005cc0 tx=0x7f4b7c031700 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 shutdown_connections 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4b700a3e40 0x7f4b700a6300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b8419bec0 0x7f4b8419c370 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b84106160 0x7f4b84197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 --2- 172.21.15.33:0/401512533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b84101090 0x7f4b84197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 >> 172.21.15.33:0/401512533 conn(0x7f4b84077f50 msgr2=0x7f4b840fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 shutdown_connections 2024-04-23T18:26:34.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.302+0000 7f4b8ba74640 1 -- 172.21.15.33:0/401512533 wait complete. 2024-04-23T18:26:34.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:34.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.418+0000 7fb605bf7640 1 -- 172.21.15.33:0/4274772257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb600107280 msgr2=0x7fb60010ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.418+0000 7fb605bf7640 1 --2- 172.21.15.33:0/4274772257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb600107280 0x7fb60010ee50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009ca0 tx=0x7fb5f802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 -- 172.21.15.33:0/4274772257 shutdown_connections 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 --2- 172.21.15.33:0/4274772257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb600107280 0x7fb60010ee50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 --2- 172.21.15.33:0/4274772257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb600101070 0x7fb600106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 --2- 172.21.15.33:0/4274772257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6001006c0 0x7fb600100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 -- 172.21.15.33:0/4274772257 >> 172.21.15.33:0/4274772257 conn(0x7fb6000773a0 msgr2=0x7fb6000fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 -- 172.21.15.33:0/4274772257 shutdown_connections 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 -- 172.21.15.33:0/4274772257 wait complete. 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 Processor -- start 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.419+0000 7fb605bf7640 1 -- start start 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb600107280 0x7fb6001a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 0x7fb60019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6001121d0 con 0x7fb6001006c0 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb600112050 con 0x7fb600107280 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb600112350 con 0x7fb60019a710 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44216/0 (socket says 172.21.15.33:44216) 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 -- 172.21.15.33:0/3255653513 learned_addr learned my addr 172.21.15.33:0/3255653513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5feffd640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb600107280 0x7fb6001a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5fffff640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 0x7fb60019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 -- 172.21.15.33:0/3255653513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 msgr2=0x7fb60019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 0x7fb60019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 -- 172.21.15.33:0/3255653513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb600107280 msgr2=0x7fb6001a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb600107280 0x7fb6001a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 -- 172.21.15.33:0/3255653513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60019b310 con 0x7fb6001006c0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5fffff640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 0x7fb60019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5ff7fe640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9f0 tx=0x7fb5f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f000cd80 con 0x7fb6001006c0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5f00108e0 con 0x7fb6001006c0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6001a73f0 con 0x7fb6001006c0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0018980 con 0x7fb6001006c0 2024-04-23T18:26:34.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.420+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6001a77f0 con 0x7fb6001006c0 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.421+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb600103770 con 0x7fb6001006c0 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.422+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fb5f0010430 con 0x7fb6001006c0 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.423+0000 7fb5fcff9640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 0x7fb5e80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.423+0000 7fb5feffd640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 0x7fb5e80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.423+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb5f00d0e70 con 0x7fb6001006c0 2024-04-23T18:26:34.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.424+0000 7fb5feffd640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 0x7fb5e80a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb60019bf60 tx=0x7fb5f4005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.428+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5f009d270 con 0x7fb6001006c0 2024-04-23T18:26:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.617+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb6001010d0 con 0x7fb6001006c0 2024-04-23T18:26:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.617+0000 7fb5fcff9640 1 -- 172.21.15.33:0/3255653513 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fb5f009cb70 con 0x7fb6001006c0 2024-04-23T18:26:34.610 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 msgr2=0x7fb5e80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 0x7fb5e80a6300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb60019bf60 tx=0x7fb5f4005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 msgr2=0x7fb6001a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000e9f0 tx=0x7fb5f000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 shutdown_connections 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5e80a3e40 0x7fb5e80a6300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb60019a710 0x7fb60019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb600107280 0x7fb6001a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 --2- 172.21.15.33:0/3255653513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6001006c0 0x7fb6001a0650 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 >> 172.21.15.33:0/3255653513 conn(0x7fb6000773a0 msgr2=0x7fb600102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 shutdown_connections 2024-04-23T18:26:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.619+0000 7fb605bf7640 1 -- 172.21.15.33:0/3255653513 wait complete. 2024-04-23T18:26:34.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4216746761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2141983198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:34.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:34.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:34 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/401512533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:34.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:34.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.737+0000 7f3cc6491640 1 -- 172.21.15.33:0/718164126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 msgr2=0x7f3cc0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.737+0000 7f3cc6491640 1 --2- 172.21.15.33:0/718164126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc0102570 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0009900 tx=0x7f3cb002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 -- 172.21.15.33:0/718164126 shutdown_connections 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 --2- 172.21.15.33:0/718164126 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc0107260 0x7f3cc010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 --2- 172.21.15.33:0/718164126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0102b40 0x7f3cc0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 --2- 172.21.15.33:0/718164126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc0102570 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 -- 172.21.15.33:0/718164126 >> 172.21.15.33:0/718164126 conn(0x7f3cc00fd990 msgr2=0x7f3cc00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 -- 172.21.15.33:0/718164126 shutdown_connections 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 -- 172.21.15.33:0/718164126 wait complete. 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 Processor -- start 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 -- start start 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.738+0000 7f3cc6491640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0107260 0x7f3cc019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc01985e0 0x7f3cc0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01121a0 con 0x7f3cc0102190 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0112020 con 0x7f3cc0107260 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0112320 con 0x7f3cc01985e0 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44220/0 (socket says 172.21.15.33:44220) 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 -- 172.21.15.33:0/2800176512 learned_addr learned my addr 172.21.15.33:0/2800176512 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc4a2e640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc01985e0 0x7f3cc0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbf7fe640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0107260 0x7f3cc019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 -- 172.21.15.33:0/2800176512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc01985e0 msgr2=0x7f3cc0198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc01985e0 0x7f3cc0198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 -- 172.21.15.33:0/2800176512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0107260 msgr2=0x7f3cc019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0107260 0x7f3cc019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 -- 172.21.15.33:0/2800176512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc0199100 con 0x7f3cc0102190 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbffff640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005940 tx=0x7f3cb0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb003f030 con 0x7f3cc0102190 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cb0003aa0 con 0x7f3cc0102190 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0035070 con 0x7f3cc0102190 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc0199390 con 0x7f3cc0102190 2024-04-23T18:26:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.739+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01a55e0 con 0x7f3cc0102190 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.740+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc00691c0 con 0x7f3cc0102190 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.742+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3cb0042030 con 0x7f3cc0102190 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.742+0000 7f3cbd7fa640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 0x7f3ca80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.742+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3cb00f1270 con 0x7f3cc0102190 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.742+0000 7f3cbf7fe640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 0x7f3ca80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.743+0000 7f3cbf7fe640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 0x7f3ca80a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00fddb0 tx=0x7f3cb4005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.746+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cb00bd7a0 con 0x7f3cc0102190 2024-04-23T18:26:34.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4216746761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2141983198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:34 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/401512533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:34.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.935+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3cc0102b40 con 0x7f3cc0102190 2024-04-23T18:26:34.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.935+0000 7f3cbd7fa640 1 -- 172.21.15.33:0/2800176512 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f3cb00bd0a0 con 0x7f3cc0102190 2024-04-23T18:26:34.928 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:26:34.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 msgr2=0x7f3ca80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 0x7f3ca80a6380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00fddb0 tx=0x7f3cb4005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 msgr2=0x7f3cc019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0005940 tx=0x7f3cb0004270 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 shutdown_connections 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3ca80a3ec0 0x7f3ca80a6380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cc01985e0 0x7f3cc0198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cc0107260 0x7f3cc019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 --2- 172.21.15.33:0/2800176512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3cc0102190 0x7f3cc019e430 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 >> 172.21.15.33:0/2800176512 conn(0x7f3cc00fd990 msgr2=0x7f3cc00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 shutdown_connections 2024-04-23T18:26:34.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:34.937+0000 7f3cc6491640 1 -- 172.21.15.33:0/2800176512 wait complete. 2024-04-23T18:26:34.941 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-23T18:26:34.941 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-23T18:26:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4216746761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: Updating smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2141983198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:26:34.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:34 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/401512533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- 172.21.15.33:0/3780222065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 msgr2=0x7fac98110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- 172.21.15.33:0/3780222065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac98110d00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fac90009ca0 tx=0x7fac9002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- 172.21.15.33:0/3780222065 shutdown_connections 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- 172.21.15.33:0/3780222065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac98110d00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- 172.21.15.33:0/3780222065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98101680 0x7fac9810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- 172.21.15.33:0/3780222065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98100d60 0x7fac98101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- 172.21.15.33:0/3780222065 >> 172.21.15.33:0/3780222065 conn(0x7fac98077f50 msgr2=0x7fac98078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- 172.21.15.33:0/3780222065 shutdown_connections 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- 172.21.15.33:0/3780222065 wait complete. 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 Processor -- start 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- start start 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98100d60 0x7fac981a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 0x7fac981a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac981143b0 con 0x7fac9810e910 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fac98114230 con 0x7fac98101680 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.051+0000 7fac9eedc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac98114530 con 0x7fac98100d60 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44242/0 (socket says 172.21.15.33:44242) 2024-04-23T18:26:35.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9cc78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98100d60 0x7fac981a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 -- 172.21.15.33:0/2274226000 learned_addr learned my addr 172.21.15.33:0/2274226000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac97fff640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 0x7fac981a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 -- 172.21.15.33:0/2274226000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98100d60 msgr2=0x7fac981a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98100d60 0x7fac981a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 -- 172.21.15.33:0/2274226000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 msgr2=0x7fac981a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 0x7fac981a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 -- 172.21.15.33:0/2274226000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9819b030 con 0x7fac9810e910 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac97fff640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 0x7fac981a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9d479640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac900098d0 tx=0x7fac900319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90003b70 con 0x7fac9810e910 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac90003d10 con 0x7fac9810e910 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9819b2c0 con 0x7fac9810e910 2024-04-23T18:26:35.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9002fa50 con 0x7fac9810e910 2024-04-23T18:26:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.052+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac981a75e0 con 0x7fac9810e910 2024-04-23T18:26:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.053+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9819b630 con 0x7fac9810e910 2024-04-23T18:26:35.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.055+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fac90004580 con 0x7fac9810e910 2024-04-23T18:26:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.055+0000 7fac95ffb640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 0x7fac840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.056+0000 7fac9cc78640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 0x7fac840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.056+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fac900f1d50 con 0x7fac9810e910 2024-04-23T18:26:35.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.056+0000 7fac9cc78640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 0x7fac840a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac80005d00 tx=0x7fac8000b410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.061+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac900f5030 con 0x7fac9810e910 2024-04-23T18:26:35.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.243+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac98062390 con 0x7fac840a3e40 2024-04-23T18:26:35.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.251+0000 7fac95ffb640 1 -- 172.21.15.33:0/2274226000 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258062 (secure 0 0 0) 0x7fac98062390 con 0x7fac840a3e40 2024-04-23T18:26:35.249 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:35.251 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:26:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 msgr2=0x7fac840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 0x7fac840a6300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac80005d00 tx=0x7fac8000b410 comp rx=0 tx=0).stop 2024-04-23T18:26:35.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 msgr2=0x7fac9819a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac900098d0 tx=0x7fac900319f0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 shutdown_connections 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fac840a3e40 0x7fac840a6300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac9810e910 0x7fac9819a770 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac98101680 0x7fac981a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 --2- 172.21.15.33:0/2274226000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac98100d60 0x7fac981a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 >> 172.21.15.33:0/2274226000 conn(0x7fac98077f50 msgr2=0x7fac9810ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 shutdown_connections 2024-04-23T18:26:35.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.254+0000 7fac9eedc640 1 -- 172.21.15.33:0/2274226000 wait complete. 2024-04-23T18:26:35.258 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-23T18:26:33.480413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191511,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1428,"num_read_kb":878,"num_write":559,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381980,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559844,"statfs":{"total":1151940427776,"available":1151549280256,"internally_reserved":0,"allocated":12271616,"data_stored":8756040,"data_compressed":268542,"data_compressed_allocated":356352,"data_compressed_original":3219456,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.985415"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468747+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:26:29.468747+0000","last_peered":"2024-04-23T18:26:29.468747+0000","last_clean":"2024-04-23T18:26:29.468747+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:26:29.468747+0000","last_undegraded":"2024-04-23T18:26:29.468747+0000","last_fullsized":"2024-04-23T18:26:29.468747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683825+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:26:29.683825+0000","last_peered":"2024-04-23T18:26:29.683825+0000","last_clean":"2024-04-23T18:26:29.683825+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:26:29.683825+0000","last_undegraded":"2024-04-23T18:26:29.683825+0000","last_fullsized":"2024-04-23T18:26:29.683825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683773+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:26:29.683773+0000","last_peered":"2024-04-23T18:26:29.683773+0000","last_clean":"2024-04-23T18:26:29.683773+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:26:29.683773+0000","last_undegraded":"2024-04-23T18:26:29.683773+0000","last_fullsized":"2024-04-23T18:26:29.683773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525627+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:26:29.525627+0000","last_peered":"2024-04-23T18:26:29.525627+0000","last_clean":"2024-04-23T18:26:29.525627+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:26:29.525627+0000","last_undegraded":"2024-04-23T18:26:29.525627+0000","last_fullsized":"2024-04-23T18:26:29.525627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563249+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:26:29.563249+0000","last_peered":"2024-04-23T18:26:29.563249+0000","last_clean":"2024-04-23T18:26:29.563249+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:26:29.563249+0000","last_undegraded":"2024-04-23T18:26:29.563249+0000","last_fullsized":"2024-04-23T18:26:29.563249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469383+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:26:29.469383+0000","last_peered":"2024-04-23T18:26:29.469383+0000","last_clean":"2024-04-23T18:26:29.469383+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:26:29.469383+0000","last_undegraded":"2024-04-23T18:26:29.469383+0000","last_fullsized":"2024-04-23T18:26:29.469383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523261+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:26:29.523261+0000","last_peered":"2024-04-23T18:26:29.523261+0000","last_clean":"2024-04-23T18:26:29.523261+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:26:29.523261+0000","last_undegraded":"2024-04-23T18:26:29.523261+0000","last_fullsized":"2024-04-23T18:26:29.523261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525241+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:26:29.525241+0000","last_peered":"2024-04-23T18:26:29.525241+0000","last_clean":"2024-04-23T18:26:29.525241+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:26:29.525241+0000","last_undegraded":"2024-04-23T18:26:29.525241+0000","last_fullsized":"2024-04-23T18:26:29.525241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683743+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:26:29.683743+0000","last_peered":"2024-04-23T18:26:29.683743+0000","last_clean":"2024-04-23T18:26:29.683743+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:26:29.683743+0000","last_undegraded":"2024-04-23T18:26:29.683743+0000","last_fullsized":"2024-04-23T18:26:29.683743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469827+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:26:29.469827+0000","last_peered":"2024-04-23T18:26:29.469827+0000","last_clean":"2024-04-23T18:26:29.469827+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:26:29.469827+0000","last_undegraded":"2024-04-23T18:26:29.469827+0000","last_fullsized":"2024-04-23T18:26:29.469827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523623+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:26:29.523623+0000","last_peered":"2024-04-23T18:26:29.523623+0000","last_clean":"2024-04-23T18:26:29.523623+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:26:29.523623+0000","last_undegraded":"2024-04-23T18:26:29.523623+0000","last_fullsized":"2024-04-23T18:26:29.523623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525576+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:26:29.525576+0000","last_peered":"2024-04-23T18:26:29.525576+0000","last_clean":"2024-04-23T18:26:29.525576+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:26:29.525576+0000","last_undegraded":"2024-04-23T18:26:29.525576+0000","last_fullsized":"2024-04-23T18:26:29.525576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523055+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:26:29.523055+0000","last_peered":"2024-04-23T18:26:29.523055+0000","last_clean":"2024-04-23T18:26:29.523055+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:26:29.523055+0000","last_undegraded":"2024-04-23T18:26:29.523055+0000","last_fullsized":"2024-04-23T18:26:29.523055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683439+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:26:29.683439+0000","last_peered":"2024-04-23T18:26:29.683439+0000","last_clean":"2024-04-23T18:26:29.683439+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:26:29.683439+0000","last_undegraded":"2024-04-23T18:26:29.683439+0000","last_fullsized":"2024-04-23T18:26:29.683439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683324+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:26:29.683324+0000","last_peered":"2024-04-23T18:26:29.683324+0000","last_clean":"2024-04-23T18:26:29.683324+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:26:29.683324+0000","last_undegraded":"2024-04-23T18:26:29.683324+0000","last_fullsized":"2024-04-23T18:26:29.683324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683321+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:26:29.683321+0000","last_peered":"2024-04-23T18:26:29.683321+0000","last_clean":"2024-04-23T18:26:29.683321+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:26:29.683321+0000","last_undegraded":"2024-04-23T18:26:29.683321+0000","last_fullsized":"2024-04-23T18:26:29.683321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468714+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:26:29.468714+0000","last_peered":"2024-04-23T18:26:29.468714+0000","last_clean":"2024-04-23T18:26:29.468714+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:26:29.468714+0000","last_undegraded":"2024-04-23T18:26:29.468714+0000","last_fullsized":"2024-04-23T18:26:29.468714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467817+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:26:29.467817+0000","last_peered":"2024-04-23T18:26:29.467817+0000","last_clean":"2024-04-23T18:26:29.467817+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:26:29.467817+0000","last_undegraded":"2024-04-23T18:26:29.467817+0000","last_fullsized":"2024-04-23T18:26:29.467817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543091+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:26:29.543091+0000","last_peered":"2024-04-23T18:26:29.543091+0000","last_clean":"2024-04-23T18:26:29.543091+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:26:29.543091+0000","last_undegraded":"2024-04-23T18:26:29.543091+0000","last_fullsized":"2024-04-23T18:26:29.543091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":153,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717616+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:26:29.717616+0000","last_peered":"2024-04-23T18:26:29.717616+0000","last_clean":"2024-04-23T18:26:29.717616+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:26:29.717616+0000","last_undegraded":"2024-04-23T18:26:29.717616+0000","last_fullsized":"2024-04-23T18:26:29.717616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525570+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:26:29.525570+0000","last_peered":"2024-04-23T18:26:29.525570+0000","last_clean":"2024-04-23T18:26:29.525570+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:26:29.525570+0000","last_undegraded":"2024-04-23T18:26:29.525570+0000","last_fullsized":"2024-04-23T18:26:29.525570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525747+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:26:29.525747+0000","last_peered":"2024-04-23T18:26:29.525747+0000","last_clean":"2024-04-23T18:26:29.525747+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:26:29.525747+0000","last_undegraded":"2024-04-23T18:26:29.525747+0000","last_fullsized":"2024-04-23T18:26:29.525747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683868+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:26:29.683868+0000","last_peered":"2024-04-23T18:26:29.683868+0000","last_clean":"2024-04-23T18:26:29.683868+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:26:29.683868+0000","last_undegraded":"2024-04-23T18:26:29.683868+0000","last_fullsized":"2024-04-23T18:26:29.683868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468673+0000","last_change":"2024-04-23T18:25:25.931554+0000","last_active":"2024-04-23T18:26:29.468673+0000","last_peered":"2024-04-23T18:26:29.468673+0000","last_clean":"2024-04-23T18:26:29.468673+0000","last_became_active":"2024-04-23T18:25:25.919193+0000","last_became_peered":"2024-04-23T18:25:25.919193+0000","last_unstale":"2024-04-23T18:26:29.468673+0000","last_undegraded":"2024-04-23T18:26:29.468673+0000","last_fullsized":"2024-04-23T18:26:29.468673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:34.700255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683406+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:26:29.683406+0000","last_peered":"2024-04-23T18:26:29.683406+0000","last_clean":"2024-04-23T18:26:29.683406+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:26:29.683406+0000","last_undegraded":"2024-04-23T18:26:29.683406+0000","last_fullsized":"2024-04-23T18:26:29.683406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563453+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:26:29.563453+0000","last_peered":"2024-04-23T18:26:29.563453+0000","last_clean":"2024-04-23T18:26:29.563453+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:26:29.563453+0000","last_undegraded":"2024-04-23T18:26:29.563453+0000","last_fullsized":"2024-04-23T18:26:29.563453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524606+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:26:29.524606+0000","last_peered":"2024-04-23T18:26:29.524606+0000","last_clean":"2024-04-23T18:26:29.524606+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:26:29.524606+0000","last_undegraded":"2024-04-23T18:26:29.524606+0000","last_fullsized":"2024-04-23T18:26:29.524606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":150,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524778+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:26:29.524778+0000","last_peered":"2024-04-23T18:26:29.524778+0000","last_clean":"2024-04-23T18:26:29.524778+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:26:29.524778+0000","last_undegraded":"2024-04-23T18:26:29.524778+0000","last_fullsized":"2024-04-23T18:26:29.524778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563167+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:26:29.563167+0000","last_peered":"2024-04-23T18:26:29.563167+0000","last_clean":"2024-04-23T18:26:29.563167+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:26:29.563167+0000","last_undegraded":"2024-04-23T18:26:29.563167+0000","last_fullsized":"2024-04-23T18:26:29.563167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469343+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:26:29.469343+0000","last_peered":"2024-04-23T18:26:29.469343+0000","last_clean":"2024-04-23T18:26:29.469343+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:26:29.469343+0000","last_undegraded":"2024-04-23T18:26:29.469343+0000","last_fullsized":"2024-04-23T18:26:29.469343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524519+0000","last_change":"2024-04-23T18:25:25.919173+0000","last_active":"2024-04-23T18:26:29.524519+0000","last_peered":"2024-04-23T18:26:29.524519+0000","last_clean":"2024-04-23T18:26:29.524519+0000","last_became_active":"2024-04-23T18:25:25.918830+0000","last_became_peered":"2024-04-23T18:25:25.918830+0000","last_unstale":"2024-04-23T18:26:29.524519+0000","last_undegraded":"2024-04-23T18:26:29.524519+0000","last_fullsized":"2024-04-23T18:26:29.524519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:51.415326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523056+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:26:29.523056+0000","last_peered":"2024-04-23T18:26:29.523056+0000","last_clean":"2024-04-23T18:26:29.523056+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:26:29.523056+0000","last_undegraded":"2024-04-23T18:26:29.523056+0000","last_fullsized":"2024-04-23T18:26:29.523056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525205+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:26:29.525205+0000","last_peered":"2024-04-23T18:26:29.525205+0000","last_clean":"2024-04-23T18:26:29.525205+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:26:29.525205+0000","last_undegraded":"2024-04-23T18:26:29.525205+0000","last_fullsized":"2024-04-23T18:26:29.525205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524341+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:26:29.524341+0000","last_peered":"2024-04-23T18:26:29.524341+0000","last_clean":"2024-04-23T18:26:29.524341+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:26:29.524341+0000","last_undegraded":"2024-04-23T18:26:29.524341+0000","last_fullsized":"2024-04-23T18:26:29.524341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467566+0000","last_change":"2024-04-23T18:25:26.153727+0000","last_active":"2024-04-23T18:26:29.467566+0000","last_peered":"2024-04-23T18:26:29.467566+0000","last_clean":"2024-04-23T18:26:29.467566+0000","last_became_active":"2024-04-23T18:25:26.153420+0000","last_became_peered":"2024-04-23T18:25:26.153420+0000","last_unstale":"2024-04-23T18:26:29.467566+0000","last_undegraded":"2024-04-23T18:26:29.467566+0000","last_fullsized":"2024-04-23T18:26:29.467566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:20.614833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.541194+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:26:29.541194+0000","last_peered":"2024-04-23T18:26:29.541194+0000","last_clean":"2024-04-23T18:26:29.541194+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:26:29.541194+0000","last_undegraded":"2024-04-23T18:26:29.541194+0000","last_fullsized":"2024-04-23T18:26:29.541194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683662+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:26:29.683662+0000","last_peered":"2024-04-23T18:26:29.683662+0000","last_clean":"2024-04-23T18:26:29.683662+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:26:29.683662+0000","last_undegraded":"2024-04-23T18:26:29.683662+0000","last_fullsized":"2024-04-23T18:26:29.683662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523432+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:26:29.523432+0000","last_peered":"2024-04-23T18:26:29.523432+0000","last_clean":"2024-04-23T18:26:29.523432+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:26:29.523432+0000","last_undegraded":"2024-04-23T18:26:29.523432+0000","last_fullsized":"2024-04-23T18:26:29.523432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683784+0000","last_change":"2024-04-23T18:25:25.916633+0000","last_active":"2024-04-23T18:26:29.683784+0000","last_peered":"2024-04-23T18:26:29.683784+0000","last_clean":"2024-04-23T18:26:29.683784+0000","last_became_active":"2024-04-23T18:25:25.916224+0000","last_became_peered":"2024-04-23T18:25:25.916224+0000","last_unstale":"2024-04-23T18:26:29.683784+0000","last_undegraded":"2024-04-23T18:26:29.683784+0000","last_fullsized":"2024-04-23T18:26:29.683784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:10.172049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524631+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:26:29.524631+0000","last_peered":"2024-04-23T18:26:29.524631+0000","last_clean":"2024-04-23T18:26:29.524631+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:26:29.524631+0000","last_undegraded":"2024-04-23T18:26:29.524631+0000","last_fullsized":"2024-04-23T18:26:29.524631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683551+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:26:29.683551+0000","last_peered":"2024-04-23T18:26:29.683551+0000","last_clean":"2024-04-23T18:26:29.683551+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:26:29.683551+0000","last_undegraded":"2024-04-23T18:26:29.683551+0000","last_fullsized":"2024-04-23T18:26:29.683551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'40","reported_seq":302,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.715191+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:26:29.715191+0000","last_peered":"2024-04-23T18:26:29.715191+0000","last_clean":"2024-04-23T18:26:29.715191+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:26:29.715191+0000","last_undegraded":"2024-04-23T18:26:29.715191+0000","last_fullsized":"2024-04-23T18:26:29.715191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":185,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563324+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:26:29.563324+0000","last_peered":"2024-04-23T18:26:29.563324+0000","last_clean":"2024-04-23T18:26:29.563324+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:26:29.563324+0000","last_undegraded":"2024-04-23T18:26:29.563324+0000","last_fullsized":"2024-04-23T18:26:29.563324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542938+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:26:29.542938+0000","last_peered":"2024-04-23T18:26:29.542938+0000","last_clean":"2024-04-23T18:26:29.542938+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:26:29.542938+0000","last_undegraded":"2024-04-23T18:26:29.542938+0000","last_fullsized":"2024-04-23T18:26:29.542938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523077+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:26:29.523077+0000","last_peered":"2024-04-23T18:26:29.523077+0000","last_clean":"2024-04-23T18:26:29.523077+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:26:29.523077+0000","last_undegraded":"2024-04-23T18:26:29.523077+0000","last_fullsized":"2024-04-23T18:26:29.523077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524481+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:26:29.524481+0000","last_peered":"2024-04-23T18:26:29.524481+0000","last_clean":"2024-04-23T18:26:29.524481+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:26:29.524481+0000","last_undegraded":"2024-04-23T18:26:29.524481+0000","last_fullsized":"2024-04-23T18:26:29.524481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523707+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:26:29.523707+0000","last_peered":"2024-04-23T18:26:29.523707+0000","last_clean":"2024-04-23T18:26:29.523707+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:26:29.523707+0000","last_undegraded":"2024-04-23T18:26:29.523707+0000","last_fullsized":"2024-04-23T18:26:29.523707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525703+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:26:29.525703+0000","last_peered":"2024-04-23T18:26:29.525703+0000","last_clean":"2024-04-23T18:26:29.525703+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:26:29.525703+0000","last_undegraded":"2024-04-23T18:26:29.525703+0000","last_fullsized":"2024-04-23T18:26:29.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523212+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:26:29.523212+0000","last_peered":"2024-04-23T18:26:29.523212+0000","last_clean":"2024-04-23T18:26:29.523212+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:26:29.523212+0000","last_undegraded":"2024-04-23T18:26:29.523212+0000","last_fullsized":"2024-04-23T18:26:29.523212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563071+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:26:29.563071+0000","last_peered":"2024-04-23T18:26:29.563071+0000","last_clean":"2024-04-23T18:26:29.563071+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:26:29.563071+0000","last_undegraded":"2024-04-23T18:26:29.563071+0000","last_fullsized":"2024-04-23T18:26:29.563071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524609+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:26:29.524609+0000","last_peered":"2024-04-23T18:26:29.524609+0000","last_clean":"2024-04-23T18:26:29.524609+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:26:29.524609+0000","last_undegraded":"2024-04-23T18:26:29.524609+0000","last_fullsized":"2024-04-23T18:26:29.524609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563501+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:26:29.563501+0000","last_peered":"2024-04-23T18:26:29.563501+0000","last_clean":"2024-04-23T18:26:29.563501+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:26:29.563501+0000","last_undegraded":"2024-04-23T18:26:29.563501+0000","last_fullsized":"2024-04-23T18:26:29.563501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468601+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:26:29.468601+0000","last_peered":"2024-04-23T18:26:29.468601+0000","last_clean":"2024-04-23T18:26:29.468601+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:26:29.468601+0000","last_undegraded":"2024-04-23T18:26:29.468601+0000","last_fullsized":"2024-04-23T18:26:29.468601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467652+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:26:29.467652+0000","last_peered":"2024-04-23T18:26:29.467652+0000","last_clean":"2024-04-23T18:26:29.467652+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:26:29.467652+0000","last_undegraded":"2024-04-23T18:26:29.467652+0000","last_fullsized":"2024-04-23T18:26:29.467652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469742+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:26:29.469742+0000","last_peered":"2024-04-23T18:26:29.469742+0000","last_clean":"2024-04-23T18:26:29.469742+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:26:29.469742+0000","last_undegraded":"2024-04-23T18:26:29.469742+0000","last_fullsized":"2024-04-23T18:26:29.469742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717743+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:26:29.717743+0000","last_peered":"2024-04-23T18:26:29.717743+0000","last_clean":"2024-04-23T18:26:29.717743+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:26:29.717743+0000","last_undegraded":"2024-04-23T18:26:29.717743+0000","last_fullsized":"2024-04-23T18:26:29.717743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469572+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:26:29.469572+0000","last_peered":"2024-04-23T18:26:29.469572+0000","last_clean":"2024-04-23T18:26:29.469572+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:26:29.469572+0000","last_undegraded":"2024-04-23T18:26:29.469572+0000","last_fullsized":"2024-04-23T18:26:29.469572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524885+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:26:29.524885+0000","last_peered":"2024-04-23T18:26:29.524885+0000","last_clean":"2024-04-23T18:26:29.524885+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:26:29.524885+0000","last_undegraded":"2024-04-23T18:26:29.524885+0000","last_fullsized":"2024-04-23T18:26:29.524885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":146,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.720370+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:26:29.720370+0000","last_peered":"2024-04-23T18:26:29.720370+0000","last_clean":"2024-04-23T18:26:29.720370+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:26:29.720370+0000","last_undegraded":"2024-04-23T18:26:29.720370+0000","last_fullsized":"2024-04-23T18:26:29.720370+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467506+0000","last_change":"2024-04-23T18:25:26.091646+0000","last_active":"2024-04-23T18:26:29.467506+0000","last_peered":"2024-04-23T18:26:29.467506+0000","last_clean":"2024-04-23T18:26:29.467506+0000","last_became_active":"2024-04-23T18:25:26.091303+0000","last_became_peered":"2024-04-23T18:25:26.091303+0000","last_unstale":"2024-04-23T18:26:29.467506+0000","last_undegraded":"2024-04-23T18:26:29.467506+0000","last_fullsized":"2024-04-23T18:26:29.467506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:27.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525409+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:26:29.525409+0000","last_peered":"2024-04-23T18:26:29.525409+0000","last_clean":"2024-04-23T18:26:29.525409+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:26:29.525409+0000","last_undegraded":"2024-04-23T18:26:29.525409+0000","last_fullsized":"2024-04-23T18:26:29.525409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523296+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:26:29.523296+0000","last_peered":"2024-04-23T18:26:29.523296+0000","last_clean":"2024-04-23T18:26:29.523296+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:26:29.523296+0000","last_undegraded":"2024-04-23T18:26:29.523296+0000","last_fullsized":"2024-04-23T18:26:29.523296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.522924+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:26:29.522924+0000","last_peered":"2024-04-23T18:26:29.522924+0000","last_clean":"2024-04-23T18:26:29.522924+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:26:29.522924+0000","last_undegraded":"2024-04-23T18:26:29.522924+0000","last_fullsized":"2024-04-23T18:26:29.522924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523487+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:26:29.523487+0000","last_peered":"2024-04-23T18:26:29.523487+0000","last_clean":"2024-04-23T18:26:29.523487+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:26:29.523487+0000","last_undegraded":"2024-04-23T18:26:29.523487+0000","last_fullsized":"2024-04-23T18:26:29.523487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563362+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:26:29.563362+0000","last_peered":"2024-04-23T18:26:29.563362+0000","last_clean":"2024-04-23T18:26:29.563362+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:26:29.563362+0000","last_undegraded":"2024-04-23T18:26:29.563362+0000","last_fullsized":"2024-04-23T18:26:29.563362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523285+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:26:29.523285+0000","last_peered":"2024-04-23T18:26:29.523285+0000","last_clean":"2024-04-23T18:26:29.523285+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:26:29.523285+0000","last_undegraded":"2024-04-23T18:26:29.523285+0000","last_fullsized":"2024-04-23T18:26:29.523285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.670822+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:26:29.670822+0000","last_peered":"2024-04-23T18:26:29.670822+0000","last_clean":"2024-04-23T18:26:29.670822+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:26:29.670822+0000","last_undegraded":"2024-04-23T18:26:29.670822+0000","last_fullsized":"2024-04-23T18:26:29.670822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525638+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:26:29.525638+0000","last_peered":"2024-04-23T18:26:29.525638+0000","last_clean":"2024-04-23T18:26:29.525638+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:26:29.525638+0000","last_undegraded":"2024-04-23T18:26:29.525638+0000","last_fullsized":"2024-04-23T18:26:29.525638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525536+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:26:29.525536+0000","last_peered":"2024-04-23T18:26:29.525536+0000","last_clean":"2024-04-23T18:26:29.525536+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:26:29.525536+0000","last_undegraded":"2024-04-23T18:26:29.525536+0000","last_fullsized":"2024-04-23T18:26:29.525536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683712+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:26:29.683712+0000","last_peered":"2024-04-23T18:26:29.683712+0000","last_clean":"2024-04-23T18:26:29.683712+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:26:29.683712+0000","last_undegraded":"2024-04-23T18:26:29.683712+0000","last_fullsized":"2024-04-23T18:26:29.683712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683415+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:26:29.683415+0000","last_peered":"2024-04-23T18:26:29.683415+0000","last_clean":"2024-04-23T18:26:29.683415+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:26:29.683415+0000","last_undegraded":"2024-04-23T18:26:29.683415+0000","last_fullsized":"2024-04-23T18:26:29.683415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563093+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:26:29.563093+0000","last_peered":"2024-04-23T18:26:29.563093+0000","last_clean":"2024-04-23T18:26:29.563093+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:26:29.563093+0000","last_undegraded":"2024-04-23T18:26:29.563093+0000","last_fullsized":"2024-04-23T18:26:29.563093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525276+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:26:29.525276+0000","last_peered":"2024-04-23T18:26:29.525276+0000","last_clean":"2024-04-23T18:26:29.525276+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:26:29.525276+0000","last_undegraded":"2024-04-23T18:26:29.525276+0000","last_fullsized":"2024-04-23T18:26:29.525276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468416+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:26:29.468416+0000","last_peered":"2024-04-23T18:26:29.468416+0000","last_clean":"2024-04-23T18:26:29.468416+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:26:29.468416+0000","last_undegraded":"2024-04-23T18:26:29.468416+0000","last_fullsized":"2024-04-23T18:26:29.468416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468530+0000","last_change":"2024-04-23T18:25:25.935443+0000","last_active":"2024-04-23T18:26:29.468530+0000","last_peered":"2024-04-23T18:26:29.468530+0000","last_clean":"2024-04-23T18:26:29.468530+0000","last_became_active":"2024-04-23T18:25:25.919406+0000","last_became_peered":"2024-04-23T18:25:25.919406+0000","last_unstale":"2024-04-23T18:26:29.468530+0000","last_undegraded":"2024-04-23T18:26:29.468530+0000","last_fullsized":"2024-04-23T18:26:29.468530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:10:03.007617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525448+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:26:29.525448+0000","last_peered":"2024-04-23T18:26:29.525448+0000","last_clean":"2024-04-23T18:26:29.525448+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:26:29.525448+0000","last_undegraded":"2024-04-23T18:26:29.525448+0000","last_fullsized":"2024-04-23T18:26:29.525448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563184+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:26:29.563184+0000","last_peered":"2024-04-23T18:26:29.563184+0000","last_clean":"2024-04-23T18:26:29.563184+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:26:29.563184+0000","last_undegraded":"2024-04-23T18:26:29.563184+0000","last_fullsized":"2024-04-23T18:26:29.563184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523641+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:26:29.523641+0000","last_peered":"2024-04-23T18:26:29.523641+0000","last_clean":"2024-04-23T18:26:29.523641+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:26:29.523641+0000","last_undegraded":"2024-04-23T18:26:29.523641+0000","last_fullsized":"2024-04-23T18:26:29.523641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543338+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:26:29.543338+0000","last_peered":"2024-04-23T18:26:29.543338+0000","last_clean":"2024-04-23T18:26:29.543338+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:26:29.543338+0000","last_undegraded":"2024-04-23T18:26:29.543338+0000","last_fullsized":"2024-04-23T18:26:29.543338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523684+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:26:29.523684+0000","last_peered":"2024-04-23T18:26:29.523684+0000","last_clean":"2024-04-23T18:26:29.523684+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:26:29.523684+0000","last_undegraded":"2024-04-23T18:26:29.523684+0000","last_fullsized":"2024-04-23T18:26:29.523684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.668447+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:26:29.668447+0000","last_peered":"2024-04-23T18:26:29.668447+0000","last_clean":"2024-04-23T18:26:29.668447+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:26:29.668447+0000","last_undegraded":"2024-04-23T18:26:29.668447+0000","last_fullsized":"2024-04-23T18:26:29.668447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523512+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:26:29.523512+0000","last_peered":"2024-04-23T18:26:29.523512+0000","last_clean":"2024-04-23T18:26:29.523512+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:26:29.523512+0000","last_undegraded":"2024-04-23T18:26:29.523512+0000","last_fullsized":"2024-04-23T18:26:29.523512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523123+0000","last_change":"2024-04-23T18:25:26.106394+0000","last_active":"2024-04-23T18:26:29.523123+0000","last_peered":"2024-04-23T18:26:29.523123+0000","last_clean":"2024-04-23T18:26:29.523123+0000","last_became_active":"2024-04-23T18:25:26.102178+0000","last_became_peered":"2024-04-23T18:25:26.102178+0000","last_unstale":"2024-04-23T18:26:29.523123+0000","last_undegraded":"2024-04-23T18:26:29.523123+0000","last_fullsized":"2024-04-23T18:26:29.523123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:20.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523645+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:26:29.523645+0000","last_peered":"2024-04-23T18:26:29.523645+0000","last_clean":"2024-04-23T18:26:29.523645+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:26:29.523645+0000","last_undegraded":"2024-04-23T18:26:29.523645+0000","last_fullsized":"2024-04-23T18:26:29.523645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468549+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:26:29.468549+0000","last_peered":"2024-04-23T18:26:29.468549+0000","last_clean":"2024-04-23T18:26:29.468549+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:26:29.468549+0000","last_undegraded":"2024-04-23T18:26:29.468549+0000","last_fullsized":"2024-04-23T18:26:29.468549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542861+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:26:29.542861+0000","last_peered":"2024-04-23T18:26:29.542861+0000","last_clean":"2024-04-23T18:26:29.542861+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:26:29.542861+0000","last_undegraded":"2024-04-23T18:26:29.542861+0000","last_fullsized":"2024-04-23T18:26:29.542861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542855+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:26:29.542855+0000","last_peered":"2024-04-23T18:26:29.542855+0000","last_clean":"2024-04-23T18:26:29.542855+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:26:29.542855+0000","last_undegraded":"2024-04-23T18:26:29.542855+0000","last_fullsized":"2024-04-23T18:26:29.542855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683626+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:26:29.683626+0000","last_peered":"2024-04-23T18:26:29.683626+0000","last_clean":"2024-04-23T18:26:29.683626+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:26:29.683626+0000","last_undegraded":"2024-04-23T18:26:29.683626+0000","last_fullsized":"2024-04-23T18:26:29.683626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523122+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:26:29.523122+0000","last_peered":"2024-04-23T18:26:29.523122+0000","last_clean":"2024-04-23T18:26:29.523122+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:26:29.523122+0000","last_undegraded":"2024-04-23T18:26:29.523122+0000","last_fullsized":"2024-04-23T18:26:29.523122+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468775+0000","last_change":"2024-04-23T18:25:25.915473+0000","last_active":"2024-04-23T18:26:29.468775+0000","last_peered":"2024-04-23T18:26:29.468775+0000","last_clean":"2024-04-23T18:26:29.468775+0000","last_became_active":"2024-04-23T18:25:25.914874+0000","last_became_peered":"2024-04-23T18:25:25.914874+0000","last_unstale":"2024-04-23T18:26:29.468775+0000","last_undegraded":"2024-04-23T18:26:29.468775+0000","last_fullsized":"2024-04-23T18:26:29.468775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:14.348994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543211+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:26:29.543211+0000","last_peered":"2024-04-23T18:26:29.543211+0000","last_clean":"2024-04-23T18:26:29.543211+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:26:29.543211+0000","last_undegraded":"2024-04-23T18:26:29.543211+0000","last_fullsized":"2024-04-23T18:26:29.543211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525495+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:26:29.525495+0000","last_peered":"2024-04-23T18:26:29.525495+0000","last_clean":"2024-04-23T18:26:29.525495+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:26:29.525495+0000","last_undegraded":"2024-04-23T18:26:29.525495+0000","last_fullsized":"2024-04-23T18:26:29.525495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524255+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:26:29.524255+0000","last_peered":"2024-04-23T18:26:29.524255+0000","last_clean":"2024-04-23T18:26:29.524255+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:26:29.524255+0000","last_undegraded":"2024-04-23T18:26:29.524255+0000","last_fullsized":"2024-04-23T18:26:29.524255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525294+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:26:29.525294+0000","last_peered":"2024-04-23T18:26:29.525294+0000","last_clean":"2024-04-23T18:26:29.525294+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:26:29.525294+0000","last_undegraded":"2024-04-23T18:26:29.525294+0000","last_fullsized":"2024-04-23T18:26:29.525294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683418+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:26:29.683418+0000","last_peered":"2024-04-23T18:26:29.683418+0000","last_clean":"2024-04-23T18:26:29.683418+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:26:29.683418+0000","last_undegraded":"2024-04-23T18:26:29.683418+0000","last_fullsized":"2024-04-23T18:26:29.683418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523149+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:26:29.523149+0000","last_peered":"2024-04-23T18:26:29.523149+0000","last_clean":"2024-04-23T18:26:29.523149+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:26:29.523149+0000","last_undegraded":"2024-04-23T18:26:29.523149+0000","last_fullsized":"2024-04-23T18:26:29.523149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468526+0000","last_change":"2024-04-23T18:25:25.917314+0000","last_active":"2024-04-23T18:26:29.468526+0000","last_peered":"2024-04-23T18:26:29.468526+0000","last_clean":"2024-04-23T18:26:29.468526+0000","last_became_active":"2024-04-23T18:25:25.916841+0000","last_became_peered":"2024-04-23T18:25:25.916841+0000","last_unstale":"2024-04-23T18:26:29.468526+0000","last_undegraded":"2024-04-23T18:26:29.468526+0000","last_fullsized":"2024-04-23T18:26:29.468526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:03.413973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107010,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600909,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1292,"num_read_kb":759,"num_write":376,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1802727,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":466241,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1142784,"data_stored":1254964,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":465780,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":859374,"data_compressed":27877,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":1060830,"data_compressed":44401,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":465888,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":533536,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":467227,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":534987,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":990138,"data_compressed":34033,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466281,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1190794,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:35.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:35.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 -- 172.21.15.33:0/3380856781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 msgr2=0x7f236c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 --2- 172.21.15.33:0/3380856781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c1014f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f235c009900 tx=0x7f235c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:35.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 -- 172.21.15.33:0/3380856781 shutdown_connections 2024-04-23T18:26:35.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 --2- 172.21.15.33:0/3380856781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 --2- 172.21.15.33:0/3380856781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c101ac0 0x7f236c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 --2- 172.21.15.33:0/3380856781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c1014f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 -- 172.21.15.33:0/3380856781 >> 172.21.15.33:0/3380856781 conn(0x7f236c077fb0 msgr2=0x7f236c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 -- 172.21.15.33:0/3380856781 shutdown_connections 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 -- 172.21.15.33:0/3380856781 wait complete. 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.408+0000 7f23729cd640 1 Processor -- start 2024-04-23T18:26:35.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 -- start start 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c10f170 0x7f236c1a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f236c112240 con 0x7f236c101110 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1120c0 con 0x7f236c10f170 2024-04-23T18:26:35.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f23729cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1123c0 con 0x7f236c106190 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44262/0 (socket says 172.21.15.33:44262) 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 -- 172.21.15.33:0/3847339901 learned_addr learned my addr 172.21.15.33:0/3847339901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f2370f6a640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c10f170 0x7f236c1a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236b7fe640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 -- 172.21.15.33:0/3847339901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 msgr2=0x7f236c10ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 -- 172.21.15.33:0/3847339901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c10f170 msgr2=0x7f236c1a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c10f170 0x7f236c1a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236bfff640 1 -- 172.21.15.33:0/3847339901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c1a70d0 con 0x7f236c101110 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.409+0000 7f236b7fe640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f236bfff640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ab0 tx=0x7f235c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c03f070 con 0x7f236c101110 2024-04-23T18:26:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235c031d60 con 0x7f236c101110 2024-04-23T18:26:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c035030 con 0x7f236c101110 2024-04-23T18:26:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c1a7360 con 0x7f236c101110 2024-04-23T18:26:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.410+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a78a0 con 0x7f236c101110 2024-04-23T18:26:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.411+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c0691c0 con 0x7f236c101110 2024-04-23T18:26:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.412+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f235c042070 con 0x7f236c101110 2024-04-23T18:26:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.413+0000 7f23697fa640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 0x7f23540a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.413+0000 7f236b7fe640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 0x7f23540a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.413+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f235c0f1180 con 0x7f236c101110 2024-04-23T18:26:35.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.413+0000 7f236b7fe640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 0x7f23540a63b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f236c110180 tx=0x7f2360005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.418+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f235c0bd650 con 0x7f236c101110 2024-04-23T18:26:35.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.605+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f236c109410 con 0x7f23540a3ef0 2024-04-23T18:26:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.612+0000 7f23697fa640 1 -- 172.21.15.33:0/3847339901 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258064 (secure 0 0 0) 0x7f236c109410 con 0x7f23540a3ef0 2024-04-23T18:26:35.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:26:35.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.615+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 msgr2=0x7f23540a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.615+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 0x7f23540a63b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f236c110180 tx=0x7f2360005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.615+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 msgr2=0x7f236c10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.615+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ab0 tx=0x7f235c003c80 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 shutdown_connections 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f23540a3ef0 0x7f23540a63b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f236c10f170 0x7f236c1a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f236c106190 0x7f236c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 --2- 172.21.15.33:0/3847339901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f236c101110 0x7f236c10e6f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 >> 172.21.15.33:0/3847339901 conn(0x7f236c077fb0 msgr2=0x7f236c0ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 shutdown_connections 2024-04-23T18:26:35.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.616+0000 7f23729cd640 1 -- 172.21.15.33:0/3847339901 wait complete. 2024-04-23T18:26:35.619 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:26:35.480747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191511,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1433,"num_read_kb":883,"num_write":559,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382012,"kb_used_data":12008,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559812,"statfs":{"total":1151940427776,"available":1151549247488,"internally_reserved":0,"allocated":12296192,"data_stored":8953063,"data_compressed":277731,"data_compressed_allocated":368640,"data_compressed_original":3403776,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985749"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468747+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:26:29.468747+0000","last_peered":"2024-04-23T18:26:29.468747+0000","last_clean":"2024-04-23T18:26:29.468747+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:26:29.468747+0000","last_undegraded":"2024-04-23T18:26:29.468747+0000","last_fullsized":"2024-04-23T18:26:29.468747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683825+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:26:29.683825+0000","last_peered":"2024-04-23T18:26:29.683825+0000","last_clean":"2024-04-23T18:26:29.683825+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:26:29.683825+0000","last_undegraded":"2024-04-23T18:26:29.683825+0000","last_fullsized":"2024-04-23T18:26:29.683825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683773+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:26:29.683773+0000","last_peered":"2024-04-23T18:26:29.683773+0000","last_clean":"2024-04-23T18:26:29.683773+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:26:29.683773+0000","last_undegraded":"2024-04-23T18:26:29.683773+0000","last_fullsized":"2024-04-23T18:26:29.683773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525627+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:26:29.525627+0000","last_peered":"2024-04-23T18:26:29.525627+0000","last_clean":"2024-04-23T18:26:29.525627+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:26:29.525627+0000","last_undegraded":"2024-04-23T18:26:29.525627+0000","last_fullsized":"2024-04-23T18:26:29.525627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563249+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:26:29.563249+0000","last_peered":"2024-04-23T18:26:29.563249+0000","last_clean":"2024-04-23T18:26:29.563249+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:26:29.563249+0000","last_undegraded":"2024-04-23T18:26:29.563249+0000","last_fullsized":"2024-04-23T18:26:29.563249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469383+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:26:29.469383+0000","last_peered":"2024-04-23T18:26:29.469383+0000","last_clean":"2024-04-23T18:26:29.469383+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:26:29.469383+0000","last_undegraded":"2024-04-23T18:26:29.469383+0000","last_fullsized":"2024-04-23T18:26:29.469383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523261+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:26:29.523261+0000","last_peered":"2024-04-23T18:26:29.523261+0000","last_clean":"2024-04-23T18:26:29.523261+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:26:29.523261+0000","last_undegraded":"2024-04-23T18:26:29.523261+0000","last_fullsized":"2024-04-23T18:26:29.523261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525241+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:26:29.525241+0000","last_peered":"2024-04-23T18:26:29.525241+0000","last_clean":"2024-04-23T18:26:29.525241+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:26:29.525241+0000","last_undegraded":"2024-04-23T18:26:29.525241+0000","last_fullsized":"2024-04-23T18:26:29.525241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683743+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:26:29.683743+0000","last_peered":"2024-04-23T18:26:29.683743+0000","last_clean":"2024-04-23T18:26:29.683743+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:26:29.683743+0000","last_undegraded":"2024-04-23T18:26:29.683743+0000","last_fullsized":"2024-04-23T18:26:29.683743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469827+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:26:29.469827+0000","last_peered":"2024-04-23T18:26:29.469827+0000","last_clean":"2024-04-23T18:26:29.469827+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:26:29.469827+0000","last_undegraded":"2024-04-23T18:26:29.469827+0000","last_fullsized":"2024-04-23T18:26:29.469827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523623+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:26:29.523623+0000","last_peered":"2024-04-23T18:26:29.523623+0000","last_clean":"2024-04-23T18:26:29.523623+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:26:29.523623+0000","last_undegraded":"2024-04-23T18:26:29.523623+0000","last_fullsized":"2024-04-23T18:26:29.523623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525576+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:26:29.525576+0000","last_peered":"2024-04-23T18:26:29.525576+0000","last_clean":"2024-04-23T18:26:29.525576+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:26:29.525576+0000","last_undegraded":"2024-04-23T18:26:29.525576+0000","last_fullsized":"2024-04-23T18:26:29.525576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523055+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:26:29.523055+0000","last_peered":"2024-04-23T18:26:29.523055+0000","last_clean":"2024-04-23T18:26:29.523055+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:26:29.523055+0000","last_undegraded":"2024-04-23T18:26:29.523055+0000","last_fullsized":"2024-04-23T18:26:29.523055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683439+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:26:29.683439+0000","last_peered":"2024-04-23T18:26:29.683439+0000","last_clean":"2024-04-23T18:26:29.683439+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:26:29.683439+0000","last_undegraded":"2024-04-23T18:26:29.683439+0000","last_fullsized":"2024-04-23T18:26:29.683439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683324+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:26:29.683324+0000","last_peered":"2024-04-23T18:26:29.683324+0000","last_clean":"2024-04-23T18:26:29.683324+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:26:29.683324+0000","last_undegraded":"2024-04-23T18:26:29.683324+0000","last_fullsized":"2024-04-23T18:26:29.683324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683321+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:26:29.683321+0000","last_peered":"2024-04-23T18:26:29.683321+0000","last_clean":"2024-04-23T18:26:29.683321+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:26:29.683321+0000","last_undegraded":"2024-04-23T18:26:29.683321+0000","last_fullsized":"2024-04-23T18:26:29.683321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468714+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:26:29.468714+0000","last_peered":"2024-04-23T18:26:29.468714+0000","last_clean":"2024-04-23T18:26:29.468714+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:26:29.468714+0000","last_undegraded":"2024-04-23T18:26:29.468714+0000","last_fullsized":"2024-04-23T18:26:29.468714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467817+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:26:29.467817+0000","last_peered":"2024-04-23T18:26:29.467817+0000","last_clean":"2024-04-23T18:26:29.467817+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:26:29.467817+0000","last_undegraded":"2024-04-23T18:26:29.467817+0000","last_fullsized":"2024-04-23T18:26:29.467817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543091+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:26:29.543091+0000","last_peered":"2024-04-23T18:26:29.543091+0000","last_clean":"2024-04-23T18:26:29.543091+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:26:29.543091+0000","last_undegraded":"2024-04-23T18:26:29.543091+0000","last_fullsized":"2024-04-23T18:26:29.543091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":153,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717616+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:26:29.717616+0000","last_peered":"2024-04-23T18:26:29.717616+0000","last_clean":"2024-04-23T18:26:29.717616+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:26:29.717616+0000","last_undegraded":"2024-04-23T18:26:29.717616+0000","last_fullsized":"2024-04-23T18:26:29.717616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525570+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:26:29.525570+0000","last_peered":"2024-04-23T18:26:29.525570+0000","last_clean":"2024-04-23T18:26:29.525570+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:26:29.525570+0000","last_undegraded":"2024-04-23T18:26:29.525570+0000","last_fullsized":"2024-04-23T18:26:29.525570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525747+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:26:29.525747+0000","last_peered":"2024-04-23T18:26:29.525747+0000","last_clean":"2024-04-23T18:26:29.525747+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:26:29.525747+0000","last_undegraded":"2024-04-23T18:26:29.525747+0000","last_fullsized":"2024-04-23T18:26:29.525747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683868+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:26:29.683868+0000","last_peered":"2024-04-23T18:26:29.683868+0000","last_clean":"2024-04-23T18:26:29.683868+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:26:29.683868+0000","last_undegraded":"2024-04-23T18:26:29.683868+0000","last_fullsized":"2024-04-23T18:26:29.683868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468673+0000","last_change":"2024-04-23T18:25:25.931554+0000","last_active":"2024-04-23T18:26:29.468673+0000","last_peered":"2024-04-23T18:26:29.468673+0000","last_clean":"2024-04-23T18:26:29.468673+0000","last_became_active":"2024-04-23T18:25:25.919193+0000","last_became_peered":"2024-04-23T18:25:25.919193+0000","last_unstale":"2024-04-23T18:26:29.468673+0000","last_undegraded":"2024-04-23T18:26:29.468673+0000","last_fullsized":"2024-04-23T18:26:29.468673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:34.700255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683406+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:26:29.683406+0000","last_peered":"2024-04-23T18:26:29.683406+0000","last_clean":"2024-04-23T18:26:29.683406+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:26:29.683406+0000","last_undegraded":"2024-04-23T18:26:29.683406+0000","last_fullsized":"2024-04-23T18:26:29.683406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563453+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:26:29.563453+0000","last_peered":"2024-04-23T18:26:29.563453+0000","last_clean":"2024-04-23T18:26:29.563453+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:26:29.563453+0000","last_undegraded":"2024-04-23T18:26:29.563453+0000","last_fullsized":"2024-04-23T18:26:29.563453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524606+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:26:29.524606+0000","last_peered":"2024-04-23T18:26:29.524606+0000","last_clean":"2024-04-23T18:26:29.524606+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:26:29.524606+0000","last_undegraded":"2024-04-23T18:26:29.524606+0000","last_fullsized":"2024-04-23T18:26:29.524606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":150,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524778+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:26:29.524778+0000","last_peered":"2024-04-23T18:26:29.524778+0000","last_clean":"2024-04-23T18:26:29.524778+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:26:29.524778+0000","last_undegraded":"2024-04-23T18:26:29.524778+0000","last_fullsized":"2024-04-23T18:26:29.524778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563167+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:26:29.563167+0000","last_peered":"2024-04-23T18:26:29.563167+0000","last_clean":"2024-04-23T18:26:29.563167+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:26:29.563167+0000","last_undegraded":"2024-04-23T18:26:29.563167+0000","last_fullsized":"2024-04-23T18:26:29.563167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469343+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:26:29.469343+0000","last_peered":"2024-04-23T18:26:29.469343+0000","last_clean":"2024-04-23T18:26:29.469343+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:26:29.469343+0000","last_undegraded":"2024-04-23T18:26:29.469343+0000","last_fullsized":"2024-04-23T18:26:29.469343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524519+0000","last_change":"2024-04-23T18:25:25.919173+0000","last_active":"2024-04-23T18:26:29.524519+0000","last_peered":"2024-04-23T18:26:29.524519+0000","last_clean":"2024-04-23T18:26:29.524519+0000","last_became_active":"2024-04-23T18:25:25.918830+0000","last_became_peered":"2024-04-23T18:25:25.918830+0000","last_unstale":"2024-04-23T18:26:29.524519+0000","last_undegraded":"2024-04-23T18:26:29.524519+0000","last_fullsized":"2024-04-23T18:26:29.524519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:51.415326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523056+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:26:29.523056+0000","last_peered":"2024-04-23T18:26:29.523056+0000","last_clean":"2024-04-23T18:26:29.523056+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:26:29.523056+0000","last_undegraded":"2024-04-23T18:26:29.523056+0000","last_fullsized":"2024-04-23T18:26:29.523056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525205+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:26:29.525205+0000","last_peered":"2024-04-23T18:26:29.525205+0000","last_clean":"2024-04-23T18:26:29.525205+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:26:29.525205+0000","last_undegraded":"2024-04-23T18:26:29.525205+0000","last_fullsized":"2024-04-23T18:26:29.525205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524341+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:26:29.524341+0000","last_peered":"2024-04-23T18:26:29.524341+0000","last_clean":"2024-04-23T18:26:29.524341+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:26:29.524341+0000","last_undegraded":"2024-04-23T18:26:29.524341+0000","last_fullsized":"2024-04-23T18:26:29.524341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467566+0000","last_change":"2024-04-23T18:25:26.153727+0000","last_active":"2024-04-23T18:26:29.467566+0000","last_peered":"2024-04-23T18:26:29.467566+0000","last_clean":"2024-04-23T18:26:29.467566+0000","last_became_active":"2024-04-23T18:25:26.153420+0000","last_became_peered":"2024-04-23T18:25:26.153420+0000","last_unstale":"2024-04-23T18:26:29.467566+0000","last_undegraded":"2024-04-23T18:26:29.467566+0000","last_fullsized":"2024-04-23T18:26:29.467566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:20.614833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.541194+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:26:29.541194+0000","last_peered":"2024-04-23T18:26:29.541194+0000","last_clean":"2024-04-23T18:26:29.541194+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:26:29.541194+0000","last_undegraded":"2024-04-23T18:26:29.541194+0000","last_fullsized":"2024-04-23T18:26:29.541194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683662+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:26:29.683662+0000","last_peered":"2024-04-23T18:26:29.683662+0000","last_clean":"2024-04-23T18:26:29.683662+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:26:29.683662+0000","last_undegraded":"2024-04-23T18:26:29.683662+0000","last_fullsized":"2024-04-23T18:26:29.683662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523432+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:26:29.523432+0000","last_peered":"2024-04-23T18:26:29.523432+0000","last_clean":"2024-04-23T18:26:29.523432+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:26:29.523432+0000","last_undegraded":"2024-04-23T18:26:29.523432+0000","last_fullsized":"2024-04-23T18:26:29.523432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683784+0000","last_change":"2024-04-23T18:25:25.916633+0000","last_active":"2024-04-23T18:26:29.683784+0000","last_peered":"2024-04-23T18:26:29.683784+0000","last_clean":"2024-04-23T18:26:29.683784+0000","last_became_active":"2024-04-23T18:25:25.916224+0000","last_became_peered":"2024-04-23T18:25:25.916224+0000","last_unstale":"2024-04-23T18:26:29.683784+0000","last_undegraded":"2024-04-23T18:26:29.683784+0000","last_fullsized":"2024-04-23T18:26:29.683784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:10.172049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524631+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:26:29.524631+0000","last_peered":"2024-04-23T18:26:29.524631+0000","last_clean":"2024-04-23T18:26:29.524631+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:26:29.524631+0000","last_undegraded":"2024-04-23T18:26:29.524631+0000","last_fullsized":"2024-04-23T18:26:29.524631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683551+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:26:29.683551+0000","last_peered":"2024-04-23T18:26:29.683551+0000","last_clean":"2024-04-23T18:26:29.683551+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:26:29.683551+0000","last_undegraded":"2024-04-23T18:26:29.683551+0000","last_fullsized":"2024-04-23T18:26:29.683551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'40","reported_seq":302,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.715191+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:26:29.715191+0000","last_peered":"2024-04-23T18:26:29.715191+0000","last_clean":"2024-04-23T18:26:29.715191+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:26:29.715191+0000","last_undegraded":"2024-04-23T18:26:29.715191+0000","last_fullsized":"2024-04-23T18:26:29.715191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":185,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563324+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:26:29.563324+0000","last_peered":"2024-04-23T18:26:29.563324+0000","last_clean":"2024-04-23T18:26:29.563324+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:26:29.563324+0000","last_undegraded":"2024-04-23T18:26:29.563324+0000","last_fullsized":"2024-04-23T18:26:29.563324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542938+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:26:29.542938+0000","last_peered":"2024-04-23T18:26:29.542938+0000","last_clean":"2024-04-23T18:26:29.542938+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:26:29.542938+0000","last_undegraded":"2024-04-23T18:26:29.542938+0000","last_fullsized":"2024-04-23T18:26:29.542938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523077+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:26:29.523077+0000","last_peered":"2024-04-23T18:26:29.523077+0000","last_clean":"2024-04-23T18:26:29.523077+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:26:29.523077+0000","last_undegraded":"2024-04-23T18:26:29.523077+0000","last_fullsized":"2024-04-23T18:26:29.523077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524481+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:26:29.524481+0000","last_peered":"2024-04-23T18:26:29.524481+0000","last_clean":"2024-04-23T18:26:29.524481+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:26:29.524481+0000","last_undegraded":"2024-04-23T18:26:29.524481+0000","last_fullsized":"2024-04-23T18:26:29.524481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523707+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:26:29.523707+0000","last_peered":"2024-04-23T18:26:29.523707+0000","last_clean":"2024-04-23T18:26:29.523707+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:26:29.523707+0000","last_undegraded":"2024-04-23T18:26:29.523707+0000","last_fullsized":"2024-04-23T18:26:29.523707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525703+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:26:29.525703+0000","last_peered":"2024-04-23T18:26:29.525703+0000","last_clean":"2024-04-23T18:26:29.525703+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:26:29.525703+0000","last_undegraded":"2024-04-23T18:26:29.525703+0000","last_fullsized":"2024-04-23T18:26:29.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523212+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:26:29.523212+0000","last_peered":"2024-04-23T18:26:29.523212+0000","last_clean":"2024-04-23T18:26:29.523212+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:26:29.523212+0000","last_undegraded":"2024-04-23T18:26:29.523212+0000","last_fullsized":"2024-04-23T18:26:29.523212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563071+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:26:29.563071+0000","last_peered":"2024-04-23T18:26:29.563071+0000","last_clean":"2024-04-23T18:26:29.563071+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:26:29.563071+0000","last_undegraded":"2024-04-23T18:26:29.563071+0000","last_fullsized":"2024-04-23T18:26:29.563071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524609+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:26:29.524609+0000","last_peered":"2024-04-23T18:26:29.524609+0000","last_clean":"2024-04-23T18:26:29.524609+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:26:29.524609+0000","last_undegraded":"2024-04-23T18:26:29.524609+0000","last_fullsized":"2024-04-23T18:26:29.524609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563501+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:26:29.563501+0000","last_peered":"2024-04-23T18:26:29.563501+0000","last_clean":"2024-04-23T18:26:29.563501+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:26:29.563501+0000","last_undegraded":"2024-04-23T18:26:29.563501+0000","last_fullsized":"2024-04-23T18:26:29.563501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468601+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:26:29.468601+0000","last_peered":"2024-04-23T18:26:29.468601+0000","last_clean":"2024-04-23T18:26:29.468601+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:26:29.468601+0000","last_undegraded":"2024-04-23T18:26:29.468601+0000","last_fullsized":"2024-04-23T18:26:29.468601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467652+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:26:29.467652+0000","last_peered":"2024-04-23T18:26:29.467652+0000","last_clean":"2024-04-23T18:26:29.467652+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:26:29.467652+0000","last_undegraded":"2024-04-23T18:26:29.467652+0000","last_fullsized":"2024-04-23T18:26:29.467652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469742+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:26:29.469742+0000","last_peered":"2024-04-23T18:26:29.469742+0000","last_clean":"2024-04-23T18:26:29.469742+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:26:29.469742+0000","last_undegraded":"2024-04-23T18:26:29.469742+0000","last_fullsized":"2024-04-23T18:26:29.469742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717743+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:26:29.717743+0000","last_peered":"2024-04-23T18:26:29.717743+0000","last_clean":"2024-04-23T18:26:29.717743+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:26:29.717743+0000","last_undegraded":"2024-04-23T18:26:29.717743+0000","last_fullsized":"2024-04-23T18:26:29.717743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469572+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:26:29.469572+0000","last_peered":"2024-04-23T18:26:29.469572+0000","last_clean":"2024-04-23T18:26:29.469572+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:26:29.469572+0000","last_undegraded":"2024-04-23T18:26:29.469572+0000","last_fullsized":"2024-04-23T18:26:29.469572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524885+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:26:29.524885+0000","last_peered":"2024-04-23T18:26:29.524885+0000","last_clean":"2024-04-23T18:26:29.524885+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:26:29.524885+0000","last_undegraded":"2024-04-23T18:26:29.524885+0000","last_fullsized":"2024-04-23T18:26:29.524885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:33.529786+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:26:33.529786+0000","last_peered":"2024-04-23T18:26:33.529786+0000","last_clean":"2024-04-23T18:26:33.529786+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:26:33.529786+0000","last_undegraded":"2024-04-23T18:26:33.529786+0000","last_fullsized":"2024-04-23T18:26:33.529786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467506+0000","last_change":"2024-04-23T18:25:26.091646+0000","last_active":"2024-04-23T18:26:29.467506+0000","last_peered":"2024-04-23T18:26:29.467506+0000","last_clean":"2024-04-23T18:26:29.467506+0000","last_became_active":"2024-04-23T18:25:26.091303+0000","last_became_peered":"2024-04-23T18:25:26.091303+0000","last_unstale":"2024-04-23T18:26:29.467506+0000","last_undegraded":"2024-04-23T18:26:29.467506+0000","last_fullsized":"2024-04-23T18:26:29.467506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:27.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525409+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:26:29.525409+0000","last_peered":"2024-04-23T18:26:29.525409+0000","last_clean":"2024-04-23T18:26:29.525409+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:26:29.525409+0000","last_undegraded":"2024-04-23T18:26:29.525409+0000","last_fullsized":"2024-04-23T18:26:29.525409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523296+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:26:29.523296+0000","last_peered":"2024-04-23T18:26:29.523296+0000","last_clean":"2024-04-23T18:26:29.523296+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:26:29.523296+0000","last_undegraded":"2024-04-23T18:26:29.523296+0000","last_fullsized":"2024-04-23T18:26:29.523296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.522924+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:26:29.522924+0000","last_peered":"2024-04-23T18:26:29.522924+0000","last_clean":"2024-04-23T18:26:29.522924+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:26:29.522924+0000","last_undegraded":"2024-04-23T18:26:29.522924+0000","last_fullsized":"2024-04-23T18:26:29.522924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523487+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:26:29.523487+0000","last_peered":"2024-04-23T18:26:29.523487+0000","last_clean":"2024-04-23T18:26:29.523487+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:26:29.523487+0000","last_undegraded":"2024-04-23T18:26:29.523487+0000","last_fullsized":"2024-04-23T18:26:29.523487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563362+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:26:29.563362+0000","last_peered":"2024-04-23T18:26:29.563362+0000","last_clean":"2024-04-23T18:26:29.563362+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:26:29.563362+0000","last_undegraded":"2024-04-23T18:26:29.563362+0000","last_fullsized":"2024-04-23T18:26:29.563362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523321+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:26:29.523321+0000","last_peered":"2024-04-23T18:26:29.523321+0000","last_clean":"2024-04-23T18:26:29.523321+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:26:29.523321+0000","last_undegraded":"2024-04-23T18:26:29.523321+0000","last_fullsized":"2024-04-23T18:26:29.523321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.670822+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:26:29.670822+0000","last_peered":"2024-04-23T18:26:29.670822+0000","last_clean":"2024-04-23T18:26:29.670822+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:26:29.670822+0000","last_undegraded":"2024-04-23T18:26:29.670822+0000","last_fullsized":"2024-04-23T18:26:29.670822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525685+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:26:29.525685+0000","last_peered":"2024-04-23T18:26:29.525685+0000","last_clean":"2024-04-23T18:26:29.525685+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:26:29.525685+0000","last_undegraded":"2024-04-23T18:26:29.525685+0000","last_fullsized":"2024-04-23T18:26:29.525685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525536+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:26:29.525536+0000","last_peered":"2024-04-23T18:26:29.525536+0000","last_clean":"2024-04-23T18:26:29.525536+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:26:29.525536+0000","last_undegraded":"2024-04-23T18:26:29.525536+0000","last_fullsized":"2024-04-23T18:26:29.525536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683712+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:26:29.683712+0000","last_peered":"2024-04-23T18:26:29.683712+0000","last_clean":"2024-04-23T18:26:29.683712+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:26:29.683712+0000","last_undegraded":"2024-04-23T18:26:29.683712+0000","last_fullsized":"2024-04-23T18:26:29.683712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683415+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:26:29.683415+0000","last_peered":"2024-04-23T18:26:29.683415+0000","last_clean":"2024-04-23T18:26:29.683415+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:26:29.683415+0000","last_undegraded":"2024-04-23T18:26:29.683415+0000","last_fullsized":"2024-04-23T18:26:29.683415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563093+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:26:29.563093+0000","last_peered":"2024-04-23T18:26:29.563093+0000","last_clean":"2024-04-23T18:26:29.563093+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:26:29.563093+0000","last_undegraded":"2024-04-23T18:26:29.563093+0000","last_fullsized":"2024-04-23T18:26:29.563093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525276+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:26:29.525276+0000","last_peered":"2024-04-23T18:26:29.525276+0000","last_clean":"2024-04-23T18:26:29.525276+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:26:29.525276+0000","last_undegraded":"2024-04-23T18:26:29.525276+0000","last_fullsized":"2024-04-23T18:26:29.525276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468416+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:26:29.468416+0000","last_peered":"2024-04-23T18:26:29.468416+0000","last_clean":"2024-04-23T18:26:29.468416+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:26:29.468416+0000","last_undegraded":"2024-04-23T18:26:29.468416+0000","last_fullsized":"2024-04-23T18:26:29.468416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468530+0000","last_change":"2024-04-23T18:25:25.935443+0000","last_active":"2024-04-23T18:26:29.468530+0000","last_peered":"2024-04-23T18:26:29.468530+0000","last_clean":"2024-04-23T18:26:29.468530+0000","last_became_active":"2024-04-23T18:25:25.919406+0000","last_became_peered":"2024-04-23T18:25:25.919406+0000","last_unstale":"2024-04-23T18:26:29.468530+0000","last_undegraded":"2024-04-23T18:26:29.468530+0000","last_fullsized":"2024-04-23T18:26:29.468530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:10:03.007617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525448+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:26:29.525448+0000","last_peered":"2024-04-23T18:26:29.525448+0000","last_clean":"2024-04-23T18:26:29.525448+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:26:29.525448+0000","last_undegraded":"2024-04-23T18:26:29.525448+0000","last_fullsized":"2024-04-23T18:26:29.525448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563184+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:26:29.563184+0000","last_peered":"2024-04-23T18:26:29.563184+0000","last_clean":"2024-04-23T18:26:29.563184+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:26:29.563184+0000","last_undegraded":"2024-04-23T18:26:29.563184+0000","last_fullsized":"2024-04-23T18:26:29.563184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523641+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:26:29.523641+0000","last_peered":"2024-04-23T18:26:29.523641+0000","last_clean":"2024-04-23T18:26:29.523641+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:26:29.523641+0000","last_undegraded":"2024-04-23T18:26:29.523641+0000","last_fullsized":"2024-04-23T18:26:29.523641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543338+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:26:29.543338+0000","last_peered":"2024-04-23T18:26:29.543338+0000","last_clean":"2024-04-23T18:26:29.543338+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:26:29.543338+0000","last_undegraded":"2024-04-23T18:26:29.543338+0000","last_fullsized":"2024-04-23T18:26:29.543338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523684+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:26:29.523684+0000","last_peered":"2024-04-23T18:26:29.523684+0000","last_clean":"2024-04-23T18:26:29.523684+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:26:29.523684+0000","last_undegraded":"2024-04-23T18:26:29.523684+0000","last_fullsized":"2024-04-23T18:26:29.523684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.668447+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:26:29.668447+0000","last_peered":"2024-04-23T18:26:29.668447+0000","last_clean":"2024-04-23T18:26:29.668447+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:26:29.668447+0000","last_undegraded":"2024-04-23T18:26:29.668447+0000","last_fullsized":"2024-04-23T18:26:29.668447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523512+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:26:29.523512+0000","last_peered":"2024-04-23T18:26:29.523512+0000","last_clean":"2024-04-23T18:26:29.523512+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:26:29.523512+0000","last_undegraded":"2024-04-23T18:26:29.523512+0000","last_fullsized":"2024-04-23T18:26:29.523512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523123+0000","last_change":"2024-04-23T18:25:26.106394+0000","last_active":"2024-04-23T18:26:29.523123+0000","last_peered":"2024-04-23T18:26:29.523123+0000","last_clean":"2024-04-23T18:26:29.523123+0000","last_became_active":"2024-04-23T18:25:26.102178+0000","last_became_peered":"2024-04-23T18:25:26.102178+0000","last_unstale":"2024-04-23T18:26:29.523123+0000","last_undegraded":"2024-04-23T18:26:29.523123+0000","last_fullsized":"2024-04-23T18:26:29.523123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:20.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523645+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:26:29.523645+0000","last_peered":"2024-04-23T18:26:29.523645+0000","last_clean":"2024-04-23T18:26:29.523645+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:26:29.523645+0000","last_undegraded":"2024-04-23T18:26:29.523645+0000","last_fullsized":"2024-04-23T18:26:29.523645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468549+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:26:29.468549+0000","last_peered":"2024-04-23T18:26:29.468549+0000","last_clean":"2024-04-23T18:26:29.468549+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:26:29.468549+0000","last_undegraded":"2024-04-23T18:26:29.468549+0000","last_fullsized":"2024-04-23T18:26:29.468549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542861+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:26:29.542861+0000","last_peered":"2024-04-23T18:26:29.542861+0000","last_clean":"2024-04-23T18:26:29.542861+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:26:29.542861+0000","last_undegraded":"2024-04-23T18:26:29.542861+0000","last_fullsized":"2024-04-23T18:26:29.542861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542855+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:26:29.542855+0000","last_peered":"2024-04-23T18:26:29.542855+0000","last_clean":"2024-04-23T18:26:29.542855+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:26:29.542855+0000","last_undegraded":"2024-04-23T18:26:29.542855+0000","last_fullsized":"2024-04-23T18:26:29.542855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683626+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:26:29.683626+0000","last_peered":"2024-04-23T18:26:29.683626+0000","last_clean":"2024-04-23T18:26:29.683626+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:26:29.683626+0000","last_undegraded":"2024-04-23T18:26:29.683626+0000","last_fullsized":"2024-04-23T18:26:29.683626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523299+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:26:29.523299+0000","last_peered":"2024-04-23T18:26:29.523299+0000","last_clean":"2024-04-23T18:26:29.523299+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:26:29.523299+0000","last_undegraded":"2024-04-23T18:26:29.523299+0000","last_fullsized":"2024-04-23T18:26:29.523299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468775+0000","last_change":"2024-04-23T18:25:25.915473+0000","last_active":"2024-04-23T18:26:29.468775+0000","last_peered":"2024-04-23T18:26:29.468775+0000","last_clean":"2024-04-23T18:26:29.468775+0000","last_became_active":"2024-04-23T18:25:25.914874+0000","last_became_peered":"2024-04-23T18:25:25.914874+0000","last_unstale":"2024-04-23T18:26:29.468775+0000","last_undegraded":"2024-04-23T18:26:29.468775+0000","last_fullsized":"2024-04-23T18:26:29.468775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:14.348994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543211+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:26:29.543211+0000","last_peered":"2024-04-23T18:26:29.543211+0000","last_clean":"2024-04-23T18:26:29.543211+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:26:29.543211+0000","last_undegraded":"2024-04-23T18:26:29.543211+0000","last_fullsized":"2024-04-23T18:26:29.543211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525495+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:26:29.525495+0000","last_peered":"2024-04-23T18:26:29.525495+0000","last_clean":"2024-04-23T18:26:29.525495+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:26:29.525495+0000","last_undegraded":"2024-04-23T18:26:29.525495+0000","last_fullsized":"2024-04-23T18:26:29.525495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524255+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:26:29.524255+0000","last_peered":"2024-04-23T18:26:29.524255+0000","last_clean":"2024-04-23T18:26:29.524255+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:26:29.524255+0000","last_undegraded":"2024-04-23T18:26:29.524255+0000","last_fullsized":"2024-04-23T18:26:29.524255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525294+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:26:29.525294+0000","last_peered":"2024-04-23T18:26:29.525294+0000","last_clean":"2024-04-23T18:26:29.525294+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:26:29.525294+0000","last_undegraded":"2024-04-23T18:26:29.525294+0000","last_fullsized":"2024-04-23T18:26:29.525294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683418+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:26:29.683418+0000","last_peered":"2024-04-23T18:26:29.683418+0000","last_clean":"2024-04-23T18:26:29.683418+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:26:29.683418+0000","last_undegraded":"2024-04-23T18:26:29.683418+0000","last_fullsized":"2024-04-23T18:26:29.683418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523149+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:26:29.523149+0000","last_peered":"2024-04-23T18:26:29.523149+0000","last_clean":"2024-04-23T18:26:29.523149+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:26:29.523149+0000","last_undegraded":"2024-04-23T18:26:29.523149+0000","last_fullsized":"2024-04-23T18:26:29.523149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468526+0000","last_change":"2024-04-23T18:25:25.917314+0000","last_active":"2024-04-23T18:26:29.468526+0000","last_peered":"2024-04-23T18:26:29.468526+0000","last_clean":"2024-04-23T18:26:29.468526+0000","last_became_active":"2024-04-23T18:25:25.916841+0000","last_became_peered":"2024-04-23T18:25:25.916841+0000","last_unstale":"2024-04-23T18:26:29.468526+0000","last_undegraded":"2024-04-23T18:26:29.468526+0000","last_fullsized":"2024-04-23T18:26:29.468526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:03.413973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107010,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600909,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1297,"num_read_kb":764,"num_write":376,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":1999750,"data_compressed":85095,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":466241,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1142784,"data_stored":1254964,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":465780,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":859374,"data_compressed":27877,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":1060830,"data_compressed":44401,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":466279,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":533536,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":467227,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":534987,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1186770,"data_compressed":43222,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466281,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1190794,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:35.624 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-23T18:26:35.624 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-23T18:26:35.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 -- 172.21.15.33:0/1298532475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340101110 msgr2=0x7fa3401014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 --2- 172.21.15.33:0/1298532475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340101110 0x7fa3401014f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa338009900 tx=0x7fa33802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 -- 172.21.15.33:0/1298532475 shutdown_connections 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 --2- 172.21.15.33:0/1298532475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa340106190 0x7fa34010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 --2- 172.21.15.33:0/1298532475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101ac0 0x7fa340105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 --2- 172.21.15.33:0/1298532475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340101110 0x7fa3401014f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 -- 172.21.15.33:0/1298532475 >> 172.21.15.33:0/1298532475 conn(0x7fa340077fb0 msgr2=0x7fa3400783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 -- 172.21.15.33:0/1298532475 shutdown_connections 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.769+0000 7fa347ac2640 1 -- 172.21.15.33:0/1298532475 wait complete. 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 Processor -- start 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 -- start start 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101110 0x7fa340199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa34019a140 0x7fa3401a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa340111150 con 0x7fa340106190 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa340110fd0 con 0x7fa340101110 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa347ac2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3401112d0 con 0x7fa34019a140 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa34505d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa34505d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44284/0 (socket says 172.21.15.33:44284) 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa34505d640 1 -- 172.21.15.33:0/3525906778 learned_addr learned my addr 172.21.15.33:0/3525906778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.770+0000 7fa34605f640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa34019a140 0x7fa3401a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34585e640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101110 0x7fa340199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 -- 172.21.15.33:0/3525906778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa34019a140 msgr2=0x7fa3401a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa34019a140 0x7fa3401a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 -- 172.21.15.33:0/3525906778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101110 msgr2=0x7fa340199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101110 0x7fa340199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 -- 172.21.15.33:0/3525906778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3401a4ec0 con 0x7fa340106190 2024-04-23T18:26:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa34505d640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d770 tx=0x7fa33400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334017070 con 0x7fa340106190 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3401a51b0 con 0x7fa340106190 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa334009c30 con 0x7fa340106190 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33400f620 con 0x7fa340106190 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.771+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3401a5750 con 0x7fa340106190 2024-04-23T18:26:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.772+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3400691c0 con 0x7fa340106190 2024-04-23T18:26:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.773+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa33401a070 con 0x7fa340106190 2024-04-23T18:26:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.774+0000 7fa33effd640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 0x7fa3280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.774+0000 7fa34585e640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 0x7fa3280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.774+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fa3340d0610 con 0x7fa340106190 2024-04-23T18:26:35.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.774+0000 7fa34585e640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 0x7fa3280a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa338031400 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:35.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.778+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa334009de0 con 0x7fa340106190 2024-04-23T18:26:35.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:35 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3255653513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:35 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2800176512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:35 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3255653513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:35 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2800176512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.962+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa340109430 con 0x7fa3280a3e40 2024-04-23T18:26:35.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.969+0000 7fa33effd640 1 -- 172.21.15.33:0/3525906778 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258064 (secure 0 0 0) 0x7fa338004310 con 0x7fa3280a3e40 2024-04-23T18:26:35.964 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:35.966 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-23T18:26:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 msgr2=0x7fa3280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 0x7fa3280a6300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa338031400 comp rx=0 tx=0).stop 2024-04-23T18:26:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 msgr2=0x7fa340199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:35.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d770 tx=0x7fa33400dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 shutdown_connections 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3280a3e40 0x7fa3280a6300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa34019a140 0x7fa3401a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa340106190 0x7fa340199bb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 --2- 172.21.15.33:0/3525906778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa340101110 0x7fa340199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 >> 172.21.15.33:0/3525906778 conn(0x7fa340077fb0 msgr2=0x7fa3400ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 shutdown_connections 2024-04-23T18:26:35.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:35.972+0000 7fa347ac2640 1 -- 172.21.15.33:0/3525906778 wait complete. 2024-04-23T18:26:35.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:35 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3255653513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:35 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2800176512' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:35.976 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-23T18:26:35.480747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1191511,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1433,"num_read_kb":883,"num_write":559,"num_write_kb":4410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1180736,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382012,"kb_used_data":12008,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124559812,"statfs":{"total":1151940427776,"available":1151549247488,"internally_reserved":0,"allocated":12296192,"data_stored":8953063,"data_compressed":277731,"data_compressed_allocated":368640,"data_compressed_original":3403776,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.985749"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468747+0000","last_change":"2024-04-23T18:24:26.884098+0000","last_active":"2024-04-23T18:26:29.468747+0000","last_peered":"2024-04-23T18:26:29.468747+0000","last_clean":"2024-04-23T18:26:29.468747+0000","last_became_active":"2024-04-23T18:24:26.883769+0000","last_became_peered":"2024-04-23T18:24:26.883769+0000","last_unstale":"2024-04-23T18:26:29.468747+0000","last_undegraded":"2024-04-23T18:26:29.468747+0000","last_fullsized":"2024-04-23T18:26:29.468747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:57:39.474197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683825+0000","last_change":"2024-04-23T18:24:26.900175+0000","last_active":"2024-04-23T18:26:29.683825+0000","last_peered":"2024-04-23T18:26:29.683825+0000","last_clean":"2024-04-23T18:26:29.683825+0000","last_became_active":"2024-04-23T18:24:26.899837+0000","last_became_peered":"2024-04-23T18:24:26.899837+0000","last_unstale":"2024-04-23T18:26:29.683825+0000","last_undegraded":"2024-04-23T18:26:29.683825+0000","last_fullsized":"2024-04-23T18:26:29.683825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:36:32.621258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683773+0000","last_change":"2024-04-23T18:24:26.896321+0000","last_active":"2024-04-23T18:26:29.683773+0000","last_peered":"2024-04-23T18:26:29.683773+0000","last_clean":"2024-04-23T18:26:29.683773+0000","last_became_active":"2024-04-23T18:24:26.895969+0000","last_became_peered":"2024-04-23T18:24:26.895969+0000","last_unstale":"2024-04-23T18:26:29.683773+0000","last_undegraded":"2024-04-23T18:26:29.683773+0000","last_fullsized":"2024-04-23T18:26:29.683773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:32:55.857906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525627+0000","last_change":"2024-04-23T18:24:26.891898+0000","last_active":"2024-04-23T18:26:29.525627+0000","last_peered":"2024-04-23T18:26:29.525627+0000","last_clean":"2024-04-23T18:26:29.525627+0000","last_became_active":"2024-04-23T18:24:26.891613+0000","last_became_peered":"2024-04-23T18:24:26.891613+0000","last_unstale":"2024-04-23T18:26:29.525627+0000","last_undegraded":"2024-04-23T18:26:29.525627+0000","last_fullsized":"2024-04-23T18:26:29.525627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:25.693842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563249+0000","last_change":"2024-04-23T18:24:26.890890+0000","last_active":"2024-04-23T18:26:29.563249+0000","last_peered":"2024-04-23T18:26:29.563249+0000","last_clean":"2024-04-23T18:26:29.563249+0000","last_became_active":"2024-04-23T18:24:26.890089+0000","last_became_peered":"2024-04-23T18:24:26.890089+0000","last_unstale":"2024-04-23T18:26:29.563249+0000","last_undegraded":"2024-04-23T18:26:29.563249+0000","last_fullsized":"2024-04-23T18:26:29.563249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:04:46.346495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469383+0000","last_change":"2024-04-23T18:24:26.890969+0000","last_active":"2024-04-23T18:26:29.469383+0000","last_peered":"2024-04-23T18:26:29.469383+0000","last_clean":"2024-04-23T18:26:29.469383+0000","last_became_active":"2024-04-23T18:24:26.890642+0000","last_became_peered":"2024-04-23T18:24:26.890642+0000","last_unstale":"2024-04-23T18:26:29.469383+0000","last_undegraded":"2024-04-23T18:26:29.469383+0000","last_fullsized":"2024-04-23T18:26:29.469383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:38:02.625722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523261+0000","last_change":"2024-04-23T18:24:26.884515+0000","last_active":"2024-04-23T18:26:29.523261+0000","last_peered":"2024-04-23T18:26:29.523261+0000","last_clean":"2024-04-23T18:26:29.523261+0000","last_became_active":"2024-04-23T18:24:26.884148+0000","last_became_peered":"2024-04-23T18:24:26.884148+0000","last_unstale":"2024-04-23T18:26:29.523261+0000","last_undegraded":"2024-04-23T18:26:29.523261+0000","last_fullsized":"2024-04-23T18:26:29.523261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:37:21.877410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525241+0000","last_change":"2024-04-23T18:24:26.898122+0000","last_active":"2024-04-23T18:26:29.525241+0000","last_peered":"2024-04-23T18:26:29.525241+0000","last_clean":"2024-04-23T18:26:29.525241+0000","last_became_active":"2024-04-23T18:24:26.897582+0000","last_became_peered":"2024-04-23T18:24:26.897582+0000","last_unstale":"2024-04-23T18:26:29.525241+0000","last_undegraded":"2024-04-23T18:26:29.525241+0000","last_fullsized":"2024-04-23T18:26:29.525241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:01:38.930370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683743+0000","last_change":"2024-04-23T18:24:26.891378+0000","last_active":"2024-04-23T18:26:29.683743+0000","last_peered":"2024-04-23T18:26:29.683743+0000","last_clean":"2024-04-23T18:26:29.683743+0000","last_became_active":"2024-04-23T18:24:26.890480+0000","last_became_peered":"2024-04-23T18:24:26.890480+0000","last_unstale":"2024-04-23T18:26:29.683743+0000","last_undegraded":"2024-04-23T18:26:29.683743+0000","last_fullsized":"2024-04-23T18:26:29.683743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:11:12.526227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469827+0000","last_change":"2024-04-23T18:24:26.890114+0000","last_active":"2024-04-23T18:26:29.469827+0000","last_peered":"2024-04-23T18:26:29.469827+0000","last_clean":"2024-04-23T18:26:29.469827+0000","last_became_active":"2024-04-23T18:24:26.889794+0000","last_became_peered":"2024-04-23T18:24:26.889794+0000","last_unstale":"2024-04-23T18:26:29.469827+0000","last_undegraded":"2024-04-23T18:26:29.469827+0000","last_fullsized":"2024-04-23T18:26:29.469827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:21:40.713998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523623+0000","last_change":"2024-04-23T18:24:26.887125+0000","last_active":"2024-04-23T18:26:29.523623+0000","last_peered":"2024-04-23T18:26:29.523623+0000","last_clean":"2024-04-23T18:26:29.523623+0000","last_became_active":"2024-04-23T18:24:26.886833+0000","last_became_peered":"2024-04-23T18:24:26.886833+0000","last_unstale":"2024-04-23T18:26:29.523623+0000","last_undegraded":"2024-04-23T18:26:29.523623+0000","last_fullsized":"2024-04-23T18:26:29.523623+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:31:26.035066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525576+0000","last_change":"2024-04-23T18:24:26.895616+0000","last_active":"2024-04-23T18:26:29.525576+0000","last_peered":"2024-04-23T18:26:29.525576+0000","last_clean":"2024-04-23T18:26:29.525576+0000","last_became_active":"2024-04-23T18:24:26.895307+0000","last_became_peered":"2024-04-23T18:24:26.895307+0000","last_unstale":"2024-04-23T18:26:29.525576+0000","last_undegraded":"2024-04-23T18:26:29.525576+0000","last_fullsized":"2024-04-23T18:26:29.525576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:15:40.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523055+0000","last_change":"2024-04-23T18:24:26.876000+0000","last_active":"2024-04-23T18:26:29.523055+0000","last_peered":"2024-04-23T18:26:29.523055+0000","last_clean":"2024-04-23T18:26:29.523055+0000","last_became_active":"2024-04-23T18:24:26.875639+0000","last_became_peered":"2024-04-23T18:24:26.875639+0000","last_unstale":"2024-04-23T18:26:29.523055+0000","last_undegraded":"2024-04-23T18:26:29.523055+0000","last_fullsized":"2024-04-23T18:26:29.523055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:59:09.679186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683439+0000","last_change":"2024-04-23T18:24:26.887197+0000","last_active":"2024-04-23T18:26:29.683439+0000","last_peered":"2024-04-23T18:26:29.683439+0000","last_clean":"2024-04-23T18:26:29.683439+0000","last_became_active":"2024-04-23T18:24:26.886870+0000","last_became_peered":"2024-04-23T18:24:26.886870+0000","last_unstale":"2024-04-23T18:26:29.683439+0000","last_undegraded":"2024-04-23T18:26:29.683439+0000","last_fullsized":"2024-04-23T18:26:29.683439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:02:53.210838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683324+0000","last_change":"2024-04-23T18:24:26.893578+0000","last_active":"2024-04-23T18:26:29.683324+0000","last_peered":"2024-04-23T18:26:29.683324+0000","last_clean":"2024-04-23T18:26:29.683324+0000","last_became_active":"2024-04-23T18:24:26.893227+0000","last_became_peered":"2024-04-23T18:24:26.893227+0000","last_unstale":"2024-04-23T18:26:29.683324+0000","last_undegraded":"2024-04-23T18:26:29.683324+0000","last_fullsized":"2024-04-23T18:26:29.683324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:45:46.968085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683321+0000","last_change":"2024-04-23T18:24:26.888579+0000","last_active":"2024-04-23T18:26:29.683321+0000","last_peered":"2024-04-23T18:26:29.683321+0000","last_clean":"2024-04-23T18:26:29.683321+0000","last_became_active":"2024-04-23T18:24:26.888260+0000","last_became_peered":"2024-04-23T18:24:26.888260+0000","last_unstale":"2024-04-23T18:26:29.683321+0000","last_undegraded":"2024-04-23T18:26:29.683321+0000","last_fullsized":"2024-04-23T18:26:29.683321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:03:34.013710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468714+0000","last_change":"2024-04-23T18:24:26.889114+0000","last_active":"2024-04-23T18:26:29.468714+0000","last_peered":"2024-04-23T18:26:29.468714+0000","last_clean":"2024-04-23T18:26:29.468714+0000","last_became_active":"2024-04-23T18:24:26.888813+0000","last_became_peered":"2024-04-23T18:24:26.888813+0000","last_unstale":"2024-04-23T18:26:29.468714+0000","last_undegraded":"2024-04-23T18:26:29.468714+0000","last_fullsized":"2024-04-23T18:26:29.468714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:45:29.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467817+0000","last_change":"2024-04-23T18:24:26.885901+0000","last_active":"2024-04-23T18:26:29.467817+0000","last_peered":"2024-04-23T18:26:29.467817+0000","last_clean":"2024-04-23T18:26:29.467817+0000","last_became_active":"2024-04-23T18:24:26.885569+0000","last_became_peered":"2024-04-23T18:24:26.885569+0000","last_unstale":"2024-04-23T18:26:29.467817+0000","last_undegraded":"2024-04-23T18:26:29.467817+0000","last_fullsized":"2024-04-23T18:26:29.467817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:52.140445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543091+0000","last_change":"2024-04-23T18:24:26.890728+0000","last_active":"2024-04-23T18:26:29.543091+0000","last_peered":"2024-04-23T18:26:29.543091+0000","last_clean":"2024-04-23T18:26:29.543091+0000","last_became_active":"2024-04-23T18:24:26.889704+0000","last_became_peered":"2024-04-23T18:24:26.889704+0000","last_unstale":"2024-04-23T18:26:29.543091+0000","last_undegraded":"2024-04-23T18:26:29.543091+0000","last_fullsized":"2024-04-23T18:26:29.543091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:54:42.139983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'14","reported_seq":153,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717616+0000","last_change":"2024-04-23T18:24:26.289564+0000","last_active":"2024-04-23T18:26:29.717616+0000","last_peered":"2024-04-23T18:26:29.717616+0000","last_clean":"2024-04-23T18:26:29.717616+0000","last_became_active":"2024-04-23T18:24:26.289066+0000","last_became_peered":"2024-04-23T18:24:26.289066+0000","last_unstale":"2024-04-23T18:26:29.717616+0000","last_undegraded":"2024-04-23T18:26:29.717616+0000","last_fullsized":"2024-04-23T18:26:29.717616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:48:22.422248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525570+0000","last_change":"2024-04-23T18:24:26.897850+0000","last_active":"2024-04-23T18:26:29.525570+0000","last_peered":"2024-04-23T18:26:29.525570+0000","last_clean":"2024-04-23T18:26:29.525570+0000","last_became_active":"2024-04-23T18:24:26.897499+0000","last_became_peered":"2024-04-23T18:24:26.897499+0000","last_unstale":"2024-04-23T18:26:29.525570+0000","last_undegraded":"2024-04-23T18:26:29.525570+0000","last_fullsized":"2024-04-23T18:26:29.525570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:48:04.759942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525747+0000","last_change":"2024-04-23T18:24:26.302894+0000","last_active":"2024-04-23T18:26:29.525747+0000","last_peered":"2024-04-23T18:26:29.525747+0000","last_clean":"2024-04-23T18:26:29.525747+0000","last_became_active":"2024-04-23T18:24:26.302566+0000","last_became_peered":"2024-04-23T18:24:26.302566+0000","last_unstale":"2024-04-23T18:26:29.525747+0000","last_undegraded":"2024-04-23T18:26:29.525747+0000","last_fullsized":"2024-04-23T18:26:29.525747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:22:40.925071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683868+0000","last_change":"2024-04-23T18:24:26.890844+0000","last_active":"2024-04-23T18:26:29.683868+0000","last_peered":"2024-04-23T18:26:29.683868+0000","last_clean":"2024-04-23T18:26:29.683868+0000","last_became_active":"2024-04-23T18:24:26.890513+0000","last_became_peered":"2024-04-23T18:24:26.890513+0000","last_unstale":"2024-04-23T18:26:29.683868+0000","last_undegraded":"2024-04-23T18:26:29.683868+0000","last_fullsized":"2024-04-23T18:26:29.683868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:25:13.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'12","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468673+0000","last_change":"2024-04-23T18:25:25.931554+0000","last_active":"2024-04-23T18:26:29.468673+0000","last_peered":"2024-04-23T18:26:29.468673+0000","last_clean":"2024-04-23T18:26:29.468673+0000","last_became_active":"2024-04-23T18:25:25.919193+0000","last_became_peered":"2024-04-23T18:25:25.919193+0000","last_unstale":"2024-04-23T18:26:29.468673+0000","last_undegraded":"2024-04-23T18:26:29.468673+0000","last_fullsized":"2024-04-23T18:26:29.468673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:34.700255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683406+0000","last_change":"2024-04-23T18:24:26.885474+0000","last_active":"2024-04-23T18:26:29.683406+0000","last_peered":"2024-04-23T18:26:29.683406+0000","last_clean":"2024-04-23T18:26:29.683406+0000","last_became_active":"2024-04-23T18:24:26.885159+0000","last_became_peered":"2024-04-23T18:24:26.885159+0000","last_unstale":"2024-04-23T18:26:29.683406+0000","last_undegraded":"2024-04-23T18:26:29.683406+0000","last_fullsized":"2024-04-23T18:26:29.683406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:50:26.815842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563453+0000","last_change":"2024-04-23T18:24:26.288733+0000","last_active":"2024-04-23T18:26:29.563453+0000","last_peered":"2024-04-23T18:26:29.563453+0000","last_clean":"2024-04-23T18:26:29.563453+0000","last_became_active":"2024-04-23T18:24:26.288345+0000","last_became_peered":"2024-04-23T18:24:26.288345+0000","last_unstale":"2024-04-23T18:26:29.563453+0000","last_undegraded":"2024-04-23T18:26:29.563453+0000","last_fullsized":"2024-04-23T18:26:29.563453+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:15:28.678898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524606+0000","last_change":"2024-04-23T18:24:26.895271+0000","last_active":"2024-04-23T18:26:29.524606+0000","last_peered":"2024-04-23T18:26:29.524606+0000","last_clean":"2024-04-23T18:26:29.524606+0000","last_became_active":"2024-04-23T18:24:26.894922+0000","last_became_peered":"2024-04-23T18:24:26.894922+0000","last_unstale":"2024-04-23T18:26:29.524606+0000","last_undegraded":"2024-04-23T18:26:29.524606+0000","last_fullsized":"2024-04-23T18:26:29.524606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:14:00.080179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'9","reported_seq":150,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524778+0000","last_change":"2024-04-23T18:24:26.292464+0000","last_active":"2024-04-23T18:26:29.524778+0000","last_peered":"2024-04-23T18:26:29.524778+0000","last_clean":"2024-04-23T18:26:29.524778+0000","last_became_active":"2024-04-23T18:24:26.292099+0000","last_became_peered":"2024-04-23T18:24:26.292099+0000","last_unstale":"2024-04-23T18:26:29.524778+0000","last_undegraded":"2024-04-23T18:26:29.524778+0000","last_fullsized":"2024-04-23T18:26:29.524778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:40:52.512950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563167+0000","last_change":"2024-04-23T18:24:26.888839+0000","last_active":"2024-04-23T18:26:29.563167+0000","last_peered":"2024-04-23T18:26:29.563167+0000","last_clean":"2024-04-23T18:26:29.563167+0000","last_became_active":"2024-04-23T18:24:26.888458+0000","last_became_peered":"2024-04-23T18:24:26.888458+0000","last_unstale":"2024-04-23T18:26:29.563167+0000","last_undegraded":"2024-04-23T18:26:29.563167+0000","last_fullsized":"2024-04-23T18:26:29.563167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:39:49.929562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469343+0000","last_change":"2024-04-23T18:24:26.289888+0000","last_active":"2024-04-23T18:26:29.469343+0000","last_peered":"2024-04-23T18:26:29.469343+0000","last_clean":"2024-04-23T18:26:29.469343+0000","last_became_active":"2024-04-23T18:24:26.289566+0000","last_became_peered":"2024-04-23T18:24:26.289566+0000","last_unstale":"2024-04-23T18:26:29.469343+0000","last_undegraded":"2024-04-23T18:26:29.469343+0000","last_fullsized":"2024-04-23T18:26:29.469343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:08.940991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524519+0000","last_change":"2024-04-23T18:25:25.919173+0000","last_active":"2024-04-23T18:26:29.524519+0000","last_peered":"2024-04-23T18:26:29.524519+0000","last_clean":"2024-04-23T18:26:29.524519+0000","last_became_active":"2024-04-23T18:25:25.918830+0000","last_became_peered":"2024-04-23T18:25:25.918830+0000","last_unstale":"2024-04-23T18:26:29.524519+0000","last_undegraded":"2024-04-23T18:26:29.524519+0000","last_fullsized":"2024-04-23T18:26:29.524519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:01:51.415326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523056+0000","last_change":"2024-04-23T18:24:26.286878+0000","last_active":"2024-04-23T18:26:29.523056+0000","last_peered":"2024-04-23T18:26:29.523056+0000","last_clean":"2024-04-23T18:26:29.523056+0000","last_became_active":"2024-04-23T18:24:26.286540+0000","last_became_peered":"2024-04-23T18:24:26.286540+0000","last_unstale":"2024-04-23T18:26:29.523056+0000","last_undegraded":"2024-04-23T18:26:29.523056+0000","last_fullsized":"2024-04-23T18:26:29.523056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:59:51.527099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525205+0000","last_change":"2024-04-23T18:24:26.891173+0000","last_active":"2024-04-23T18:26:29.525205+0000","last_peered":"2024-04-23T18:26:29.525205+0000","last_clean":"2024-04-23T18:26:29.525205+0000","last_became_active":"2024-04-23T18:24:26.890811+0000","last_became_peered":"2024-04-23T18:24:26.890811+0000","last_unstale":"2024-04-23T18:26:29.525205+0000","last_undegraded":"2024-04-23T18:26:29.525205+0000","last_fullsized":"2024-04-23T18:26:29.525205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:42:51.910234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524341+0000","last_change":"2024-04-23T18:24:26.292983+0000","last_active":"2024-04-23T18:26:29.524341+0000","last_peered":"2024-04-23T18:26:29.524341+0000","last_clean":"2024-04-23T18:26:29.524341+0000","last_became_active":"2024-04-23T18:24:26.292647+0000","last_became_peered":"2024-04-23T18:24:26.292647+0000","last_unstale":"2024-04-23T18:26:29.524341+0000","last_undegraded":"2024-04-23T18:26:29.524341+0000","last_fullsized":"2024-04-23T18:26:29.524341+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:49:44.650358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467566+0000","last_change":"2024-04-23T18:25:26.153727+0000","last_active":"2024-04-23T18:26:29.467566+0000","last_peered":"2024-04-23T18:26:29.467566+0000","last_clean":"2024-04-23T18:26:29.467566+0000","last_became_active":"2024-04-23T18:25:26.153420+0000","last_became_peered":"2024-04-23T18:25:26.153420+0000","last_unstale":"2024-04-23T18:26:29.467566+0000","last_undegraded":"2024-04-23T18:26:29.467566+0000","last_fullsized":"2024-04-23T18:26:29.467566+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:14:20.614833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,10],"acting":[9,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'7","reported_seq":260,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.541194+0000","last_change":"2024-04-23T18:24:26.287368+0000","last_active":"2024-04-23T18:26:29.541194+0000","last_peered":"2024-04-23T18:26:29.541194+0000","last_clean":"2024-04-23T18:26:29.541194+0000","last_became_active":"2024-04-23T18:24:26.286550+0000","last_became_peered":"2024-04-23T18:24:26.286550+0000","last_unstale":"2024-04-23T18:26:29.541194+0000","last_undegraded":"2024-04-23T18:26:29.541194+0000","last_fullsized":"2024-04-23T18:26:29.541194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:11:45.145271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":232,"num_read_kb":232,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683662+0000","last_change":"2024-04-23T18:24:26.893956+0000","last_active":"2024-04-23T18:26:29.683662+0000","last_peered":"2024-04-23T18:26:29.683662+0000","last_clean":"2024-04-23T18:26:29.683662+0000","last_became_active":"2024-04-23T18:24:26.893655+0000","last_became_peered":"2024-04-23T18:24:26.893655+0000","last_unstale":"2024-04-23T18:26:29.683662+0000","last_undegraded":"2024-04-23T18:26:29.683662+0000","last_fullsized":"2024-04-23T18:26:29.683662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:53:33.639961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'4","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523432+0000","last_change":"2024-04-23T18:24:26.291993+0000","last_active":"2024-04-23T18:26:29.523432+0000","last_peered":"2024-04-23T18:26:29.523432+0000","last_clean":"2024-04-23T18:26:29.523432+0000","last_became_active":"2024-04-23T18:24:26.291608+0000","last_became_peered":"2024-04-23T18:24:26.291608+0000","last_unstale":"2024-04-23T18:26:29.523432+0000","last_undegraded":"2024-04-23T18:26:29.523432+0000","last_fullsized":"2024-04-23T18:26:29.523432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:51:28.807290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683784+0000","last_change":"2024-04-23T18:25:25.916633+0000","last_active":"2024-04-23T18:26:29.683784+0000","last_peered":"2024-04-23T18:26:29.683784+0000","last_clean":"2024-04-23T18:26:29.683784+0000","last_became_active":"2024-04-23T18:25:25.916224+0000","last_became_peered":"2024-04-23T18:25:25.916224+0000","last_unstale":"2024-04-23T18:26:29.683784+0000","last_undegraded":"2024-04-23T18:26:29.683784+0000","last_fullsized":"2024-04-23T18:26:29.683784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:34:10.172049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524631+0000","last_change":"2024-04-23T18:24:26.293011+0000","last_active":"2024-04-23T18:26:29.524631+0000","last_peered":"2024-04-23T18:26:29.524631+0000","last_clean":"2024-04-23T18:26:29.524631+0000","last_became_active":"2024-04-23T18:24:26.292708+0000","last_became_peered":"2024-04-23T18:24:26.292708+0000","last_unstale":"2024-04-23T18:26:29.524631+0000","last_undegraded":"2024-04-23T18:26:29.524631+0000","last_fullsized":"2024-04-23T18:26:29.524631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:03:51.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683551+0000","last_change":"2024-04-23T18:24:26.895208+0000","last_active":"2024-04-23T18:26:29.683551+0000","last_peered":"2024-04-23T18:26:29.683551+0000","last_clean":"2024-04-23T18:26:29.683551+0000","last_became_active":"2024-04-23T18:24:26.894860+0000","last_became_peered":"2024-04-23T18:24:26.894860+0000","last_unstale":"2024-04-23T18:26:29.683551+0000","last_undegraded":"2024-04-23T18:26:29.683551+0000","last_fullsized":"2024-04-23T18:26:29.683551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:19:23.411828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'40","reported_seq":302,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.715191+0000","last_change":"2024-04-23T18:24:26.287517+0000","last_active":"2024-04-23T18:26:29.715191+0000","last_peered":"2024-04-23T18:26:29.715191+0000","last_clean":"2024-04-23T18:26:29.715191+0000","last_became_active":"2024-04-23T18:24:26.287171+0000","last_became_peered":"2024-04-23T18:24:26.287171+0000","last_unstale":"2024-04-23T18:26:29.715191+0000","last_undegraded":"2024-04-23T18:26:29.715191+0000","last_fullsized":"2024-04-23T18:26:29.715191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:39:00.325002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66735,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":241,"num_read_kb":185,"num_write":42,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563324+0000","last_change":"2024-04-23T18:24:26.898130+0000","last_active":"2024-04-23T18:26:29.563324+0000","last_peered":"2024-04-23T18:26:29.563324+0000","last_clean":"2024-04-23T18:26:29.563324+0000","last_became_active":"2024-04-23T18:24:26.897807+0000","last_became_peered":"2024-04-23T18:24:26.897807+0000","last_unstale":"2024-04-23T18:26:29.563324+0000","last_undegraded":"2024-04-23T18:26:29.563324+0000","last_fullsized":"2024-04-23T18:26:29.563324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:35:55.430764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542938+0000","last_change":"2024-04-23T18:24:26.303103+0000","last_active":"2024-04-23T18:26:29.542938+0000","last_peered":"2024-04-23T18:26:29.542938+0000","last_clean":"2024-04-23T18:26:29.542938+0000","last_became_active":"2024-04-23T18:24:26.302775+0000","last_became_peered":"2024-04-23T18:24:26.302775+0000","last_unstale":"2024-04-23T18:26:29.542938+0000","last_undegraded":"2024-04-23T18:26:29.542938+0000","last_fullsized":"2024-04-23T18:26:29.542938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:48:28.593220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523077+0000","last_change":"2024-04-23T18:24:26.890938+0000","last_active":"2024-04-23T18:26:29.523077+0000","last_peered":"2024-04-23T18:26:29.523077+0000","last_clean":"2024-04-23T18:26:29.523077+0000","last_became_active":"2024-04-23T18:24:26.890595+0000","last_became_peered":"2024-04-23T18:24:26.890595+0000","last_unstale":"2024-04-23T18:26:29.523077+0000","last_undegraded":"2024-04-23T18:26:29.523077+0000","last_fullsized":"2024-04-23T18:26:29.523077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:27:51.649343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'5","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524481+0000","last_change":"2024-04-23T18:24:26.301247+0000","last_active":"2024-04-23T18:26:29.524481+0000","last_peered":"2024-04-23T18:26:29.524481+0000","last_clean":"2024-04-23T18:26:29.524481+0000","last_became_active":"2024-04-23T18:24:26.300974+0000","last_became_peered":"2024-04-23T18:24:26.300974+0000","last_unstale":"2024-04-23T18:26:29.524481+0000","last_undegraded":"2024-04-23T18:26:29.524481+0000","last_fullsized":"2024-04-23T18:26:29.524481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:31:55.539484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523707+0000","last_change":"2024-04-23T18:24:26.897284+0000","last_active":"2024-04-23T18:26:29.523707+0000","last_peered":"2024-04-23T18:26:29.523707+0000","last_clean":"2024-04-23T18:26:29.523707+0000","last_became_active":"2024-04-23T18:24:26.896985+0000","last_became_peered":"2024-04-23T18:24:26.896985+0000","last_unstale":"2024-04-23T18:26:29.523707+0000","last_undegraded":"2024-04-23T18:26:29.523707+0000","last_fullsized":"2024-04-23T18:26:29.523707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:42:52.488272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'8","reported_seq":148,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525703+0000","last_change":"2024-04-23T18:24:26.288569+0000","last_active":"2024-04-23T18:26:29.525703+0000","last_peered":"2024-04-23T18:26:29.525703+0000","last_clean":"2024-04-23T18:26:29.525703+0000","last_became_active":"2024-04-23T18:24:26.288138+0000","last_became_peered":"2024-04-23T18:24:26.288138+0000","last_unstale":"2024-04-23T18:26:29.525703+0000","last_undegraded":"2024-04-23T18:26:29.525703+0000","last_fullsized":"2024-04-23T18:26:29.525703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:23:44.038003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523212+0000","last_change":"2024-04-23T18:24:26.896282+0000","last_active":"2024-04-23T18:26:29.523212+0000","last_peered":"2024-04-23T18:26:29.523212+0000","last_clean":"2024-04-23T18:26:29.523212+0000","last_became_active":"2024-04-23T18:24:26.895965+0000","last_became_peered":"2024-04-23T18:24:26.895965+0000","last_unstale":"2024-04-23T18:26:29.523212+0000","last_undegraded":"2024-04-23T18:26:29.523212+0000","last_fullsized":"2024-04-23T18:26:29.523212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:47:20.847126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'17","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563071+0000","last_change":"2024-04-23T18:24:26.294272+0000","last_active":"2024-04-23T18:26:29.563071+0000","last_peered":"2024-04-23T18:26:29.563071+0000","last_clean":"2024-04-23T18:26:29.563071+0000","last_became_active":"2024-04-23T18:24:26.291551+0000","last_became_peered":"2024-04-23T18:24:26.291551+0000","last_unstale":"2024-04-23T18:26:29.563071+0000","last_undegraded":"2024-04-23T18:26:29.563071+0000","last_fullsized":"2024-04-23T18:26:29.563071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:59:51.282592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524609+0000","last_change":"2024-04-23T18:24:26.892064+0000","last_active":"2024-04-23T18:26:29.524609+0000","last_peered":"2024-04-23T18:26:29.524609+0000","last_clean":"2024-04-23T18:26:29.524609+0000","last_became_active":"2024-04-23T18:24:26.891714+0000","last_became_peered":"2024-04-23T18:24:26.891714+0000","last_unstale":"2024-04-23T18:26:29.524609+0000","last_undegraded":"2024-04-23T18:26:29.524609+0000","last_fullsized":"2024-04-23T18:26:29.524609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:39:19.013605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'3","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563501+0000","last_change":"2024-04-23T18:24:26.290534+0000","last_active":"2024-04-23T18:26:29.563501+0000","last_peered":"2024-04-23T18:26:29.563501+0000","last_clean":"2024-04-23T18:26:29.563501+0000","last_became_active":"2024-04-23T18:24:26.290127+0000","last_became_peered":"2024-04-23T18:24:26.290127+0000","last_unstale":"2024-04-23T18:26:29.563501+0000","last_undegraded":"2024-04-23T18:26:29.563501+0000","last_fullsized":"2024-04-23T18:26:29.563501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:18:42.648733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468601+0000","last_change":"2024-04-23T18:24:26.893685+0000","last_active":"2024-04-23T18:26:29.468601+0000","last_peered":"2024-04-23T18:26:29.468601+0000","last_clean":"2024-04-23T18:26:29.468601+0000","last_became_active":"2024-04-23T18:24:26.893374+0000","last_became_peered":"2024-04-23T18:24:26.893374+0000","last_unstale":"2024-04-23T18:26:29.468601+0000","last_undegraded":"2024-04-23T18:26:29.468601+0000","last_fullsized":"2024-04-23T18:26:29.468601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T21:45:33.953766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"85'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467652+0000","last_change":"2024-04-23T18:24:26.293487+0000","last_active":"2024-04-23T18:26:29.467652+0000","last_peered":"2024-04-23T18:26:29.467652+0000","last_clean":"2024-04-23T18:26:29.467652+0000","last_became_active":"2024-04-23T18:24:26.293124+0000","last_became_peered":"2024-04-23T18:24:26.293124+0000","last_unstale":"2024-04-23T18:26:29.467652+0000","last_undegraded":"2024-04-23T18:26:29.467652+0000","last_fullsized":"2024-04-23T18:26:29.467652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:28:17.101604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469742+0000","last_change":"2024-04-23T18:24:26.883402+0000","last_active":"2024-04-23T18:26:29.469742+0000","last_peered":"2024-04-23T18:26:29.469742+0000","last_clean":"2024-04-23T18:26:29.469742+0000","last_became_active":"2024-04-23T18:24:26.882957+0000","last_became_peered":"2024-04-23T18:24:26.882957+0000","last_unstale":"2024-04-23T18:26:29.469742+0000","last_undegraded":"2024-04-23T18:26:29.469742+0000","last_fullsized":"2024-04-23T18:26:29.469742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:13:31.749732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.717743+0000","last_change":"2024-04-23T18:24:26.286404+0000","last_active":"2024-04-23T18:26:29.717743+0000","last_peered":"2024-04-23T18:26:29.717743+0000","last_clean":"2024-04-23T18:26:29.717743+0000","last_became_active":"2024-04-23T18:24:26.286128+0000","last_became_peered":"2024-04-23T18:24:26.286128+0000","last_unstale":"2024-04-23T18:26:29.717743+0000","last_undegraded":"2024-04-23T18:26:29.717743+0000","last_fullsized":"2024-04-23T18:26:29.717743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:47:06.664678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.469572+0000","last_change":"2024-04-23T18:24:26.891973+0000","last_active":"2024-04-23T18:26:29.469572+0000","last_peered":"2024-04-23T18:26:29.469572+0000","last_clean":"2024-04-23T18:26:29.469572+0000","last_became_active":"2024-04-23T18:24:26.889350+0000","last_became_peered":"2024-04-23T18:24:26.889350+0000","last_unstale":"2024-04-23T18:26:29.469572+0000","last_undegraded":"2024-04-23T18:26:29.469572+0000","last_fullsized":"2024-04-23T18:26:29.469572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:08:48.677342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524885+0000","last_change":"2024-04-23T18:24:26.879960+0000","last_active":"2024-04-23T18:26:29.524885+0000","last_peered":"2024-04-23T18:26:29.524885+0000","last_clean":"2024-04-23T18:26:29.524885+0000","last_became_active":"2024-04-23T18:24:26.879649+0000","last_became_peered":"2024-04-23T18:24:26.879649+0000","last_unstale":"2024-04-23T18:26:29.524885+0000","last_undegraded":"2024-04-23T18:26:29.524885+0000","last_fullsized":"2024-04-23T18:26:29.524885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:15:53.698126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'7","reported_seq":151,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:33.529786+0000","last_change":"2024-04-23T18:24:26.296587+0000","last_active":"2024-04-23T18:26:33.529786+0000","last_peered":"2024-04-23T18:26:33.529786+0000","last_clean":"2024-04-23T18:26:33.529786+0000","last_became_active":"2024-04-23T18:24:26.296023+0000","last_became_peered":"2024-04-23T18:24:26.296023+0000","last_unstale":"2024-04-23T18:26:33.529786+0000","last_undegraded":"2024-04-23T18:26:33.529786+0000","last_fullsized":"2024-04-23T18:26:33.529786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.463304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":122,"num_read_kb":122,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.467506+0000","last_change":"2024-04-23T18:25:26.091646+0000","last_active":"2024-04-23T18:26:29.467506+0000","last_peered":"2024-04-23T18:26:29.467506+0000","last_clean":"2024-04-23T18:26:29.467506+0000","last_became_active":"2024-04-23T18:25:26.091303+0000","last_became_peered":"2024-04-23T18:25:26.091303+0000","last_unstale":"2024-04-23T18:26:29.467506+0000","last_undegraded":"2024-04-23T18:26:29.467506+0000","last_fullsized":"2024-04-23T18:26:29.467506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:50:27.478121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,1],"acting":[9,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.3","version":"85'6","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525409+0000","last_change":"2024-04-23T18:24:26.291297+0000","last_active":"2024-04-23T18:26:29.525409+0000","last_peered":"2024-04-23T18:26:29.525409+0000","last_clean":"2024-04-23T18:26:29.525409+0000","last_became_active":"2024-04-23T18:24:26.290911+0000","last_became_peered":"2024-04-23T18:24:26.290911+0000","last_unstale":"2024-04-23T18:26:29.525409+0000","last_undegraded":"2024-04-23T18:26:29.525409+0000","last_fullsized":"2024-04-23T18:26:29.525409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:05:52.614083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523296+0000","last_change":"2024-04-23T18:23:13.414057+0000","last_active":"2024-04-23T18:26:29.523296+0000","last_peered":"2024-04-23T18:26:29.523296+0000","last_clean":"2024-04-23T18:26:29.523296+0000","last_became_active":"2024-04-23T18:23:13.413499+0000","last_became_peered":"2024-04-23T18:23:13.413499+0000","last_unstale":"2024-04-23T18:26:29.523296+0000","last_undegraded":"2024-04-23T18:26:29.523296+0000","last_fullsized":"2024-04-23T18:26:29.523296+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:19:48.601105+0000","last_clean_scrub_stamp":"2024-04-23T18:19:48.601105+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T18:44:28.598691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.522924+0000","last_change":"2024-04-23T18:24:26.889548+0000","last_active":"2024-04-23T18:26:29.522924+0000","last_peered":"2024-04-23T18:26:29.522924+0000","last_clean":"2024-04-23T18:26:29.522924+0000","last_became_active":"2024-04-23T18:24:26.889229+0000","last_became_peered":"2024-04-23T18:24:26.889229+0000","last_unstale":"2024-04-23T18:26:29.522924+0000","last_undegraded":"2024-04-23T18:26:29.522924+0000","last_fullsized":"2024-04-23T18:26:29.522924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:09:46.608237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523487+0000","last_change":"2024-04-23T18:24:26.298914+0000","last_active":"2024-04-23T18:26:29.523487+0000","last_peered":"2024-04-23T18:26:29.523487+0000","last_clean":"2024-04-23T18:26:29.523487+0000","last_became_active":"2024-04-23T18:24:26.298628+0000","last_became_peered":"2024-04-23T18:24:26.298628+0000","last_unstale":"2024-04-23T18:26:29.523487+0000","last_undegraded":"2024-04-23T18:26:29.523487+0000","last_fullsized":"2024-04-23T18:26:29.523487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:06:05.348187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563362+0000","last_change":"2024-04-23T18:24:26.293649+0000","last_active":"2024-04-23T18:26:29.563362+0000","last_peered":"2024-04-23T18:26:29.563362+0000","last_clean":"2024-04-23T18:26:29.563362+0000","last_became_active":"2024-04-23T18:24:26.293349+0000","last_became_peered":"2024-04-23T18:24:26.293349+0000","last_unstale":"2024-04-23T18:26:29.563362+0000","last_undegraded":"2024-04-23T18:26:29.563362+0000","last_fullsized":"2024-04-23T18:26:29.563362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:29:31.793035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523321+0000","last_change":"2024-04-23T18:24:26.881451+0000","last_active":"2024-04-23T18:26:29.523321+0000","last_peered":"2024-04-23T18:26:29.523321+0000","last_clean":"2024-04-23T18:26:29.523321+0000","last_became_active":"2024-04-23T18:24:26.881152+0000","last_became_peered":"2024-04-23T18:24:26.881152+0000","last_unstale":"2024-04-23T18:26:29.523321+0000","last_undegraded":"2024-04-23T18:26:29.523321+0000","last_fullsized":"2024-04-23T18:26:29.523321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:15:57.976142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'48","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.670822+0000","last_change":"2024-04-23T18:24:26.295024+0000","last_active":"2024-04-23T18:26:29.670822+0000","last_peered":"2024-04-23T18:26:29.670822+0000","last_clean":"2024-04-23T18:26:29.670822+0000","last_became_active":"2024-04-23T18:24:26.294611+0000","last_became_peered":"2024-04-23T18:24:26.294611+0000","last_unstale":"2024-04-23T18:26:29.670822+0000","last_undegraded":"2024-04-23T18:26:29.670822+0000","last_fullsized":"2024-04-23T18:26:29.670822+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:28:52.643631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525685+0000","last_change":"2024-04-23T18:24:26.890573+0000","last_active":"2024-04-23T18:26:29.525685+0000","last_peered":"2024-04-23T18:26:29.525685+0000","last_clean":"2024-04-23T18:26:29.525685+0000","last_became_active":"2024-04-23T18:24:26.890248+0000","last_became_peered":"2024-04-23T18:24:26.890248+0000","last_unstale":"2024-04-23T18:26:29.525685+0000","last_undegraded":"2024-04-23T18:26:29.525685+0000","last_fullsized":"2024-04-23T18:26:29.525685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:18:13.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'7","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525536+0000","last_change":"2024-04-23T18:24:26.290335+0000","last_active":"2024-04-23T18:26:29.525536+0000","last_peered":"2024-04-23T18:26:29.525536+0000","last_clean":"2024-04-23T18:26:29.525536+0000","last_became_active":"2024-04-23T18:24:26.289998+0000","last_became_peered":"2024-04-23T18:24:26.289998+0000","last_unstale":"2024-04-23T18:26:29.525536+0000","last_undegraded":"2024-04-23T18:26:29.525536+0000","last_fullsized":"2024-04-23T18:26:29.525536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:00:57.116001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683712+0000","last_change":"2024-04-23T18:24:26.894832+0000","last_active":"2024-04-23T18:26:29.683712+0000","last_peered":"2024-04-23T18:26:29.683712+0000","last_clean":"2024-04-23T18:26:29.683712+0000","last_became_active":"2024-04-23T18:24:26.894345+0000","last_became_peered":"2024-04-23T18:24:26.894345+0000","last_unstale":"2024-04-23T18:26:29.683712+0000","last_undegraded":"2024-04-23T18:26:29.683712+0000","last_fullsized":"2024-04-23T18:26:29.683712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:51:45.046949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683415+0000","last_change":"2024-04-23T18:24:26.292544+0000","last_active":"2024-04-23T18:26:29.683415+0000","last_peered":"2024-04-23T18:26:29.683415+0000","last_clean":"2024-04-23T18:26:29.683415+0000","last_became_active":"2024-04-23T18:24:26.292107+0000","last_became_peered":"2024-04-23T18:24:26.292107+0000","last_unstale":"2024-04-23T18:26:29.683415+0000","last_undegraded":"2024-04-23T18:26:29.683415+0000","last_fullsized":"2024-04-23T18:26:29.683415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:22:24.670829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563093+0000","last_change":"2024-04-23T18:24:26.897443+0000","last_active":"2024-04-23T18:26:29.563093+0000","last_peered":"2024-04-23T18:26:29.563093+0000","last_clean":"2024-04-23T18:26:29.563093+0000","last_became_active":"2024-04-23T18:24:26.897073+0000","last_became_peered":"2024-04-23T18:24:26.897073+0000","last_unstale":"2024-04-23T18:26:29.563093+0000","last_undegraded":"2024-04-23T18:26:29.563093+0000","last_fullsized":"2024-04-23T18:26:29.563093+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:40:51.161767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'7","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525276+0000","last_change":"2024-04-23T18:24:26.295956+0000","last_active":"2024-04-23T18:26:29.525276+0000","last_peered":"2024-04-23T18:26:29.525276+0000","last_clean":"2024-04-23T18:26:29.525276+0000","last_became_active":"2024-04-23T18:24:26.295621+0000","last_became_peered":"2024-04-23T18:24:26.295621+0000","last_unstale":"2024-04-23T18:26:29.525276+0000","last_undegraded":"2024-04-23T18:26:29.525276+0000","last_fullsized":"2024-04-23T18:26:29.525276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:37:52.743651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468416+0000","last_change":"2024-04-23T18:24:26.883986+0000","last_active":"2024-04-23T18:26:29.468416+0000","last_peered":"2024-04-23T18:26:29.468416+0000","last_clean":"2024-04-23T18:26:29.468416+0000","last_became_active":"2024-04-23T18:24:26.883605+0000","last_became_peered":"2024-04-23T18:24:26.883605+0000","last_unstale":"2024-04-23T18:26:29.468416+0000","last_undegraded":"2024-04-23T18:26:29.468416+0000","last_fullsized":"2024-04-23T18:26:29.468416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:45.813551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'4","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468530+0000","last_change":"2024-04-23T18:25:25.935443+0000","last_active":"2024-04-23T18:26:29.468530+0000","last_peered":"2024-04-23T18:26:29.468530+0000","last_clean":"2024-04-23T18:26:29.468530+0000","last_became_active":"2024-04-23T18:25:25.919406+0000","last_became_peered":"2024-04-23T18:25:25.919406+0000","last_unstale":"2024-04-23T18:26:29.468530+0000","last_undegraded":"2024-04-23T18:26:29.468530+0000","last_fullsized":"2024-04-23T18:26:29.468530+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T01:10:03.007617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525448+0000","last_change":"2024-04-23T18:24:26.896895+0000","last_active":"2024-04-23T18:26:29.525448+0000","last_peered":"2024-04-23T18:26:29.525448+0000","last_clean":"2024-04-23T18:26:29.525448+0000","last_became_active":"2024-04-23T18:24:26.896580+0000","last_became_peered":"2024-04-23T18:24:26.896580+0000","last_unstale":"2024-04-23T18:26:29.525448+0000","last_undegraded":"2024-04-23T18:26:29.525448+0000","last_fullsized":"2024-04-23T18:26:29.525448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:05:53.013271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.563184+0000","last_change":"2024-04-23T18:24:26.291145+0000","last_active":"2024-04-23T18:26:29.563184+0000","last_peered":"2024-04-23T18:26:29.563184+0000","last_clean":"2024-04-23T18:26:29.563184+0000","last_became_active":"2024-04-23T18:24:26.290767+0000","last_became_peered":"2024-04-23T18:24:26.290767+0000","last_unstale":"2024-04-23T18:26:29.563184+0000","last_undegraded":"2024-04-23T18:26:29.563184+0000","last_fullsized":"2024-04-23T18:26:29.563184+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:20:15.551482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523641+0000","last_change":"2024-04-23T18:24:26.885364+0000","last_active":"2024-04-23T18:26:29.523641+0000","last_peered":"2024-04-23T18:26:29.523641+0000","last_clean":"2024-04-23T18:26:29.523641+0000","last_became_active":"2024-04-23T18:24:26.885002+0000","last_became_peered":"2024-04-23T18:24:26.885002+0000","last_unstale":"2024-04-23T18:26:29.523641+0000","last_undegraded":"2024-04-23T18:26:29.523641+0000","last_fullsized":"2024-04-23T18:26:29.523641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:26:29.532461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":145,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543338+0000","last_change":"2024-04-23T18:24:26.292634+0000","last_active":"2024-04-23T18:26:29.543338+0000","last_peered":"2024-04-23T18:26:29.543338+0000","last_clean":"2024-04-23T18:26:29.543338+0000","last_became_active":"2024-04-23T18:24:26.292311+0000","last_became_peered":"2024-04-23T18:24:26.292311+0000","last_unstale":"2024-04-23T18:26:29.543338+0000","last_undegraded":"2024-04-23T18:26:29.543338+0000","last_fullsized":"2024-04-23T18:26:29.543338+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T22:58:59.507558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523684+0000","last_change":"2024-04-23T18:24:26.889981+0000","last_active":"2024-04-23T18:26:29.523684+0000","last_peered":"2024-04-23T18:26:29.523684+0000","last_clean":"2024-04-23T18:26:29.523684+0000","last_became_active":"2024-04-23T18:24:26.889681+0000","last_became_peered":"2024-04-23T18:24:26.889681+0000","last_unstale":"2024-04-23T18:26:29.523684+0000","last_undegraded":"2024-04-23T18:26:29.523684+0000","last_fullsized":"2024-04-23T18:26:29.523684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:02:40.128999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'41","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.668447+0000","last_change":"2024-04-23T18:24:26.301246+0000","last_active":"2024-04-23T18:26:29.668447+0000","last_peered":"2024-04-23T18:26:29.668447+0000","last_clean":"2024-04-23T18:26:29.668447+0000","last_became_active":"2024-04-23T18:24:26.300865+0000","last_became_peered":"2024-04-23T18:24:26.300865+0000","last_unstale":"2024-04-23T18:26:29.668447+0000","last_undegraded":"2024-04-23T18:26:29.668447+0000","last_fullsized":"2024-04-23T18:26:29.668447+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T03:29:12.200536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523512+0000","last_change":"2024-04-23T18:24:26.891418+0000","last_active":"2024-04-23T18:26:29.523512+0000","last_peered":"2024-04-23T18:26:29.523512+0000","last_clean":"2024-04-23T18:26:29.523512+0000","last_became_active":"2024-04-23T18:24:26.891084+0000","last_became_peered":"2024-04-23T18:24:26.891084+0000","last_unstale":"2024-04-23T18:26:29.523512+0000","last_undegraded":"2024-04-23T18:26:29.523512+0000","last_fullsized":"2024-04-23T18:26:29.523512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T20:45:27.584852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'5","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523123+0000","last_change":"2024-04-23T18:25:26.106394+0000","last_active":"2024-04-23T18:26:29.523123+0000","last_peered":"2024-04-23T18:26:29.523123+0000","last_clean":"2024-04-23T18:26:29.523123+0000","last_became_active":"2024-04-23T18:25:26.102178+0000","last_became_peered":"2024-04-23T18:25:26.102178+0000","last_unstale":"2024-04-23T18:26:29.523123+0000","last_undegraded":"2024-04-23T18:26:29.523123+0000","last_fullsized":"2024-04-23T18:26:29.523123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:25.258866+0000","last_clean_scrub_stamp":"2024-04-23T18:24:25.258866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:51:20.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523645+0000","last_change":"2024-04-23T18:24:26.887285+0000","last_active":"2024-04-23T18:26:29.523645+0000","last_peered":"2024-04-23T18:26:29.523645+0000","last_clean":"2024-04-23T18:26:29.523645+0000","last_became_active":"2024-04-23T18:24:26.886990+0000","last_became_peered":"2024-04-23T18:24:26.886990+0000","last_unstale":"2024-04-23T18:26:29.523645+0000","last_undegraded":"2024-04-23T18:26:29.523645+0000","last_fullsized":"2024-04-23T18:26:29.523645+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:56:43.364716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468549+0000","last_change":"2024-04-23T18:24:26.890546+0000","last_active":"2024-04-23T18:26:29.468549+0000","last_peered":"2024-04-23T18:26:29.468549+0000","last_clean":"2024-04-23T18:26:29.468549+0000","last_became_active":"2024-04-23T18:24:26.890220+0000","last_became_peered":"2024-04-23T18:24:26.890220+0000","last_unstale":"2024-04-23T18:26:29.468549+0000","last_undegraded":"2024-04-23T18:26:29.468549+0000","last_fullsized":"2024-04-23T18:26:29.468549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T05:55:44.194217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542861+0000","last_change":"2024-04-23T18:24:26.894976+0000","last_active":"2024-04-23T18:26:29.542861+0000","last_peered":"2024-04-23T18:26:29.542861+0000","last_clean":"2024-04-23T18:26:29.542861+0000","last_became_active":"2024-04-23T18:24:26.894647+0000","last_became_peered":"2024-04-23T18:24:26.894647+0000","last_unstale":"2024-04-23T18:26:29.542861+0000","last_undegraded":"2024-04-23T18:26:29.542861+0000","last_fullsized":"2024-04-23T18:26:29.542861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:02:36.759159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.542855+0000","last_change":"2024-04-23T18:24:26.893052+0000","last_active":"2024-04-23T18:26:29.542855+0000","last_peered":"2024-04-23T18:26:29.542855+0000","last_clean":"2024-04-23T18:26:29.542855+0000","last_became_active":"2024-04-23T18:24:26.892710+0000","last_became_peered":"2024-04-23T18:24:26.892710+0000","last_unstale":"2024-04-23T18:26:29.542855+0000","last_undegraded":"2024-04-23T18:26:29.542855+0000","last_fullsized":"2024-04-23T18:26:29.542855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:43:27.635201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683626+0000","last_change":"2024-04-23T18:24:26.895563+0000","last_active":"2024-04-23T18:26:29.683626+0000","last_peered":"2024-04-23T18:26:29.683626+0000","last_clean":"2024-04-23T18:26:29.683626+0000","last_became_active":"2024-04-23T18:24:26.894954+0000","last_became_peered":"2024-04-23T18:24:26.894954+0000","last_unstale":"2024-04-23T18:26:29.683626+0000","last_undegraded":"2024-04-23T18:26:29.683626+0000","last_fullsized":"2024-04-23T18:26:29.683626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T04:49:07.297930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523299+0000","last_change":"2024-04-23T18:24:26.889362+0000","last_active":"2024-04-23T18:26:29.523299+0000","last_peered":"2024-04-23T18:26:29.523299+0000","last_clean":"2024-04-23T18:26:29.523299+0000","last_became_active":"2024-04-23T18:24:26.889044+0000","last_became_peered":"2024-04-23T18:24:26.889044+0000","last_unstale":"2024-04-23T18:26:29.523299+0000","last_undegraded":"2024-04-23T18:26:29.523299+0000","last_fullsized":"2024-04-23T18:26:29.523299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:02:50.069800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468775+0000","last_change":"2024-04-23T18:25:25.915473+0000","last_active":"2024-04-23T18:26:29.468775+0000","last_peered":"2024-04-23T18:26:29.468775+0000","last_clean":"2024-04-23T18:26:29.468775+0000","last_became_active":"2024-04-23T18:25:25.914874+0000","last_became_peered":"2024-04-23T18:25:25.914874+0000","last_unstale":"2024-04-23T18:26:29.468775+0000","last_undegraded":"2024-04-23T18:26:29.468775+0000","last_fullsized":"2024-04-23T18:26:29.468775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:55:14.348994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.543211+0000","last_change":"2024-04-23T18:24:26.888942+0000","last_active":"2024-04-23T18:26:29.543211+0000","last_peered":"2024-04-23T18:26:29.543211+0000","last_clean":"2024-04-23T18:26:29.543211+0000","last_became_active":"2024-04-23T18:24:26.888609+0000","last_became_peered":"2024-04-23T18:24:26.888609+0000","last_unstale":"2024-04-23T18:26:29.543211+0000","last_undegraded":"2024-04-23T18:26:29.543211+0000","last_fullsized":"2024-04-23T18:26:29.543211+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T06:07:40.126483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525495+0000","last_change":"2024-04-23T18:24:26.893898+0000","last_active":"2024-04-23T18:26:29.525495+0000","last_peered":"2024-04-23T18:26:29.525495+0000","last_clean":"2024-04-23T18:26:29.525495+0000","last_became_active":"2024-04-23T18:24:26.893602+0000","last_became_peered":"2024-04-23T18:24:26.893602+0000","last_unstale":"2024-04-23T18:26:29.525495+0000","last_undegraded":"2024-04-23T18:26:29.525495+0000","last_fullsized":"2024-04-23T18:26:29.525495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:26:53.039029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.524255+0000","last_change":"2024-04-23T18:24:26.892579+0000","last_active":"2024-04-23T18:26:29.524255+0000","last_peered":"2024-04-23T18:26:29.524255+0000","last_clean":"2024-04-23T18:26:29.524255+0000","last_became_active":"2024-04-23T18:24:26.892275+0000","last_became_peered":"2024-04-23T18:24:26.892275+0000","last_unstale":"2024-04-23T18:26:29.524255+0000","last_undegraded":"2024-04-23T18:26:29.524255+0000","last_fullsized":"2024-04-23T18:26:29.524255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T02:13:14.008807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.525294+0000","last_change":"2024-04-23T18:24:26.891541+0000","last_active":"2024-04-23T18:26:29.525294+0000","last_peered":"2024-04-23T18:26:29.525294+0000","last_clean":"2024-04-23T18:26:29.525294+0000","last_became_active":"2024-04-23T18:24:26.891218+0000","last_became_peered":"2024-04-23T18:24:26.891218+0000","last_unstale":"2024-04-23T18:26:29.525294+0000","last_undegraded":"2024-04-23T18:26:29.525294+0000","last_fullsized":"2024-04-23T18:26:29.525294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:13:33.924828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.683418+0000","last_change":"2024-04-23T18:24:26.891072+0000","last_active":"2024-04-23T18:26:29.683418+0000","last_peered":"2024-04-23T18:26:29.683418+0000","last_clean":"2024-04-23T18:26:29.683418+0000","last_became_active":"2024-04-23T18:24:26.890764+0000","last_became_peered":"2024-04-23T18:24:26.890764+0000","last_unstale":"2024-04-23T18:26:29.683418+0000","last_undegraded":"2024-04-23T18:26:29.683418+0000","last_fullsized":"2024-04-23T18:26:29.683418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T19:34:16.060141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.523149+0000","last_change":"2024-04-23T18:24:26.888393+0000","last_active":"2024-04-23T18:26:29.523149+0000","last_peered":"2024-04-23T18:26:29.523149+0000","last_clean":"2024-04-23T18:26:29.523149+0000","last_became_active":"2024-04-23T18:24:26.888067+0000","last_became_peered":"2024-04-23T18:24:26.888067+0000","last_unstale":"2024-04-23T18:26:29.523149+0000","last_undegraded":"2024-04-23T18:26:29.523149+0000","last_fullsized":"2024-04-23T18:26:29.523149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T00:41:17.593544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-23T18:26:29.468526+0000","last_change":"2024-04-23T18:25:25.917314+0000","last_active":"2024-04-23T18:26:29.468526+0000","last_peered":"2024-04-23T18:26:29.468526+0000","last_clean":"2024-04-23T18:26:29.468526+0000","last_became_active":"2024-04-23T18:25:25.916841+0000","last_became_peered":"2024-04-23T18:25:25.916841+0000","last_unstale":"2024-04-23T18:26:29.468526+0000","last_undegraded":"2024-04-23T18:26:29.468526+0000","last_fullsized":"2024-04-23T18:26:29.468526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-23T18:24:26.269427+0000","last_clean_scrub_stamp":"2024-04-23T18:24:26.269427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-24T23:12:03.413973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107010,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600909,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1297,"num_read_kb":764,"num_write":376,"num_write_kb":1470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":1999750,"data_compressed":85095,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":466241,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1142784,"data_stored":1254964,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":465780,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070517,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":859374,"data_compressed":27877,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":1060830,"data_compressed":44401,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":466279,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1040384,"data_stored":533536,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":991232,"data_stored":467227,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1060864,"data_stored":534987,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":1186770,"data_compressed":43222,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":466281,"data_compressed":8731,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":1190794,"data_compressed":50557,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35670,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":70329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135651,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200994,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-23T18:26:35.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/338053613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc106170 msgr2=0x7f8ccc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/338053613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc106170 0x7f8ccc10dd40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009ca0 tx=0x7f8cc402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/338053613 shutdown_connections 2024-04-23T18:26:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/338053613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc106170 0x7f8ccc10dd40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/338053613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc101a50 0x7f8ccc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/338053613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc1010a0 0x7f8ccc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/338053613 >> 172.21.15.33:0/338053613 conn(0x7f8ccc077f20 msgr2=0x7f8ccc078330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/338053613 shutdown_connections 2024-04-23T18:26:36.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.126+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/338053613 wait complete. 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 Processor -- start 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 -- start start 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc1010a0 0x7f8ccc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc106170 0x7f8ccc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1110e0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc110f60 con 0x7f8ccc1010a0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cd0c8a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc111260 con 0x7f8ccc106170 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44302/0 (socket says 172.21.15.33:44302) 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8ccad9d640 1 -- 172.21.15.33:0/78321426 learned_addr learned my addr 172.21.15.33:0/78321426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cc3fff640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc106170 0x7f8ccc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.127+0000 7f8cca59c640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc1010a0 0x7f8ccc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 -- 172.21.15.33:0/78321426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc106170 msgr2=0x7f8ccc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc106170 0x7f8ccc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 -- 172.21.15.33:0/78321426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc1010a0 msgr2=0x7f8ccc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc1010a0 0x7f8ccc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 -- 172.21.15.33:0/78321426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc19dab0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8ccad9d640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005cc0 tx=0x7f8cc402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4003bf0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc19dd40 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4003d90 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc402fbc0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.128+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a5560 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.129+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc0691c0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.130+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f8cc402fd60 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.131+0000 7f8cc37fe640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 0x7f8cb80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.131+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f8cc40f2260 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.131+0000 7f8cca59c640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 0x7f8cb80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.132+0000 7f8cca59c640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 0x7f8cb80a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005fd0 tx=0x7f8cbc00b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.135+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cc407fff0 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.362+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ccc101a50 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.363+0000 7f8cc37fe640 1 -- 172.21.15.33:0/78321426 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f8cc40be770 con 0x7f8ccc19cfc0 2024-04-23T18:26:36.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:36.356 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:26:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:26:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 msgr2=0x7f8cb80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 0x7f8cb80a6300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005fd0 tx=0x7f8cbc00b390 comp rx=0 tx=0).stop 2024-04-23T18:26:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 msgr2=0x7f8ccc19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4005cc0 tx=0x7f8cc402f380 comp rx=0 tx=0).stop 2024-04-23T18:26:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 shutdown_connections 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8cb80a3e40 0x7f8cb80a6300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ccc19cfc0 0x7f8ccc19d470 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ccc106170 0x7f8ccc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 --2- 172.21.15.33:0/78321426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ccc1010a0 0x7f8ccc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 >> 172.21.15.33:0/78321426 conn(0x7f8ccc077f20 msgr2=0x7f8ccc06a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 shutdown_connections 2024-04-23T18:26:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.365+0000 7f8cd0c8a640 1 -- 172.21.15.33:0/78321426 wait complete. 2024-04-23T18:26:36.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 -- 172.21.15.33:0/203012512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c101110 msgr2=0x7f956c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 --2- 172.21.15.33:0/203012512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c101110 0x7f956c1014f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f955c009900 tx=0x7f955c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 -- 172.21.15.33:0/203012512 shutdown_connections 2024-04-23T18:26:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 --2- 172.21.15.33:0/203012512 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c106190 0x7f956c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 --2- 172.21.15.33:0/203012512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c101ac0 0x7f956c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 --2- 172.21.15.33:0/203012512 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c101110 0x7f956c1014f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 -- 172.21.15.33:0/203012512 >> 172.21.15.33:0/203012512 conn(0x7f956c077fb0 msgr2=0x7f956c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 -- 172.21.15.33:0/203012512 shutdown_connections 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 -- 172.21.15.33:0/203012512 wait complete. 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.479+0000 7f9572108640 1 Processor -- start 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 -- start start 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c101110 0x7f956c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 0x7f956c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f956c112210 con 0x7f956c19d010 2024-04-23T18:26:36.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f956c112090 con 0x7f956c101110 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f9572108640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f956c112390 con 0x7f956c106190 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44316/0 (socket says 172.21.15.33:44316) 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 -- 172.21.15.33:0/1538701801 learned_addr learned my addr 172.21.15.33:0/1538701801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956b7fe640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c101110 0x7f956c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956affd640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 0x7f956c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 -- 172.21.15.33:0/1538701801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 msgr2=0x7f956c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 0x7f956c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 -- 172.21.15.33:0/1538701801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c101110 msgr2=0x7f956c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c101110 0x7f956c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 -- 172.21.15.33:0/1538701801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956c19db00 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956affd640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 0x7f956c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.480+0000 7f956bfff640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f956400da40 tx=0x7f956400df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.481+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564017070 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.481+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9564003e20 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.481+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956c19ddf0 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.481+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564010460 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.481+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956c1a5590 con 0x7f956c19d010 2024-04-23T18:26:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.482+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f956c0684e0 con 0x7f956c19d010 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.483+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f9564004020 con 0x7f956c19d010 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.483+0000 7f9568ff9640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 0x7f95540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.484+0000 7f956b7fe640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 0x7f95540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.484+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f95640d0730 con 0x7f956c19d010 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.484+0000 7f956b7fe640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 0x7f95540a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f955c031440 tx=0x7f955c005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.487+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f956409cb90 con 0x7f956c19d010 2024-04-23T18:26:36.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.676+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f956c199aa0 con 0x7f956c19d010 2024-04-23T18:26:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.677+0000 7f9568ff9640 1 -- 172.21.15.33:0/1538701801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23101 (secure 0 0 0) 0x7f956c199aa0 con 0x7f956c19d010 2024-04-23T18:26:36.670 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:26:29.462863+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 msgr2=0x7f95540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 0x7f95540a6300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f955c031440 tx=0x7f955c005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:36.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 msgr2=0x7f956c19d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f956400da40 tx=0x7f956400df00 comp rx=0 tx=0).stop 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 shutdown_connections 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95540a3e40 0x7f95540a6300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f956c19d010 0x7f956c19d4c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956c106190 0x7f956c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 --2- 172.21.15.33:0/1538701801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f956c101110 0x7f956c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 >> 172.21.15.33:0/1538701801 conn(0x7f956c077fb0 msgr2=0x7f956c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 shutdown_connections 2024-04-23T18:26:36.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.679+0000 7f9572108640 1 -- 172.21.15.33:0/1538701801 wait complete. 2024-04-23T18:26:36.684 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:26:36.684 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:26:36.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 -- 172.21.15.33:0/2236951414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 msgr2=0x7f5ffc10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 --2- 172.21.15.33:0/2236951414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc10bfd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009900 tx=0x7f5ff002a510 comp rx=0 tx=0).stop 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 -- 172.21.15.33:0/2236951414 shutdown_connections 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 --2- 172.21.15.33:0/2236951414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 0x7f5ffc10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 --2- 172.21.15.33:0/2236951414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc10bfd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 --2- 172.21.15.33:0/2236951414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 0x7f5ffc1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.796+0000 7f60014e1640 1 -- 172.21.15.33:0/2236951414 >> 172.21.15.33:0/2236951414 conn(0x7f5ffc077f50 msgr2=0x7f5ffc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- 172.21.15.33:0/2236951414 shutdown_connections 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- 172.21.15.33:0/2236951414 wait complete. 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 Processor -- start 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- start start 2024-04-23T18:26:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 0x7f5ffc100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 0x7f5ffc1018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc112200 con 0x7f5ffc103af0 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc112080 con 0x7f5ffc10c510 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.797+0000 7f60014e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ffc112380 con 0x7f5ffc1031d0 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45986/0 (socket says 172.21.15.33:45986) 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 -- 172.21.15.33:0/121145625 learned_addr learned my addr 172.21.15.33:0/121145625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffaffd640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 0x7f5ffc100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffb7fe640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 0x7f5ffc1018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 -- 172.21.15.33:0/121145625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 msgr2=0x7f5ffc100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 0x7f5ffc100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 -- 172.21.15.33:0/121145625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 msgr2=0x7f5ffc1018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 0x7f5ffc1018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 -- 172.21.15.33:0/121145625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ffc1a6cc0 con 0x7f5ffc103af0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5ffa7fc640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005940 tx=0x7f5ff00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff003f030 con 0x7f5ffc103af0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ffc1a6f50 con 0x7f5ffc103af0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ff0003cb0 con 0x7f5ffc103af0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0035070 con 0x7f5ffc103af0 2024-04-23T18:26:36.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.798+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ffc1a73c0 con 0x7f5ffc103af0 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.799+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ffc068d70 con 0x7f5ffc103af0 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.801+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5ff0042030 con 0x7f5ffc103af0 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.801+0000 7f5fe3fff640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 0x7f5fe40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.801+0000 7f5ffaffd640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 0x7f5fe40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.801+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5ff00f11c0 con 0x7f5ffc103af0 2024-04-23T18:26:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.802+0000 7f5ffaffd640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 0x7f5fe40a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00bbe0 tx=0x7f5fec00b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:36.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.806+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ff00bd0a0 con 0x7f5ffc103af0 2024-04-23T18:26:36.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:36 smithi122 ceph-mon[32546]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:36 smithi122 ceph-mon[32546]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:36.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:36 smithi122 ceph-mon[32546]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:36 smithi122 ceph-mon[32546]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:36 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/78321426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:36 smithi033 ceph-mon[29274]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:36 smithi033 ceph-mon[29274]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:36 smithi033 ceph-mon[29274]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:36 smithi033 ceph-mon[29274]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:36 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/78321426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.990+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ffc107690 con 0x7f5ffc103af0 2024-04-23T18:26:36.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.991+0000 7f5fe3fff640 1 -- 172.21.15.33:0/121145625 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f5ff00dd0a0 con 0x7f5ffc103af0 2024-04-23T18:26:36.984 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:26:36.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.992+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 msgr2=0x7f5fe40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.992+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 0x7f5fe40a6300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00bbe0 tx=0x7f5fec00b390 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.992+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 msgr2=0x7f5ffc101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.992+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0005940 tx=0x7f5ff00042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 shutdown_connections 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5fe40a3e40 0x7f5fe40a6300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ffc10c510 0x7f5ffc1018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ffc103af0 0x7f5ffc101360 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 --2- 172.21.15.33:0/121145625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ffc1031d0 0x7f5ffc100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 >> 172.21.15.33:0/121145625 conn(0x7f5ffc077f50 msgr2=0x7f5ffc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 shutdown_connections 2024-04-23T18:26:36.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:36.993+0000 7f60014e1640 1 -- 172.21.15.33:0/121145625 wait complete. 2024-04-23T18:26:36.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:36 smithi106 ceph-mon[32716]: from='client.15513 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:36 smithi106 ceph-mon[32716]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-23T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:36 smithi106 ceph-mon[32716]: from='client.15519 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:36 smithi106 ceph-mon[32716]: from='client.15525 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-23T18:26:37.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:36 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/78321426' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.108+0000 7fc7f827c640 1 -- 172.21.15.33:0/1187494056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f01031c0 msgr2=0x7fc7f01035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.108+0000 7fc7f827c640 1 --2- 172.21.15.33:0/1187494056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f01031c0 0x7fc7f01035a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009900 tx=0x7fc7e802a510 comp rx=0 tx=0).stop 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- 172.21.15.33:0/1187494056 shutdown_connections 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- 172.21.15.33:0/1187494056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f010c500 0x7fc7f010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- 172.21.15.33:0/1187494056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 0x7fc7f010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- 172.21.15.33:0/1187494056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f01031c0 0x7fc7f01035a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- 172.21.15.33:0/1187494056 >> 172.21.15.33:0/1187494056 conn(0x7fc7f0078000 msgr2=0x7fc7f0078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- 172.21.15.33:0/1187494056 shutdown_connections 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- 172.21.15.33:0/1187494056 wait complete. 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 Processor -- start 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- start start 2024-04-23T18:26:37.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f01031c0 0x7fc7f0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 0x7fc7f0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01121c0 con 0x7fc7f010c500 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0112040 con 0x7fc7f01031c0 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.109+0000 7fc7f827c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0112340 con 0x7fc7f0103ae0 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45996/0 (socket says 172.21.15.33:45996) 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 -- 172.21.15.33:0/499319177 learned_addr learned my addr 172.21.15.33:0/499319177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6018640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f01031c0 0x7fc7f0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f5817640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 0x7fc7f0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 -- 172.21.15.33:0/499319177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 msgr2=0x7fc7f0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 0x7fc7f0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 -- 172.21.15.33:0/499319177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f01031c0 msgr2=0x7fc7f0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f01031c0 0x7fc7f0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 -- 172.21.15.33:0/499319177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f019d710 con 0x7fc7f010c500 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f6819640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000eb10 tx=0x7fc7e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0004b20 con 0x7fc7f010c500 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7e0004cc0 con 0x7fc7f010c500 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f019da00 con 0x7fc7f010c500 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0010670 con 0x7fc7f010c500 2024-04-23T18:26:37.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.110+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f01a5290 con 0x7fc7f010c500 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.111+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f0068d70 con 0x7fc7f010c500 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.112+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc7e0005020 con 0x7fc7f010c500 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.113+0000 7fc7ef7fe640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 0x7fc7d80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.113+0000 7fc7f6018640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 0x7fc7d80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.113+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc7e00d0f50 con 0x7fc7f010c500 2024-04-23T18:26:37.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.114+0000 7fc7f6018640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 0x7fc7d80a6380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005ab0 tx=0x7fc7e8005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.118+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7e009d350 con 0x7fc7f010c500 2024-04-23T18:26:37.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.302+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc7f002d5c0 con 0x7fc7f010c500 2024-04-23T18:26:37.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.302+0000 7fc7ef7fe640 1 -- 172.21.15.33:0/499319177 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fc7e009cc50 con 0x7fc7f010c500 2024-04-23T18:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 msgr2=0x7fc7d80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 0x7fc7d80a6380 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8005ab0 tx=0x7fc7e8005a20 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 msgr2=0x7fc7f019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000eb10 tx=0x7fc7e000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 shutdown_connections 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7d80a3ec0 0x7fc7d80a6380 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7f010c500 0x7fc7f019cfe0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7f0103ae0 0x7fc7f0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 --2- 172.21.15.33:0/499319177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7f01031c0 0x7fc7f0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 >> 172.21.15.33:0/499319177 conn(0x7fc7f0078000 msgr2=0x7fc7f00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 shutdown_connections 2024-04-23T18:26:37.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.304+0000 7fc7f827c640 1 -- 172.21.15.33:0/499319177 wait complete. 2024-04-23T18:26:37.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:26:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- 172.21.15.33:0/1232329093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218101090 msgr2=0x7fc218101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 --2- 172.21.15.33:0/1232329093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218101090 0x7fc218101470 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc214009900 tx=0x7fc21402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- 172.21.15.33:0/1232329093 shutdown_connections 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 --2- 172.21.15.33:0/1232329093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc218106160 0x7fc21810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 --2- 172.21.15.33:0/1232329093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101a40 0x7fc218105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 --2- 172.21.15.33:0/1232329093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218101090 0x7fc218101470 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- 172.21.15.33:0/1232329093 >> 172.21.15.33:0/1232329093 conn(0x7fc218077f50 msgr2=0x7fc218078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- 172.21.15.33:0/1232329093 shutdown_connections 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- 172.21.15.33:0/1232329093 wait complete. 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 Processor -- start 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.421+0000 7fc21fda1640 1 -- start start 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101090 0x7fc21819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2181985a0 0x7fc218198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2181121e0 con 0x7fc218106160 2024-04-23T18:26:37.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc218112060 con 0x7fc218101090 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc218112360 con 0x7fc2181985a0 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46024/0 (socket says 172.21.15.33:46024) 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 -- 172.21.15.33:0/3936221862 learned_addr learned my addr 172.21.15.33:0/3936221862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21db3d640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101090 0x7fc21819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 -- 172.21.15.33:0/3936221862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2181985a0 msgr2=0x7fc218198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2181985a0 0x7fc218198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 -- 172.21.15.33:0/3936221862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101090 msgr2=0x7fc21819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101090 0x7fc21819e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 -- 172.21.15.33:0/3936221862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2181990c0 con 0x7fc218106160 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21d33c640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc20800d770 tx=0x7fc20800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208017070 con 0x7fc218106160 2024-04-23T18:26:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc208009c30 con 0x7fc218106160 2024-04-23T18:26:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2181993b0 con 0x7fc218106160 2024-04-23T18:26:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800f620 con 0x7fc218106160 2024-04-23T18:26:37.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.422+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2181a5650 con 0x7fc218106160 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.423+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc218068d70 con 0x7fc218106160 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.424+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc20801a070 con 0x7fc218106160 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.425+0000 7fc20effd640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 0x7fc2040a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.425+0000 7fc21db3d640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 0x7fc2040a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.426+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc2080d05f0 con 0x7fc218106160 2024-04-23T18:26:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.426+0000 7fc21db3d640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 0x7fc2040a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc214031440 tx=0x7fc214005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.430+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc208009de0 con 0x7fc218106160 2024-04-23T18:26:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.618+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc218101a40 con 0x7fc218106160 2024-04-23T18:26:37.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.618+0000 7fc20effd640 1 -- 172.21.15.33:0/3936221862 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fc20809c9f0 con 0x7fc218106160 2024-04-23T18:26:37.612 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 msgr2=0x7fc2040a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 0x7fc2040a6380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc214031440 tx=0x7fc214005b50 comp rx=0 tx=0).stop 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 msgr2=0x7fc21819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc20800d770 tx=0x7fc20800dc30 comp rx=0 tx=0).stop 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 shutdown_connections 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2040a3ec0 0x7fc2040a6380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2181985a0 0x7fc218198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc218106160 0x7fc21819e930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 --2- 172.21.15.33:0/3936221862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc218101090 0x7fc21819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 >> 172.21.15.33:0/3936221862 conn(0x7fc218077f50 msgr2=0x7fc2180ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 shutdown_connections 2024-04-23T18:26:37.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.620+0000 7fc21fda1640 1 -- 172.21.15.33:0/3936221862 wait complete. 2024-04-23T18:26:37.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.733+0000 7eff5434a640 1 -- 172.21.15.33:0/4076664467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 msgr2=0x7eff4c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.733+0000 7eff5434a640 1 --2- 172.21.15.33:0/4076664467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c10bfe0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7eff3c009900 tx=0x7eff3c02a510 comp rx=0 tx=0).stop 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- 172.21.15.33:0/4076664467 shutdown_connections 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- 172.21.15.33:0/4076664467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 0x7eff4c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- 172.21.15.33:0/4076664467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c10bfe0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- 172.21.15.33:0/4076664467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 0x7eff4c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- 172.21.15.33:0/4076664467 >> 172.21.15.33:0/4076664467 conn(0x7eff4c077f50 msgr2=0x7eff4c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- 172.21.15.33:0/4076664467 shutdown_connections 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- 172.21.15.33:0/4076664467 wait complete. 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 Processor -- start 2024-04-23T18:26:37.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- start start 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 0x7eff4c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 0x7eff4c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c1121e0 con 0x7eff4c103b00 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c112060 con 0x7eff4c1031e0 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff5434a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c112360 con 0x7eff4c10c520 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.734+0000 7eff518e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46054/0 (socket says 172.21.15.33:46054) 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 -- 172.21.15.33:0/2246605294 learned_addr learned my addr 172.21.15.33:0/2246605294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff520e6640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 0x7eff4c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff528e7640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 0x7eff4c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 -- 172.21.15.33:0/2246605294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 msgr2=0x7eff4c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 0x7eff4c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 -- 172.21.15.33:0/2246605294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 msgr2=0x7eff4c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 0x7eff4c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 -- 172.21.15.33:0/2246605294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c198d20 con 0x7eff4c103b00 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff518e5640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005ab0 tx=0x7eff3c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c03f070 con 0x7eff4c103b00 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff3c031d60 con 0x7eff4c103b00 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c198fb0 con 0x7eff4c103b00 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c035030 con 0x7eff4c103b00 2024-04-23T18:26:37.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.735+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c1a52b0 con 0x7eff4c103b00 2024-04-23T18:26:37.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.736+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c068d70 con 0x7eff4c103b00 2024-04-23T18:26:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.737+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7eff3c042070 con 0x7eff4c103b00 2024-04-23T18:26:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.738+0000 7eff437fe640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 0x7eff380a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.738+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7eff3c0f0dd0 con 0x7eff4c103b00 2024-04-23T18:26:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.738+0000 7eff520e6640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 0x7eff380a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:37.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.738+0000 7eff520e6640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 0x7eff380a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7eff48009bd0 tx=0x7eff48005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.743+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff3c0bd300 con 0x7eff4c103b00 2024-04-23T18:26:37.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1538701801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:37.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/121145625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/499319177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:37 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3936221862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.968+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff4c1076a0 con 0x7eff4c103b00 2024-04-23T18:26:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.969+0000 7eff437fe640 1 -- 172.21.15.33:0/2246605294 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7eff3c0bcc00 con 0x7eff4c103b00 2024-04-23T18:26:37.963 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:37.963 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:26:37.963 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 msgr2=0x7eff380a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 0x7eff380a6380 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7eff48009bd0 tx=0x7eff48005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 msgr2=0x7eff4c19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff3c005ab0 tx=0x7eff3c003c80 comp rx=0 tx=0).stop 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 shutdown_connections 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7eff380a3ec0 0x7eff380a6380 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff4c10c520 0x7eff4c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff4c103b00 0x7eff4c19e930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 --2- 172.21.15.33:0/2246605294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff4c1031e0 0x7eff4c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:37.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 >> 172.21.15.33:0/2246605294 conn(0x7eff4c077f50 msgr2=0x7eff4c0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:37.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 shutdown_connections 2024-04-23T18:26:37.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:37.971+0000 7eff5434a640 1 -- 172.21.15.33:0/2246605294 wait complete. 2024-04-23T18:26:37.966 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1538701801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:37.966 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/121145625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.966 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/499319177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.966 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:37 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3936221862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1538701801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:37.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/121145625' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/499319177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:37 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3936221862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:26:37.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- 172.21.15.33:0/2462090021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 msgr2=0x7f5b64101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 --2- 172.21.15.33:0/2462090021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64101480 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009900 tx=0x7f5b5402a410 comp rx=0 tx=0).stop 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- 172.21.15.33:0/2462090021 shutdown_connections 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 --2- 172.21.15.33:0/2462090021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b64106170 0x7f5b6410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 --2- 172.21.15.33:0/2462090021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64101a50 0x7f5b64105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 --2- 172.21.15.33:0/2462090021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64101480 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- 172.21.15.33:0/2462090021 >> 172.21.15.33:0/2462090021 conn(0x7f5b64077f50 msgr2=0x7f5b64078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- 172.21.15.33:0/2462090021 shutdown_connections 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- 172.21.15.33:0/2462090021 wait complete. 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 Processor -- start 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.085+0000 7f5b68e31640 1 -- start start 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 0x7f5b64198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6419d010 0x7f5b6419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b641121f0 con 0x7f5b641010a0 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64112070 con 0x7f5b6419d010 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b64112370 con 0x7f5b64106170 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46068/0 (socket says 172.21.15.33:46068) 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 -- 172.21.15.33:0/122941712 learned_addr learned my addr 172.21.15.33:0/122941712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b62d9d640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6419d010 0x7f5b6419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b5bfff640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 0x7f5b64198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 -- 172.21.15.33:0/122941712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 msgr2=0x7f5b64198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 0x7f5b64198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 -- 172.21.15.33:0/122941712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6419d010 msgr2=0x7f5b6419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6419d010 0x7f5b6419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 -- 172.21.15.33:0/122941712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6419db00 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b5bfff640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 0x7f5b64198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b6259c640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b54005940 tx=0x7f5b540040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5403f030 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b540044c0 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54035070 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6419dd90 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.086+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b641a5550 con 0x7f5b641010a0 2024-04-23T18:26:38.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.087+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b64068d70 con 0x7f5b641010a0 2024-04-23T18:26:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.088+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5b54042030 con 0x7f5b641010a0 2024-04-23T18:26:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.089+0000 7f5b5b7fe640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.090+0000 7f5b5bfff640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.090+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5b540f1020 con 0x7f5b641010a0 2024-04-23T18:26:38.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.090+0000 7f5b5bfff640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b64100a30 tx=0x7f5b4c005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.093+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b540bd550 con 0x7f5b641010a0 2024-04-23T18:26:38.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.280+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b64199aa0 con 0x7f5b641010a0 2024-04-23T18:26:38.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.282+0000 7f5b5b7fe640 1 -- 172.21.15.33:0/122941712 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+23101 (secure 0 0 0) 0x7f5b64199aa0 con 0x7f5b641010a0 2024-04-23T18:26:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:38.276 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:26:29.462863+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 msgr2=0x7f5b500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b64100a30 tx=0x7f5b4c005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 msgr2=0x7f5b64198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5b54005940 tx=0x7f5b540040c0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 shutdown_connections 2024-04-23T18:26:38.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6419d010 0x7f5b6419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b64106170 0x7f5b64198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 --2- 172.21.15.33:0/122941712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b641010a0 0x7f5b64198750 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 >> 172.21.15.33:0/122941712 conn(0x7f5b64077f50 msgr2=0x7f5b640ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 shutdown_connections 2024-04-23T18:26:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.284+0000 7f5b68e31640 1 -- 172.21.15.33:0/122941712 wait complete. 2024-04-23T18:26:38.289 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 959411 2024-04-23T18:26:38.290 DEBUG:tasks.quiescer:Yielding 2024-04-23T18:26:38.290 INFO:teuthology.run_tasks:Running task workunit... 2024-04-23T18:26:38.305 INFO:tasks.workunit:Pulling workunits from ref 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:26:38.306 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-23T18:26:38.306 INFO:tasks.workunit:timeout=3h 2024-04-23T18:26:38.306 INFO:tasks.workunit:cleanup=True 2024-04-23T18:26:38.306 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:38.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-23 18:24:33.785150771 +0000 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-23 18:24:33.785150771 +0000 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-23 18:26:27.366152073 +0000 2024-04-23T18:26:38.322 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-23 18:24:33.785150771 +0000 2024-04-23T18:26:38.323 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-23T18:26:38.323 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-23T18:26:38.394 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 57e0caf76fdb7ab8b1358588f292d08519163844 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.416+0000 7f7253c10640 1 -- 172.21.15.33:0/1929780563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c101a50 msgr2=0x7f724c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.416+0000 7f7253c10640 1 --2- 172.21.15.33:0/1929780563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c101a50 0x7f724c105a90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f723c009900 tx=0x7f723c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- 172.21.15.33:0/1929780563 shutdown_connections 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- 172.21.15.33:0/1929780563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c106170 0x7f724c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- 172.21.15.33:0/1929780563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c101a50 0x7f724c105a90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- 172.21.15.33:0/1929780563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c1010a0 0x7f724c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- 172.21.15.33:0/1929780563 >> 172.21.15.33:0/1929780563 conn(0x7f724c077f50 msgr2=0x7f724c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- 172.21.15.33:0/1929780563 shutdown_connections 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- 172.21.15.33:0/1929780563 wait complete. 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 Processor -- start 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- start start 2024-04-23T18:26:38.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 0x7f724c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c19d000 0x7f724c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f724c111090 con 0x7f724c106170 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f724c110f10 con 0x7f724c19d000 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.417+0000 7f7253c10640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f724c111210 con 0x7f724c1010a0 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72519ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 0x7f724c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72519ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 0x7f724c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:34354/0 (socket says 172.21.15.33:34354) 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72519ac640 1 -- 172.21.15.33:0/1727675615 learned_addr learned my addr 172.21.15.33:0/1727675615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46078/0 (socket says 172.21.15.33:46078) 2024-04-23T18:26:38.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72521ad640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c19d000 0x7f724c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 -- 172.21.15.33:0/1727675615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 msgr2=0x7f724c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 0x7f724c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 -- 172.21.15.33:0/1727675615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c19d000 msgr2=0x7f724c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c19d000 0x7f724c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 -- 172.21.15.33:0/1727675615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c19daf0 con 0x7f724c106170 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f72511ab640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f723c02a930 tx=0x7f723c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c03f070 con 0x7f724c106170 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f723c004490 con 0x7f724c106170 2024-04-23T18:26:38.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c19dd80 con 0x7f724c106170 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723c035070 con 0x7f724c106170 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.418+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c1a55a0 con 0x7f724c106170 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.419+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724c068d70 con 0x7f724c106170 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.421+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f723c03b460 con 0x7f724c106170 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.422+0000 7f7242ffd640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 0x7f72380a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.422+0000 7f72519ac640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 0x7f72380a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.422+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f723c0f1050 con 0x7f724c106170 2024-04-23T18:26:38.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.422+0000 7f72519ac640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 0x7f72380a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7244009c90 tx=0x7f7244005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.426+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f723c0bcf30 con 0x7f724c106170 2024-04-23T18:26:38.426 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-23T18:26:38.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.654+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f724c101a50 con 0x7f724c106170 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.655+0000 7f7242ffd640 1 -- 172.21.15.33:0/1727675615 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f723c0c1230 con 0x7f724c106170 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:26:38.649 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 msgr2=0x7f72380a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 0x7f72380a6380 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7244009c90 tx=0x7f7244005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 msgr2=0x7f724c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f723c02a930 tx=0x7f723c031a20 comp rx=0 tx=0).stop 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 shutdown_connections 2024-04-23T18:26:38.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f72380a3ec0 0x7f72380a6380 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f724c19d000 0x7f724c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f724c106170 0x7f724c198c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 --2- 172.21.15.33:0/1727675615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f724c1010a0 0x7f724c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 >> 172.21.15.33:0/1727675615 conn(0x7f724c077f50 msgr2=0x7f724c069520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 shutdown_connections 2024-04-23T18:26:38.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.657+0000 7f7253c10640 1 -- 172.21.15.33:0/1727675615 wait complete. 2024-04-23T18:26:38.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-23T18:24:26.661481+0000', 'modified': '2024-04-23T18:26:28.070986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24551, 'mds_1': 24515, 'mds_2': 24490, 'mds_3': 24482, 'mds_4': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14628': {'gid': 14628, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/1373616976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 1373616976}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 1373616976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14640': {'gid': 14640, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/363309287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 363309287}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 363309287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6833/2194426420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 2194426420}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 2194426420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24446': {'gid': 24446, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6833/637332963', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6832', 'nonce': 637332963}, {'type': 'v1', 'addr': '172.21.15.106:6833', 'nonce': 637332963}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6835/3209242154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 3209242154}, {'type': 'v1', 'addr': '172.21.15.122:6835', 'nonce': 3209242154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.122:6837/3846576090', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 3846576090}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 3846576090}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.106:6835/3607877814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3607877814}, {'type': 'v1', 'addr': '172.21.15.106:6835', 'nonce': 3607877814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6839/538103378', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 538103378}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 538103378}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24515': {'gid': 24515, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.106:6837/319312146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 319312146}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 319312146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24551': {'gid': 24551, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/830372376', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 830372376}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 830372376}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24551, 'qdb_cluster': [24466, 24515, 24482, 24490, 24551]} 2024-04-23T18:26:38.662 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-23T18:26:38.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-23T18:26:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1612549383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 msgr2=0x7f3ff810ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1612549383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff810ddd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009be0 tx=0x7f3ff002a6c0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1612549383 shutdown_connections 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1612549383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff810ddd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1612549383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101ae0 0x7f3ff8105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1612549383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff8101130 0x7f3ff8101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1612549383 >> 172.21.15.33:0/1612549383 conn(0x7f3ff8077f80 msgr2=0x7f3ff8078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1612549383 shutdown_connections 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.814+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1612549383 wait complete. 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 Processor -- start 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 -- start start 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff81985c0 0x7f3ff8198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8111170 con 0x7f3ff8101130 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff8110ff0 con 0x7f3ff81985c0 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ffe1a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff81112f0 con 0x7f3ff8106200 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46096/0 (socket says 172.21.15.33:46096) 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1113776344 learned_addr learned my addr 172.21.15.33:0/1113776344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff81985c0 0x7f3ff8198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1113776344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 msgr2=0x7f3ff819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1113776344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff81985c0 msgr2=0x7f3ff8198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff81985c0 0x7f3ff8198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.815+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1113776344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff81990e0 con 0x7f3ff8101130 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ff4ff9640 1 -- 172.21.15.33:0/1113776344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800ccc0 con 0x7f3ff8101130 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff81993d0 con 0x7f3ff8101130 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ff4ff9640 1 -- 172.21.15.33:0/1113776344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe800ce60 con 0x7f3ff8101130 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff81a5620 con 0x7f3ff8101130 2024-04-23T18:26:38.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.816+0000 7f3ff4ff9640 1 -- 172.21.15.33:0/1113776344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8018430 con 0x7f3ff8101130 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f3ff4ff9640 1 -- 172.21.15.33:0/1113776344 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3fe80046c0 con 0x7f3ff8101130 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f4000f95740 1 Processor -- start 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f4000f95740 1 -- start start 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f4000f95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d65190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f4000f95740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.818+0000 7f4000f95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d72220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f4000f95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8ccac50 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f4000f95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8c67830 con 0x55c6a8d64db0 2024-04-23T18:26:38.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f4000f95740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8cc6c10 con 0x55c6a8d65760 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff4ff9640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 0x7f3fe00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d72220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:34366/0 (socket says 172.21.15.33:34366) 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 -- 172.21.15.33:0/2968626927 learned_addr learned my addr 172.21.15.33:0/2968626927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d65190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 0x7f3fe00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff4ff9640 1 -- 172.21.15.33:0/1113776344 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3fe80d10d0 con 0x7f3ff8101130 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 -- 172.21.15.33:0/2968626927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 msgr2=0x55c6a8d65190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d65190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 -- 172.21.15.33:0/2968626927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 msgr2=0x55c6a8d72220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d72220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 -- 172.21.15.33:0/2968626927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a89dd2d0 con 0x55c6a8d65760 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80a44c0 tx=0x7f3fe8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6800070213d2ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 0x7f3fe00a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009370 tx=0x7f3ff0009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3fddffb640 1 -- 172.21.15.33:0/2968626927 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe809ce30 con 0x55c6a8d65760 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3fddffb640 1 -- 172.21.15.33:0/2968626927 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe80a11e0 con 0x55c6a8d65760 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.819+0000 7f3fddffb640 1 -- 172.21.15.33:0/2968626927 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80cebb0 con 0x55c6a8d65760 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- 172.21.15.33:0/2968626927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 msgr2=0x55c6a8d6f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80a44c0 tx=0x7f3fe8005740 comp rx=0 tx=0).stop 2024-04-23T18:26:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- 172.21.15.33:0/2968626927 shutdown_connections 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d72220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d6f800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 --2- 172.21.15.33:0/2968626927 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d65190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- 172.21.15.33:0/2968626927 >> 172.21.15.33:0/2968626927 conn(0x55c6a8d61070 msgr2=0x55c6a8d61450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- 172.21.15.33:0/2968626927 shutdown_connections 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- 172.21.15.33:0/2968626927 wait complete. 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.820+0000 7f4000f95740 1 -- start start 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 Processor -- start 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d6d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d72c80 con 0x55c6a8d64db0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8c65470 con 0x55c6a8d65760 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d72e00 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f4000f95740 10 client.0 _ll_register_callbacks cb 0x55c6a8c8af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46120/0 (socket says 172.21.15.33:46120) 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1209035937 learned_addr learned my addr 172.21.15.33:0/1209035937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d6d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d6fdd0 msgr2=0x55c6a8d6d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1209035937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 msgr2=0x55c6a8d6d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d6d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.821+0000 7f3ff77fe640 1 -- 172.21.15.33:0/1209035937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a8aa76b0 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c6a0 tx=0x7f3fe809c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639dc2dd0653194f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe809ce30 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3fe809c280 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fe80a11e0 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fe80a5aa0 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 10 client.15585 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6a8d7eed0 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe80a11e0 con 0x55c6a8d64db0 2024-04-23T18:26:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3fe80a2370 con 0x55c6a8d64db0 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 10 client.15585 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 10 client.15585 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 20 client.15585 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8d80220 0x55c6a8d82420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 4 client.15585 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.822+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a8cd8670 con 0x55c6a8d80220 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.823+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8d80220 0x55c6a8d82420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.823+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8d80220 0x55c6a8d82420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7227878439c7b724 server_cookie=7aef8159186df6b8 in_seq=0 out_seq=0 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.823+0000 7f3fd6ffd640 10 client.15585 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.823+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1209035937 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6a8cd8670 con 0x55c6a8d80220 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.823+0000 7f3fd6ffd640 10 client.15585 handle_command_reply: tid=0 2024-04-23T18:26:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 client.15585 shutdown 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 msgr2=0x55c6a8d6cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c6a0 tx=0x7f3fe809c880 comp rx=0 tx=0).stop 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 shutdown_connections 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8d80220 0x55c6a8d82420 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d6d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f3fd6ffd640 0 client.15585 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 --2- 172.21.15.33:0/1209035937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d64db0 0x55c6a8d6cc90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 >> 172.21.15.33:0/1209035937 conn(0x55c6a8d61070 msgr2=0x55c6a8d66ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 shutdown_connections 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 1 -- 172.21.15.33:0/1209035937 wait complete. 2024-04-23T18:26:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.824+0000 7f4000f95740 20 client.15585 trim_cache size 0 max 16384 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 Processor -- start 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 -- start start 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d81680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d80760 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d80e30 con 0x55c6a8d64db0 2024-04-23T18:26:38.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f4000f95740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d81500 con 0x55c6a8d65760 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f3ff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f3ff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46126/0 (socket says 172.21.15.33:46126) 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f3ff7fff640 1 -- 172.21.15.33:0/909861515 learned_addr learned my addr 172.21.15.33:0/909861515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.842+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d81680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 -- 172.21.15.33:0/909861515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 msgr2=0x55c6a8d67470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d67470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 -- 172.21.15.33:0/909861515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 msgr2=0x55c6a8d81680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d81680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 -- 172.21.15.33:0/909861515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a8d72680 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:38.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d81680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009be0 tx=0x7f3ff0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b7cd7f974ff09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3fdd7fa640 1 -- 172.21.15.33:0/909861515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0036040 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3fdd7fa640 1 -- 172.21.15.33:0/909861515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0003c60 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.843+0000 7f3fdd7fa640 1 -- 172.21.15.33:0/909861515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00317c0 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- 172.21.15.33:0/909861515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 msgr2=0x55c6a8d6bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009be0 tx=0x7f3ff0040000 comp rx=0 tx=0).stop 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- 172.21.15.33:0/909861515 shutdown_connections 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d6bcc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d65760 0x55c6a8d67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- 172.21.15.33:0/909861515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d64db0 0x55c6a8d81680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- 172.21.15.33:0/909861515 >> 172.21.15.33:0/909861515 conn(0x55c6a8d61070 msgr2=0x55c6a8d61450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- 172.21.15.33:0/909861515 shutdown_connections 2024-04-23T18:26:38.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- 172.21.15.33:0/909861515 wait complete. 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- start start 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 Processor -- start 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 0x55c6a8d8a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d8ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d822a0 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d81bd0 con 0x55c6a8d65760 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a8d82970 con 0x55c6a8d64db0 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.844+0000 7f4000f95740 10 client.0 _ll_register_callbacks cb 0x7f3fec0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46132/0 (socket says 172.21.15.33:46132) 2024-04-23T18:26:38.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 -- 172.21.15.33:0/1419066061 learned_addr learned my addr 172.21.15.33:0/1419066061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 0x55c6a8d8a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff6ffd640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d8ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 -- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 msgr2=0x55c6a8d8a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 0x55c6a8d8a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 -- 172.21.15.33:0/1419066061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 msgr2=0x55c6a8d8ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d8ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 -- 172.21.15.33:0/1419066061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a8d84670 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 0x55c6a8d8a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3ff7fff640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ff002abe0 tx=0x7f3ff0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf80708c14e719a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3fd6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0036030 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3ff0004450 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff0003c60 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0066050 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f4000f95740 10 client.15591 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.845+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c6a8d88e90 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0003c60 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ff0039070 con 0x55c6a8d6fdd0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 10 client.15591 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 10 client.15591 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 20 client.15591 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8deb260 0x55c6a8de9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 4 client.15591 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 --> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] -- command(tid 0: {"prefix": "status"}) -- 0x55c6a8cd24e0 con 0x55c6a8deb260 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8deb260 0x55c6a8de9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f3fd6ffd640 10 client.15591 ms_handle_connect on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.846+0000 7f3ff77fe640 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8deb260 0x55c6a8de9f70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ceef4308409c3a0 server_cookie=f59680678139e277 in_seq=0 out_seq=0 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3fd6ffd640 1 -- 172.21.15.33:0/1419066061 <== mds.4 v2:172.21.15.122:6836/3846576090 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c6a8cd24e0 con 0x55c6a8deb260 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3fd6ffd640 10 client.15591 handle_command_reply: tid=0 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 client.15591 shutdown 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 msgr2=0x55c6a8d83e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3ff002abe0 tx=0x7f3ff0003f10 comp rx=0 tx=0).stop 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 shutdown_connections 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:6836/3846576090,v1:172.21.15.122:6837/3846576090] conn(0x55c6a8deb260 0x55c6a8de9f70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a8d6fdd0 0x55c6a8d83e70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a8d65760 0x55c6a8d8ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3fd6ffd640 0 client.15591 ms_handle_reset on v2:172.21.15.122:6836/3846576090 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 --2- 172.21.15.33:0/1419066061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a8d64db0 0x55c6a8d8a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 >> 172.21.15.33:0/1419066061 conn(0x55c6a8d61070 msgr2=0x55c6a8d7b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 shutdown_connections 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 1 -- 172.21.15.33:0/1419066061 wait complete. 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f4000f95740 20 client.15591 trim_cache size 0 max 16384 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24466, 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:38.850 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 125.508891279, 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 150.23240218800001 2024-04-23T18:26:38.851 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 msgr2=0x7f3fe00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 0x7f3fe00a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009370 tx=0x7f3ff0009500 comp rx=0 tx=0).stop 2024-04-23T18:26:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 msgr2=0x7f3ff819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800e9f0 tx=0x7f3fe800edf0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 shutdown_connections 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3fe00a3e40 0x7f3fe00a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ff81985c0 0x7f3ff8198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ff8106200 0x7f3ff819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 --2- 172.21.15.33:0/1113776344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ff8101130 0x7f3ff819e410 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.847+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 >> 172.21.15.33:0/1113776344 conn(0x7f3ff8077f80 msgr2=0x7f3ff806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.848+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 shutdown_connections 2024-04-23T18:26:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.848+0000 7f3ffe1a8640 1 -- 172.21.15.33:0/1113776344 wait complete. 2024-04-23T18:26:38.853 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-04-23T18:26:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:38 smithi122 ceph-mon[32546]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:38 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2246605294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:38 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/122941712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- 172.21.15.33:0/1056692197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60501010e0 msgr2=0x7f6050101560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 --2- 172.21.15.33:0/1056692197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60501010e0 0x7f6050101560 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6040009900 tx=0x7f604002a470 comp rx=0 tx=0).stop 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- 172.21.15.33:0/1056692197 shutdown_connections 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 --2- 172.21.15.33:0/1056692197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6050101aa0 0x7f605010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 --2- 172.21.15.33:0/1056692197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60501010e0 0x7f6050101560 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 --2- 172.21.15.33:0/1056692197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050105e70 0x7f6050106250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- 172.21.15.33:0/1056692197 >> 172.21.15.33:0/1056692197 conn(0x7f6050077cb0 msgr2=0x7f60500780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- 172.21.15.33:0/1056692197 shutdown_connections 2024-04-23T18:26:38.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- 172.21.15.33:0/1056692197 wait complete. 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 Processor -- start 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.963+0000 7f60571ef640 1 -- start start 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60501010e0 0x7f6050199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050101aa0 0x7f6050199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60501110f0 con 0x7f6050105e70 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6050110f70 con 0x7f6050101aa0 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6050111270 con 0x7f60501010e0 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46152/0 (socket says 172.21.15.33:46152) 2024-04-23T18:26:38.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 -- 172.21.15.33:0/2430832019 learned_addr learned my addr 172.21.15.33:0/2430832019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f6054f8b640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60501010e0 0x7f6050199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f6047fff640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050101aa0 0x7f6050199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 -- 172.21.15.33:0/2430832019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60501010e0 msgr2=0x7f6050199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60501010e0 0x7f6050199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 -- 172.21.15.33:0/2430832019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050101aa0 msgr2=0x7f6050199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050101aa0 0x7f6050199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 -- 172.21.15.33:0/2430832019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501a4b10 con 0x7f6050105e70 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f605578c640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f604800eb10 tx=0x7f604800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f6045ffb640 1 -- 172.21.15.33:0/2430832019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048004b20 con 0x7f6050105e70 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501a4ce0 con 0x7f6050105e70 2024-04-23T18:26:38.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f6045ffb640 1 -- 172.21.15.33:0/2430832019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6048004cc0 con 0x7f6050105e70 2024-04-23T18:26:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.964+0000 7f6045ffb640 1 -- 172.21.15.33:0/2430832019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048010670 con 0x7f6050105e70 2024-04-23T18:26:38.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.965+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a51b0 con 0x7f6050105e70 2024-04-23T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6045ffb640 1 -- 172.21.15.33:0/2430832019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6048005020 con 0x7f6050105e70 2024-04-23T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 Processor -- start 2024-04-23T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 -- start start 2024-04-23T18:26:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6045ffb640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 0x55da3f248800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f248dd0 0x55da3f24b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55da3f1a3c50 con 0x55da3f23ddb0 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55da3f140830 con 0x55da3f248dd0 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6059fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55da3f19fc10 con 0x55da3f23e760 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.967+0000 7f6047fff640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6045ffb640 1 -- 172.21.15.33:0/2430832019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f60480d1060 con 0x7f6050105e70 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6047fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 0x55da3f248800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6047fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 0x55da3f248800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:34426/0 (socket says 172.21.15.33:34426) 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f605578c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f248dd0 0x55da3f24b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6047fff640 1 -- 172.21.15.33:0/2677828939 learned_addr learned my addr 172.21.15.33:0/2677828939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46166/0 (socket says 172.21.15.33:46166) 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 -- 172.21.15.33:0/2677828939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 msgr2=0x55da3f248800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 0x55da3f248800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 -- 172.21.15.33:0/2677828939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f248dd0 msgr2=0x55da3f24b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f248dd0 0x55da3f24b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 -- 172.21.15.33:0/2677828939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3eeb62d0 con 0x55da3f23ddb0 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6047fff640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 0x7f60380a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6040031fd0 tx=0x7f6040031aa0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6054f8b640 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f604c00bb40 tx=0x7f604c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f963d7c22e5f48c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6036ffd640 1 -- 172.21.15.33:0/2677828939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c010dd0 con 0x55da3f23ddb0 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6036ffd640 1 -- 172.21.15.33:0/2677828939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c005020 con 0x55da3f23ddb0 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.968+0000 7f6036ffd640 1 -- 172.21.15.33:0/2677828939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c004ad0 con 0x55da3f23ddb0 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- 172.21.15.33:0/2677828939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 msgr2=0x55da3f23e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f604c00bb40 tx=0x7f604c00bf40 comp rx=0 tx=0).stop 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- 172.21.15.33:0/2677828939 shutdown_connections 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f248dd0 0x55da3f24b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23e760 0x55da3f248800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 --2- 172.21.15.33:0/2677828939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f23e190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- 172.21.15.33:0/2677828939 >> 172.21.15.33:0/2677828939 conn(0x55da3f23a070 msgr2=0x55da3f23a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- 172.21.15.33:0/2677828939 shutdown_connections 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- 172.21.15.33:0/2677828939 wait complete. 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 -- start start 2024-04-23T18:26:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.969+0000 7f6059fd6740 1 Processor -- start 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 0x55da3f245c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f2466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55da3f24bd40 con 0x55da3f23e760 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55da3f13e470 con 0x55da3f23ddb0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55da3f24bec0 con 0x55da3f248dd0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6059fd6740 10 client.0 _ll_register_callbacks cb 0x55da3f163f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46182/0 (socket says 172.21.15.33:46182) 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 -- 172.21.15.33:0/3608257350 learned_addr learned my addr 172.21.15.33:0/3608257350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f605578c640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f2466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6054f8b640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 0x55da3f245c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 -- 172.21.15.33:0/3608257350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 msgr2=0x55da3f2466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f2466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 -- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 msgr2=0x55da3f245c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 0x55da3f245c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 -- 172.21.15.33:0/3608257350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3ef806b0 con 0x55da3f23e760 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6054f8b640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 0x55da3f245c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f6047fff640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6040009230 tx=0x7f604002a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30f20e6e039312b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.970+0000 7f602ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604004f030 con 0x55da3f23e760 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f60400038c0 con 0x55da3f23e760 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604c005020 con 0x55da3f23e760 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6040003ac0 con 0x55da3f23e760 2024-04-23T18:26:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 10 client.15603 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da3f257fe0 con 0x55da3f23e760 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604c005020 con 0x55da3f23e760 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f604005dc50 con 0x55da3f23e760 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 10 client.15603 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 10 client.15603 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 20 client.15603 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f25a690 0x55da3f25bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 4 client.15603 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.971+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da3f1b1670 con 0x55da3f25a690 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.972+0000 7f6054f8b640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f25a690 0x55da3f25bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.972+0000 7f6054f8b640 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f25a690 0x55da3f25bb00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eaea4fdf4e9df68f server_cookie=152b2bb7b5da228e in_seq=0 out_seq=0 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.972+0000 7f602ffff640 10 client.15603 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f602ffff640 1 -- 172.21.15.33:0/3608257350 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da3f1b1670 con 0x55da3f25a690 2024-04-23T18:26:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f602ffff640 10 client.15603 handle_command_reply: tid=0 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 client.15603 shutdown 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 msgr2=0x55da3f2461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6040009230 tx=0x7f604002a470 comp rx=0 tx=0).stop 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 shutdown_connections 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f25a690 0x55da3f25bb00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f2466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23e760 0x55da3f2461a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 --2- 172.21.15.33:0/3608257350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23ddb0 0x55da3f245c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f602ffff640 0 client.15603 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 >> 172.21.15.33:0/3608257350 conn(0x55da3f23a070 msgr2=0x55da3f23fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 shutdown_connections 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 1 -- 172.21.15.33:0/3608257350 wait complete. 2024-04-23T18:26:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.973+0000 7f6059fd6740 20 client.15603 trim_cache size 0 max 16384 2024-04-23T18:26:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:38 smithi033 ceph-mon[29274]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2246605294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/122941712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 Processor -- start 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 -- start start 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23ddb0 0x55da3f2405f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f244550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55da3f259e40 con 0x55da3f248dd0 2024-04-23T18:26:38.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55da3f25a510 con 0x55da3f23e760 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6059fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55da3f25abe0 con 0x55da3f23ddb0 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f605578c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f605578c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46190/0 (socket says 172.21.15.33:46190) 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.991+0000 7f6054f8b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23ddb0 0x55da3f2405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 -- 172.21.15.33:0/901216468 learned_addr learned my addr 172.21.15.33:0/901216468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 -- 172.21.15.33:0/901216468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23ddb0 msgr2=0x55da3f2405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23ddb0 0x55da3f2405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 -- 172.21.15.33:0/901216468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 msgr2=0x55da3f244550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f244550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 -- 172.21.15.33:0/901216468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da3f2441d0 con 0x55da3f248dd0 2024-04-23T18:26:38.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f605578c640 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f60480cedc0 tx=0x7f604809cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac07d1bdbab33d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f60367fc640 1 -- 172.21.15.33:0/901216468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604809d4c0 con 0x55da3f248dd0 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f60367fc640 1 -- 172.21.15.33:0/901216468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604809cdc0 con 0x55da3f248dd0 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.992+0000 7f60367fc640 1 -- 172.21.15.33:0/901216468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048011700 con 0x55da3f248dd0 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- 172.21.15.33:0/901216468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 msgr2=0x55da3f244a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f60480cedc0 tx=0x7f604809cc50 comp rx=0 tx=0).stop 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- 172.21.15.33:0/901216468 shutdown_connections 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f248dd0 0x55da3f244a90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f244550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- 172.21.15.33:0/901216468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f23ddb0 0x55da3f2405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- 172.21.15.33:0/901216468 >> 172.21.15.33:0/901216468 conn(0x55da3f23a070 msgr2=0x55da3f240f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- 172.21.15.33:0/901216468 shutdown_connections 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- 172.21.15.33:0/901216468 wait complete. 2024-04-23T18:26:38.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- start start 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 Processor -- start 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f243210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f262230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55da3f25b980 con 0x55da3f23ddb0 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55da3f25b2b0 con 0x55da3f23e760 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55da3f25c050 con 0x55da3f248dd0 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.993+0000 7f6059fd6740 10 client.0 _ll_register_callbacks cb 0x7f60400057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46204/0 (socket says 172.21.15.33:46204) 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 -- 172.21.15.33:0/4112440924 learned_addr learned my addr 172.21.15.33:0/4112440924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f605578c640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f262230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6047fff640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f243210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 -- 172.21.15.33:0/4112440924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 msgr2=0x55da3f262230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f262230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 -- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 msgr2=0x55da3f243210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f243210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 -- 172.21.15.33:0/4112440924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da3f243ac0 con 0x55da3f23ddb0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f605578c640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f262230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6054f8b640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f604c005890 tx=0x7f604c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f877198544808217 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f602ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604c030060 con 0x55da3f23ddb0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f604c005450 con 0x55da3f23ddb0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604809cdc0 con 0x55da3f23ddb0 2024-04-23T18:26:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f604c004680 con 0x55da3f23ddb0 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6059fd6740 10 client.15609 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.994+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55da3f2c0bc0 con 0x55da3f23ddb0 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604809cdc0 con 0x55da3f23ddb0 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f604c02a3d0 con 0x55da3f23ddb0 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 10 client.15609 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 10 client.15609 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 20 client.15609 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f2c5430 0x55da3f2c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 4 client.15609 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 --> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] -- command(tid 0: {"prefix": "status"}) -- 0x55da3f1ab4e0 con 0x55da3f2c5430 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6047fff640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f2c5430 0x55da3f2c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f602ffff640 10 client.15609 ms_handle_connect on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.995+0000 7f6047fff640 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f2c5430 0x55da3f2c4140 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5636a2f9e132694c server_cookie=33da467c2f2a8c54 in_seq=0 out_seq=0 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f602ffff640 1 -- 172.21.15.33:0/4112440924 <== mds.3 v2:172.21.15.106:6834/3607877814 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55da3f1ab4e0 con 0x55da3f2c5430 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f602ffff640 10 client.15609 handle_command_reply: tid=0 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 client.15609 shutdown 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 msgr2=0x55da3f242cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f604c005890 tx=0x7f604c0058e0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 shutdown_connections 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:6834/3607877814,v1:172.21.15.106:6835/3607877814] conn(0x55da3f2c5430 0x55da3f2c4140 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f248dd0 0x55da3f262230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55da3f23e760 0x55da3f243210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 --2- 172.21.15.33:0/4112440924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55da3f23ddb0 0x55da3f242cd0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 >> 172.21.15.33:0/4112440924 conn(0x55da3f23a070 msgr2=0x55da3f2443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f602ffff640 0 client.15609 ms_handle_reset on v2:172.21.15.106:6834/3607877814 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 shutdown_connections 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 1 -- 172.21.15.33:0/4112440924 wait complete. 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.996+0000 7f6059fd6740 20 client.15609 trim_cache size 0 max 16384 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24482, 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:38.998 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 126.664619405, 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 156.00649060399999 2024-04-23T18:26:38.999 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 msgr2=0x7f60380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 0x7f60380a6300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6040031fd0 tx=0x7f6040031aa0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 msgr2=0x7f60501a45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f604800eb10 tx=0x7f604800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 shutdown_connections 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f60380a3e40 0x7f60380a6300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6050105e70 0x7f60501a45d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6050101aa0 0x7f6050199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 --2- 172.21.15.33:0/2430832019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60501010e0 0x7f6050199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 >> 172.21.15.33:0/2430832019 conn(0x7f6050077cb0 msgr2=0x7f60500ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 shutdown_connections 2024-04-23T18:26:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:38.997+0000 7f60571ef640 1 -- 172.21.15.33:0/2430832019 wait complete. 2024-04-23T18:26:39.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.111+0000 7f65308fa640 1 -- 172.21.15.33:0/1851878658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c069d00 msgr2=0x7f652c106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.111+0000 7f65308fa640 1 --2- 172.21.15.33:0/1851878658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c069d00 0x7f652c106b60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6520009900 tx=0x7f652002a410 comp rx=0 tx=0).stop 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 -- 172.21.15.33:0/1851878658 shutdown_connections 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 --2- 172.21.15.33:0/1851878658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c107240 0x7f652c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 --2- 172.21.15.33:0/1851878658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c069d00 0x7f652c106b60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 --2- 172.21.15.33:0/1851878658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c069350 0x7f652c069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 -- 172.21.15.33:0/1851878658 >> 172.21.15.33:0/1851878658 conn(0x7f652c077cb0 msgr2=0x7f652c0780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 -- 172.21.15.33:0/1851878658 shutdown_connections 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 -- 172.21.15.33:0/1851878658 wait complete. 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 Processor -- start 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.112+0000 7f65308fa640 1 -- start start 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c069350 0x7f652c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 0x7f652c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112180 con 0x7f652c19cfa0 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112000 con 0x7f652c107240 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f652c112300 con 0x7f652c069350 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46214/0 (socket says 172.21.15.33:46214) 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 -- 172.21.15.33:0/1669410380 learned_addr learned my addr 172.21.15.33:0/1669410380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652a59c640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c069350 0x7f652c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f6529d9b640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 0x7f652c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 -- 172.21.15.33:0/1669410380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c069350 msgr2=0x7f652c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c069350 0x7f652c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 -- 172.21.15.33:0/1669410380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 msgr2=0x7f652c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 0x7f652c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 -- 172.21.15.33:0/1669410380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c19da90 con 0x7f652c19cfa0 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f6529d9b640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 0x7f652c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f652ad9d640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f652400e970 tx=0x7f652400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f651b7fe640 1 -- 172.21.15.33:0/1669410380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652400ccc0 con 0x7f652c19cfa0 2024-04-23T18:26:39.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652c19dd80 con 0x7f652c19cfa0 2024-04-23T18:26:39.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f651b7fe640 1 -- 172.21.15.33:0/1669410380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f652400ce60 con 0x7f652c19cfa0 2024-04-23T18:26:39.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f651b7fe640 1 -- 172.21.15.33:0/1669410380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6524010430 con 0x7f652c19cfa0 2024-04-23T18:26:39.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.113+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f652c1a5540 con 0x7f652c19cfa0 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.115+0000 7f651b7fe640 1 -- 172.21.15.33:0/1669410380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f6524004f90 con 0x7f652c19cfa0 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 Processor -- start 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 -- start start 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f55bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563465eb0c50 con 0x563465f4ac50 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563465e4d830 con 0x563465f56210 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6533615740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563465eacc10 con 0x563465f4b600 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f651b7fe640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 0x7f65140a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46222/0 (socket says 172.21.15.33:46222) 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.116+0000 7f6529d9b640 1 -- 172.21.15.33:0/3662530526 learned_addr learned my addr 172.21.15.33:0/3662530526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f652a59c640 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f55bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f652ad9d640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 0x7f65140a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f651b7fe640 1 -- 172.21.15.33:0/1669410380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f65240d0ee0 con 0x7f652c19cfa0 2024-04-23T18:26:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 -- 172.21.15.33:0/3662530526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 msgr2=0x563465f55bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f55bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 -- 172.21.15.33:0/3662530526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 msgr2=0x563465f56670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f56670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 -- 172.21.15.33:0/3662530526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563465bc32d0 con 0x563465f4ac50 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6529d9b640 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6520005c40 tx=0x7f6520005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3b7796d76397bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f652ad9d640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 0x7f65140a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65240ce4c0 tx=0x7f65240bd040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6518ff9640 1 -- 172.21.15.33:0/3662530526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520035050 con 0x563465f4ac50 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6518ff9640 1 -- 172.21.15.33:0/3662530526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520031620 con 0x563465f4ac50 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.117+0000 7f6518ff9640 1 -- 172.21.15.33:0/3662530526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520005580 con 0x563465f4ac50 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- 172.21.15.33:0/3662530526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 msgr2=0x563465f4b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6520005c40 tx=0x7f6520005bd0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- 172.21.15.33:0/3662530526 shutdown_connections 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f56670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f55bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 --2- 172.21.15.33:0/3662530526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f4b030 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- 172.21.15.33:0/3662530526 >> 172.21.15.33:0/3662530526 conn(0x563465f47070 msgr2=0x563465f47450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- 172.21.15.33:0/3662530526 shutdown_connections 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- 172.21.15.33:0/3662530526 wait complete. 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 -- start start 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.118+0000 7f6533615740 1 Processor -- start 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f5a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f5a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563465f5d130 con 0x563465f4ac50 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563465e4b470 con 0x563465f56210 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563465f5d2b0 con 0x563465f4b600 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 10 client.0 _ll_register_callbacks cb 0x563465e70f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46238/0 (socket says 172.21.15.33:46238) 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 -- 172.21.15.33:0/172036324 learned_addr learned my addr 172.21.15.33:0/172036324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f652a59c640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f5a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f5a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 -- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 msgr2=0x563465f5a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f5a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 -- 172.21.15.33:0/172036324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 msgr2=0x563465f5a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f5a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 -- 172.21.15.33:0/172036324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563465c8d6b0 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6529d9b640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6520005ec0 tx=0x7f6520005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96dd0186b3e0fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520035070 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f6520004260 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.119+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6520031620 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6520004460 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 10 client.15621 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563465f693d0 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520031620 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6520004880 con 0x563465f4ac50 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 10 client.15621 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 10 client.15621 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 20 client.15621 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465f6b4e0 0x563465f6c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 4 client.15621 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563465ebe670 con 0x563465f6b4e0 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.120+0000 7f652a59c640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465f6b4e0 0x563465f6c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6511ffb640 10 client.15621 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f652a59c640 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465f6b4e0 0x563465f6c950 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=574dcdaf8cbfa361 server_cookie=2bc2de475e13875b in_seq=0 out_seq=0 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6511ffb640 1 -- 172.21.15.33:0/172036324 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563465ebe670 con 0x563465f6b4e0 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6511ffb640 10 client.15621 handle_command_reply: tid=0 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 client.15621 shutdown 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 msgr2=0x563465f59c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6520005ec0 tx=0x7f6520005260 comp rx=0 tx=0).stop 2024-04-23T18:26:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 shutdown_connections 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465f6b4e0 0x563465f6c950 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f5a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f5a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 --2- 172.21.15.33:0/172036324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f59c50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6511ffb640 0 client.15621 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.121+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 >> 172.21.15.33:0/172036324 conn(0x563465f47070 msgr2=0x563465f4cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.122+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 shutdown_connections 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.122+0000 7f6533615740 1 -- 172.21.15.33:0/172036324 wait complete. 2024-04-23T18:26:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.122+0000 7f6533615740 20 client.15621 trim_cache size 0 max 16384 2024-04-23T18:26:39.126 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:38 smithi106 ceph-mon[32716]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 132 KiB/s wr, 80 op/s 2024-04-23T18:26:39.126 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:38 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2246605294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.126 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:38 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/122941712' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 Processor -- start 2024-04-23T18:26:39.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 -- start start 2024-04-23T18:26:39.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f4d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f57600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563465f6ac90 con 0x563465f4ac50 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563465f6b360 con 0x563465f56210 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6533615740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563465f6ba30 con 0x563465f4b600 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46250/0 (socket says 172.21.15.33:46250) 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 -- 172.21.15.33:0/530949267 learned_addr learned my addr 172.21.15.33:0/530949267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f652a59c640 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f4d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 -- 172.21.15.33:0/530949267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 msgr2=0x563465f4d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f4d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 -- 172.21.15.33:0/530949267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 msgr2=0x563465f57600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f57600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.140+0000 7f6529d9b640 1 -- 172.21.15.33:0/530949267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563465f5d2a0 con 0x563465f4ac50 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6529d9b640 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6520005310 tx=0x7f6520005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cf84fd330ca174 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6513fff640 1 -- 172.21.15.33:0/530949267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520035040 con 0x563465f4ac50 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6513fff640 1 -- 172.21.15.33:0/530949267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520004260 con 0x563465f4ac50 2024-04-23T18:26:39.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6513fff640 1 -- 172.21.15.33:0/530949267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200050c0 con 0x563465f4ac50 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 -- 172.21.15.33:0/530949267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 msgr2=0x563465f66af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6520005310 tx=0x7f6520005980 comp rx=0 tx=0).stop 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 -- 172.21.15.33:0/530949267 shutdown_connections 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f56210 0x563465f57600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f4d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 --2- 172.21.15.33:0/530949267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f4ac50 0x563465f66af0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 -- 172.21.15.33:0/530949267 >> 172.21.15.33:0/530949267 conn(0x563465f47070 msgr2=0x563465f55910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 -- 172.21.15.33:0/530949267 shutdown_connections 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.141+0000 7f6533615740 1 -- 172.21.15.33:0/530949267 wait complete. 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 -- start start 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 Processor -- start 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f4ac50 0x563465f4d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563465f6c7d0 con 0x563465f56210 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563465f6c100 con 0x563465f4ac50 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563465f6cea0 con 0x563465f4b600 2024-04-23T18:26:39.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6533615740 10 client.0 _ll_register_callbacks cb 0x7f651c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46264/0 (socket says 172.21.15.33:46264) 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 -- 172.21.15.33:0/917232769 learned_addr learned my addr 172.21.15.33:0/917232769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f4ac50 0x563465f4d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f652a59c640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 -- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 msgr2=0x563465f565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 -- 172.21.15.33:0/917232769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f4ac50 msgr2=0x563465f4d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f4ac50 0x563465f4d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f6529d9b640 1 -- 172.21.15.33:0/917232769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563465f57380 con 0x563465f56210 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.142+0000 7f652a59c640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6529d9b640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6520004430 tx=0x7f65200095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0272468f2f5033b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200319e0 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6520004260 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f6520031b80 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65200035f0 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 10 client.15627 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563465f74c30 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6520004260 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f652003a7f0 con 0x563465f56210 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 10 client.15627 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 10 client.15627 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-23T18:26:39.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 20 client.15627 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465fd5030 0x563465fd1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 4 client.15627 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.143+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 --> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] -- command(tid 0: {"prefix": "status"}) -- 0x563465eb84e0 con 0x563465fd5030 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f652a59c640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465fd5030 0x563465fd1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f652a59c640 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465fd5030 0x563465fd1490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e300bed47fe1a1e9 server_cookie=50314d0768ac7386 in_seq=0 out_seq=0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6511ffb640 10 client.15627 ms_handle_connect on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6511ffb640 1 -- 172.21.15.33:0/917232769 <== mds.2 v2:172.21.15.122:6838/538103378 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563465eb84e0 con 0x563465fd5030 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6511ffb640 10 client.15627 handle_command_reply: tid=0 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 client.15627 shutdown 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 msgr2=0x563465f56b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6520004430 tx=0x7f65200095e0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 shutdown_connections 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:6838/538103378,v1:172.21.15.122:6839/538103378] conn(0x563465fd5030 0x563465fd1490 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563465f56210 0x563465f56b50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563465f4b600 0x563465f565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6511ffb640 0 client.15627 ms_handle_reset on v2:172.21.15.122:6838/538103378 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 --2- 172.21.15.33:0/917232769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563465f4ac50 0x563465f4d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 >> 172.21.15.33:0/917232769 conn(0x563465f47070 msgr2=0x563465f549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 shutdown_connections 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 1 -- 172.21.15.33:0/917232769 wait complete. 2024-04-23T18:26:39.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.144+0000 7f6533615740 20 client.15627 trim_cache size 0 max 16384 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24490, 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 127.823408594, 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-04-23T18:26:39.149 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T18:26:39.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 144.861959128 2024-04-23T18:26:39.150 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:39.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 msgr2=0x7f65140a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 0x7f65140a6300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65240ce4c0 tx=0x7f65240bd040 comp rx=0 tx=0).stop 2024-04-23T18:26:39.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 msgr2=0x7f652c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f652400e970 tx=0x7f652400ed70 comp rx=0 tx=0).stop 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 shutdown_connections 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65140a3e40 0x7f65140a6300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f652c19cfa0 0x7f652c19d450 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f652c107240 0x7f652c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 --2- 172.21.15.33:0/1669410380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f652c069350 0x7f652c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 >> 172.21.15.33:0/1669410380 conn(0x7f652c077cb0 msgr2=0x7f652c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 shutdown_connections 2024-04-23T18:26:39.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.145+0000 7f65308fa640 1 -- 172.21.15.33:0/1669410380 wait complete. 2024-04-23T18:26:39.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-23T18:26:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 -- 172.21.15.33:0/3908199898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0105e70 msgr2=0x7fd4d0106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/3908199898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0105e70 0x7fd4d0106250 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009900 tx=0x7fd4c802a470 comp rx=0 tx=0).stop 2024-04-23T18:26:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 -- 172.21.15.33:0/3908199898 shutdown_connections 2024-04-23T18:26:39.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/3908199898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d0101aa0 0x7fd4d010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/3908199898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d01010e0 0x7fd4d0101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/3908199898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0105e70 0x7fd4d0106250 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 -- 172.21.15.33:0/3908199898 >> 172.21.15.33:0/3908199898 conn(0x7fd4d0077cb0 msgr2=0x7fd4d00780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 -- 172.21.15.33:0/3908199898 shutdown_connections 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.262+0000 7fd4d55a2640 1 -- 172.21.15.33:0/3908199898 wait complete. 2024-04-23T18:26:39.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 Processor -- start 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 -- start start 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 0x7fd4d0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d0105e70 0x7fd4d019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0112250 con 0x7fd4d0101aa0 2024-04-23T18:26:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d01120d0 con 0x7fd4d01010e0 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4d55a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d01123d0 con 0x7fd4d0105e70 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46280/0 (socket says 172.21.15.33:46280) 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/232926192 learned_addr learned my addr 172.21.15.33:0/232926192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d0105e70 0x7fd4d019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.263+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 0x7fd4d0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/232926192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d0105e70 msgr2=0x7fd4d019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d0105e70 0x7fd4d019cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/232926192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 msgr2=0x7fd4d0198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 0x7fd4d0198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/232926192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d019d6b0 con 0x7fd4d0101aa0 2024-04-23T18:26:39.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 0x7fd4d0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c920 tx=0x7fd4c400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4b7fff640 1 -- 172.21.15.33:0/232926192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4007c70 con 0x7fd4d0101aa0 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d019d9a0 con 0x7fd4d0101aa0 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4b7fff640 1 -- 172.21.15.33:0/232926192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c4007e10 con 0x7fd4d0101aa0 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4b7fff640 1 -- 172.21.15.33:0/232926192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4018820 con 0x7fd4d0101aa0 2024-04-23T18:26:39.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.264+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d01a5230 con 0x7fd4d0101aa0 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.266+0000 7fd4b7fff640 1 -- 172.21.15.33:0/232926192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fd4c400f450 con 0x7fd4d0101aa0 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 Processor -- start 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 -- start start 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d16190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d20800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eec9c7bc50 con 0x55eec9d20dd0 2024-04-23T18:26:39.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eec9c18830 con 0x55eec9d15db0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4d839a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eec9c77c10 con 0x55eec9d16760 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4b7fff640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 0x7fd4b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46288/0 (socket says 172.21.15.33:46288) 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/2642391099 learned_addr learned my addr 172.21.15.33:0/2642391099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d20800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d16190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 0x7fd4b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4b7fff640 1 -- 172.21.15.33:0/232926192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd4c40d0c30 con 0x7fd4d0101aa0 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/2642391099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 msgr2=0x55eec9d20800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d20800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/2642391099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 msgr2=0x55eec9d16190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d16190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/2642391099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec998e2d0 con 0x55eec9d20dd0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.267+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d20800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40ce240 tx=0x7fd4c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2fce25724f76f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 0x7fd4b80a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c0009600 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4b57fa640 1 -- 172.21.15.33:0/2642391099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c409d030 con 0x55eec9d20dd0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4b57fa640 1 -- 172.21.15.33:0/2642391099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c409c930 con 0x55eec9d20dd0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4b57fa640 1 -- 172.21.15.33:0/2642391099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40970f0 con 0x55eec9d20dd0 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4d839a740 1 -- 172.21.15.33:0/2642391099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 msgr2=0x55eec9d23220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.268+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40ce240 tx=0x7fd4c4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- 172.21.15.33:0/2642391099 shutdown_connections 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d23220 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d20800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2642391099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d16190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- 172.21.15.33:0/2642391099 >> 172.21.15.33:0/2642391099 conn(0x55eec9d12070 msgr2=0x55eec9d12450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- 172.21.15.33:0/2642391099 shutdown_connections 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- 172.21.15.33:0/2642391099 wait complete. 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- start start 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 Processor -- start 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d1e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d23d40 con 0x55eec9d16760 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eec9c16470 con 0x55eec9d15db0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d23ec0 con 0x55eec9d20dd0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4d839a740 10 client.0 _ll_register_callbacks cb 0x55eec9c3bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46302/0 (socket says 172.21.15.33:46302) 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d1e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.269+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/459502019 learned_addr learned my addr 172.21.15.33:0/459502019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/459502019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 msgr2=0x55eec9d1e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d1e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 msgr2=0x55eec9d1dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 -- 172.21.15.33:0/459502019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec9a586b0 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4099300 tx=0x7fd4c40a0dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6fca152cbc3932 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c409d030 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd4c400f740 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c409c930 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c4098fa0 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4d839a740 10 client.15639 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eec9d30010 con 0x55eec9d16760 2024-04-23T18:26:39.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.270+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c409c930 con 0x55eec9d16760 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c40bb050 con 0x55eec9d16760 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 10 client.15639 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 10 client.15639 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 20 client.15639 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d31360 0x55eec9d334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 4 client.15639 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec9c89670 con 0x55eec9d31360 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d31360 0x55eec9d334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d31360 0x55eec9d334d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6dd15369dc331388 server_cookie=1a0c2923c091acc7 in_seq=0 out_seq=0 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.271+0000 7fd4b2ff5640 10 client.15639 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/459502019 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eec9c89670 con 0x55eec9d31360 2024-04-23T18:26:39.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4b2ff5640 10 client.15639 handle_command_reply: tid=0 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 client.15639 shutdown 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 msgr2=0x55eec9d1e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4099300 tx=0x7fd4c40a0dd0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 shutdown_connections 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d31360 0x55eec9d334d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d1e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d16760 0x55eec9d1e250 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4b2ff5640 0 client.15639 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 --2- 172.21.15.33:0/459502019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 >> 172.21.15.33:0/459502019 conn(0x55eec9d12070 msgr2=0x55eec9d17ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 shutdown_connections 2024-04-23T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.272+0000 7fd4d839a740 1 -- 172.21.15.33:0/459502019 wait complete. 2024-04-23T18:26:39.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.273+0000 7fd4d839a740 20 client.15639 trim_cache size 0 max 16384 2024-04-23T18:26:39.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 Processor -- start 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 -- start start 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d31810 con 0x55eec9d20dd0 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d31ee0 con 0x55eec9d15db0 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4d839a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d325b0 con 0x55eec9d16760 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4ce7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46316/0 (socket says 172.21.15.33:46316) 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 -- 172.21.15.33:0/3349451850 learned_addr learned my addr 172.21.15.33:0/3349451850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 -- 172.21.15.33:0/3349451850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 msgr2=0x55eec9d1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d1fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 -- 172.21.15.33:0/3349451850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 msgr2=0x55eec9d1d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 -- 172.21.15.33:0/3349451850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec9d16500 con 0x55eec9d20dd0 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.291+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0012970 tx=0x7fd4c0009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc45e5eb710623f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4b4ff9640 1 -- 172.21.15.33:0/3349451850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0012e20 con 0x55eec9d20dd0 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4b4ff9640 1 -- 172.21.15.33:0/3349451850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c0022e30 con 0x55eec9d20dd0 2024-04-23T18:26:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4b4ff9640 1 -- 172.21.15.33:0/3349451850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c002ae00 con 0x55eec9d20dd0 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- 172.21.15.33:0/3349451850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 msgr2=0x55eec9d18200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0012970 tx=0x7fd4c0009a00 comp rx=0 tx=0).stop 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- 172.21.15.33:0/3349451850 shutdown_connections 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d20dd0 0x55eec9d18200 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d16760 0x55eec9d1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 --2- 172.21.15.33:0/3349451850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d15db0 0x55eec9d1d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- 172.21.15.33:0/3349451850 >> 172.21.15.33:0/3349451850 conn(0x55eec9d12070 msgr2=0x55eec9d1f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- 172.21.15.33:0/3349451850 shutdown_connections 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- 172.21.15.33:0/3349451850 wait complete. 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.292+0000 7fd4d839a740 1 -- start start 2024-04-23T18:26:39.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 Processor -- start 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d16760 0x55eec9d34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d33350 con 0x55eec9d15db0 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d32c80 con 0x55eec9d16760 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eec9d33a20 con 0x55eec9d20dd0 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4d839a740 10 client.0 _ll_register_callbacks cb 0x7fd4c0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46320/0 (socket says 172.21.15.33:46320) 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 -- 172.21.15.33:0/2663328008 learned_addr learned my addr 172.21.15.33:0/2663328008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d16760 0x55eec9d34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 -- 172.21.15.33:0/2663328008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 msgr2=0x55eec9d35360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d35360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 -- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d16760 msgr2=0x55eec9d34e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d16760 0x55eec9d34e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 -- 172.21.15.33:0/2663328008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec9d35b90 con 0x55eec9d15db0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4cf7fe640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4ceffd640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005ec0 tx=0x7fd4c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72a3d808fb2584f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c8035040 con 0x55eec9d15db0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.293+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd4c8044dc0 con 0x55eec9d15db0 2024-04-23T18:26:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4c0022e30 con 0x55eec9d15db0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c8051050 con 0x55eec9d15db0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 10 client.15645 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eec9d98010 con 0x55eec9d15db0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4c0022e30 con 0x55eec9d15db0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c804c030 con 0x55eec9d15db0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 10 client.15645 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 10 client.15645 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 20 client.15645 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d9cbc0 0x55eec9d9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 4 client.15645 mds_command: new command op to 24515 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 --> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] -- command(tid 0: {"prefix": "status"}) -- 0x55eec9c834e0 con 0x55eec9d9cbc0 2024-04-23T18:26:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.294+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d9cbc0 0x55eec9d9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4b2ff5640 10 client.15645 ms_handle_connect on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4ce7fc640 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d9cbc0 0x55eec9d9b8d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ec13549021f8c5d server_cookie=e305293dc39c1f47 in_seq=0 out_seq=0 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4b2ff5640 1 -- 172.21.15.33:0/2663328008 <== mds.1 v2:172.21.15.106:6836/319312146 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55eec9c834e0 con 0x55eec9d9cbc0 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4b2ff5640 10 client.15645 handle_command_reply: tid=0 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 client.15645 shutdown 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 msgr2=0x55eec9d3bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8005ec0 tx=0x7fd4c8004270 comp rx=0 tx=0).stop 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 shutdown_connections 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:6836/319312146,v1:172.21.15.106:6837/319312146] conn(0x55eec9d9cbc0 0x55eec9d9b8d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eec9d20dd0 0x55eec9d35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4b2ff5640 0 client.15645 ms_handle_reset on v2:172.21.15.106:6836/319312146 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eec9d16760 0x55eec9d34e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 --2- 172.21.15.33:0/2663328008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eec9d15db0 0x55eec9d3bb00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 >> 172.21.15.33:0/2663328008 conn(0x55eec9d12070 msgr2=0x55eec9d111d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 shutdown_connections 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 1 -- 172.21.15.33:0/2663328008 wait complete. 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.295+0000 7fd4d839a740 20 client.15645 trim_cache size 0 max 16384 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:39.298 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24515, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 128.981065847, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 147.78368174799999 2024-04-23T18:26:39.299 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 msgr2=0x7fd4b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 0x7fd4b80a6300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009fd0 tx=0x7fd4c0009600 comp rx=0 tx=0).stop 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 msgr2=0x7fd4d0198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c920 tx=0x7fd4c400cde0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 shutdown_connections 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4b80a3e40 0x7fd4b80a6300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4d0105e70 0x7fd4d019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4d0101aa0 0x7fd4d0198c50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 --2- 172.21.15.33:0/232926192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d01010e0 0x7fd4d0198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 >> 172.21.15.33:0/232926192 conn(0x7fd4d0077cb0 msgr2=0x7fd4d00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 shutdown_connections 2024-04-23T18:26:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.296+0000 7fd4d55a2640 1 -- 172.21.15.33:0/232926192 wait complete. 2024-04-23T18:26:39.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- 172.21.15.33:0/3575678824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 msgr2=0x7f7b90103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 --2- 172.21.15.33:0/3575678824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b90103610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009900 tx=0x7f7b8c02a480 comp rx=0 tx=0).stop 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- 172.21.15.33:0/3575678824 shutdown_connections 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 --2- 172.21.15.33:0/3575678824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b9010ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 --2- 172.21.15.33:0/3575678824 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 --2- 172.21.15.33:0/3575678824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b90103610 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- 172.21.15.33:0/3575678824 >> 172.21.15.33:0/3575678824 conn(0x7f7b90077f80 msgr2=0x7f7b90078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- 172.21.15.33:0/3575678824 shutdown_connections 2024-04-23T18:26:39.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- 172.21.15.33:0/3575678824 wait complete. 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 Processor -- start 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.412+0000 7f7b98631640 1 -- start start 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b901985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90112240 con 0x7f7b90103230 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901120c0 con 0x7f7b90103b50 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b901123c0 con 0x7f7b9010c600 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46330/0 (socket says 172.21.15.33:46330) 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 -- 172.21.15.33:0/1733478102 learned_addr learned my addr 172.21.15.33:0/1733478102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b901985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 -- 172.21.15.33:0/1733478102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 msgr2=0x7f7b901985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b901985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 -- 172.21.15.33:0/1733478102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 msgr2=0x7f7b9019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 -- 172.21.15.33:0/1733478102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b90198d40 con 0x7f7b90103230 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b901985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005940 tx=0x7f7b8c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b877fe640 1 -- 172.21.15.33:0/1733478102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c03f030 con 0x7f7b90103230 2024-04-23T18:26:39.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b90198fd0 con 0x7f7b90103230 2024-04-23T18:26:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b877fe640 1 -- 172.21.15.33:0/1733478102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c0044c0 con 0x7f7b90103230 2024-04-23T18:26:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b877fe640 1 -- 172.21.15.33:0/1733478102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c035070 con 0x7f7b90103230 2024-04-23T18:26:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.413+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b901a52d0 con 0x7f7b90103230 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b877fe640 1 -- 172.21.15.33:0/1733478102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7b8c042030 con 0x7f7b90103230 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 Processor -- start 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 -- start start 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e48190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e55220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555af3dadc50 con 0x555af3e48760 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b877fe640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555af3d4a830 con 0x555af3e52dd0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.416+0000 7f7b9b423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555af3da9c10 con 0x555af3e47db0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46334/0 (socket says 172.21.15.33:46334) 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 -- 172.21.15.33:0/3842724214 learned_addr learned my addr 172.21.15.33:0/3842724214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e55220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e48190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b877fe640 1 -- 172.21.15.33:0/1733478102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7b8c0f0e30 con 0x7f7b90103230 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 -- 172.21.15.33:0/3842724214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 msgr2=0x555af3e48190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e48190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 -- 172.21.15.33:0/3842724214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 msgr2=0x555af3e55220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e55220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 -- 172.21.15.33:0/3842724214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af3ac02d0 con 0x555af3e48760 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88009bb0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0eeb90 tx=0x7f7b8c08e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3c85bdc1f265bb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b84bfe640 1 -- 172.21.15.33:0/3842724214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c049070 con 0x555af3e48760 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b84bfe640 1 -- 172.21.15.33:0/3842724214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c087770 con 0x555af3e48760 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.417+0000 7f7b84bfe640 1 -- 172.21.15.33:0/3842724214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0e70c0 con 0x555af3e48760 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- 172.21.15.33:0/3842724214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 msgr2=0x555af3e52800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0eeb90 tx=0x7f7b8c08e6a0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- 172.21.15.33:0/3842724214 shutdown_connections 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e55220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e48760 0x555af3e52800 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3842724214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e48190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- 172.21.15.33:0/3842724214 >> 172.21.15.33:0/3842724214 conn(0x555af3e44070 msgr2=0x555af3e44450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- 172.21.15.33:0/3842724214 shutdown_connections 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- 172.21.15.33:0/3842724214 wait complete. 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.418+0000 7f7b9b423740 1 -- start start 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 Processor -- start 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555af3e55b00 con 0x555af3e52dd0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555af3d48470 con 0x555af3e48760 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555af3e55c80 con 0x555af3e47db0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b9b423740 10 client.0 _ll_register_callbacks cb 0x555af3d6df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46346/0 (socket says 172.21.15.33:46346) 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 -- 172.21.15.33:0/1152959400 learned_addr learned my addr 172.21.15.33:0/1152959400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 -- 172.21.15.33:0/1152959400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 msgr2=0x555af3e4fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 -- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 msgr2=0x555af3e501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 -- 172.21.15.33:0/1152959400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af3b8a6b0 con 0x555af3e52dd0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b96bce640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b88012f90 tx=0x7f7b88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28836a5ef56c096 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.419+0000 7f7b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88026070 con 0x555af3e52dd0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7b8801fd70 con 0x555af3e52dd0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8c087770 con 0x555af3e52dd0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b88033450 con 0x555af3e52dd0 2024-04-23T18:26:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 10 client.15657 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555af3e61c90 con 0x555af3e52dd0 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c087770 con 0x555af3e52dd0 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8801e070 con 0x555af3e52dd0 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 10 client.15657 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 10 client.15657 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 20 client.15657 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3e64390 0x555af3e65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 4 client.15657 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.420+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555af3dbb670 con 0x555af3e64390 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.421+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3e64390 0x555af3e65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.421+0000 7f7b963cd640 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3e64390 0x555af3e65800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e893aef0cd3f8b5 server_cookie=c90f5a2ef9997d50 in_seq=0 out_seq=0 2024-04-23T18:26:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.421+0000 7f7b79ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b79ffb640 1 -- 172.21.15.33:0/1152959400 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555af3dbb670 con 0x555af3e64390 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b79ffb640 10 client.15657 handle_command_reply: tid=0 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 client.15657 shutdown 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 msgr2=0x555af3e50720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b88012f90 tx=0x7f7b88009500 comp rx=0 tx=0).stop 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 shutdown_connections 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3e64390 0x555af3e65800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e50720 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b79ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 --2- 172.21.15.33:0/1152959400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 >> 172.21.15.33:0/1152959400 conn(0x555af3e44070 msgr2=0x555af3e49ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 shutdown_connections 2024-04-23T18:26:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 1 -- 172.21.15.33:0/1152959400 wait complete. 2024-04-23T18:26:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.422+0000 7f7b9b423740 20 client.15657 trim_cache size 0 max 16384 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.440+0000 7f7b9b423740 1 Processor -- start 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.440+0000 7f7b9b423740 1 -- start start 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e4e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555af3e63b40 con 0x555af3e52dd0 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555af3e64210 con 0x555af3e48760 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b9b423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555af3e648e0 con 0x555af3e47db0 2024-04-23T18:26:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46350/0 (socket says 172.21.15.33:46350) 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 -- 172.21.15.33:0/3539002011 learned_addr learned my addr 172.21.15.33:0/3539002011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 -- 172.21.15.33:0/3539002011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 msgr2=0x555af3e4a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 -- 172.21.15.33:0/3539002011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 msgr2=0x555af3e4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 -- 172.21.15.33:0/3539002011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555af3e4e1f0 con 0x555af3e52dd0 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b880095e0 tx=0x7f7b8801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5481b484cd1ca31 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3539002011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88026040 con 0x555af3e52dd0 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3539002011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8801f0e0 con 0x555af3e52dd0 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.441+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3539002011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b880335f0 con 0x555af3e52dd0 2024-04-23T18:26:39.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- 172.21.15.33:0/3539002011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 msgr2=0x555af3e4eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b880095e0 tx=0x7f7b8801e000 comp rx=0 tx=0).stop 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- 172.21.15.33:0/3539002011 shutdown_connections 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e52dd0 0x555af3e4eab0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e48760 0x555af3e4e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3539002011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e47db0 0x555af3e4a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- 172.21.15.33:0/3539002011 >> 172.21.15.33:0/3539002011 conn(0x555af3e44070 msgr2=0x555af3e3ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- 172.21.15.33:0/3539002011 shutdown_connections 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- 172.21.15.33:0/3539002011 wait complete. 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 -- start start 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.442+0000 7f7b9b423740 1 Processor -- start 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 0x555af3e4cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e6ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555af3e65680 con 0x555af3e47db0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555af3e64fb0 con 0x555af3e52dd0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555af3e65d50 con 0x555af3e48760 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 10 client.0 _ll_register_callbacks cb 0x7f7b88009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46356/0 (socket says 172.21.15.33:46356) 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 -- 172.21.15.33:0/3084743648 learned_addr learned my addr 172.21.15.33:0/3084743648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 0x555af3e4cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e6ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 -- 172.21.15.33:0/3084743648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 msgr2=0x555af3e4cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 0x555af3e4cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 -- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 msgr2=0x555af3e6ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e6ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 -- 172.21.15.33:0/3084743648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555af3e6cfc0 con 0x555af3e47db0 2024-04-23T18:26:39.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 0x555af3e4cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b96bce640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e6ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b963cd640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c032af0 tx=0x7f7b8c0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164ab434abcf8ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c049040 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7b8c0b92d0 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b8801f0e0 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b8c0c1010 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 10 client.15663 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.443+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555af3e6e280 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8801f0e0 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7b8c0eece0 con 0x555af3e47db0 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 10 client.15663 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 10 client.15663 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-23T18:26:39.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 20 client.15663 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3ece8b0 0x555af3ecd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 4 client.15663 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "status"}) -- 0x555af3db54e0 con 0x555af3ece8b0 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.444+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3ece8b0 0x555af3ecd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b95bcc640 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3ece8b0 0x555af3ecd5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c359a04cd4f2d47b server_cookie=11b66d2ff89b73b2 in_seq=0 out_seq=0 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b79ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b79ffb640 1 -- 172.21.15.33:0/3084743648 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555af3db54e0 con 0x555af3ece8b0 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b79ffb640 10 client.15663 handle_command_reply: tid=0 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 client.15663 shutdown 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 msgr2=0x555af3e4c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c032af0 tx=0x7f7b8c0dd040 comp rx=0 tx=0).stop 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 shutdown_connections 2024-04-23T18:26:39.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555af3ece8b0 0x555af3ecd5c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555af3e52dd0 0x555af3e6ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555af3e48760 0x555af3e4cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 --2- 172.21.15.33:0/3084743648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555af3e47db0 0x555af3e4c700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b79ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 >> 172.21.15.33:0/3084743648 conn(0x555af3e44070 msgr2=0x555af3e4e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 shutdown_connections 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 1 -- 172.21.15.33:0/3084743648 wait complete. 2024-04-23T18:26:39.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.445+0000 7f7b9b423740 20 client.15663 trim_cache size 0 max 16384 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "766ecab6-019d-11ef-bc93-c7b262605968", 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24551, 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 132.164672658, 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 28, 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-04-23T18:26:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 87, 2024-04-23T18:26:39.449 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 136.29378875099999 2024-04-23T18:26:39.449 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 msgr2=0x7f7b7c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b88005fd0 tx=0x7f7b88009bb0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 msgr2=0x7f7b9019e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005940 tx=0x7f7b8c0040c0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 shutdown_connections 2024-04-23T18:26:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b9010c600 0x7f7b901985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b90103b50 0x7f7b9019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 --2- 172.21.15.33:0/1733478102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b90103230 0x7f7b9019e410 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 >> 172.21.15.33:0/1733478102 conn(0x7f7b90077f80 msgr2=0x7f7b900fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 shutdown_connections 2024-04-23T18:26:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:39.446+0000 7f7b98631640 1 -- 172.21.15.33:0/1733478102 wait complete. 2024-04-23T18:26:39.451 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-23T18:26:39.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:39 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1727675615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:39.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:39 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1727675615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:39 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1727675615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:40.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:40 smithi122 ceph-mon[32546]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:40.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:40 smithi033 ceph-mon[29274]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:40 smithi106 ceph-mon[32716]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 102 KiB/s wr, 62 op/s 2024-04-23T18:26:42.949 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:42 smithi122 ceph-mon[32546]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:26:42.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:42 smithi033 ceph-mon[29274]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:26:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:42 smithi106 ceph-mon[32716]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 56 op/s 2024-04-23T18:26:44.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:44 smithi122 ceph-mon[32546]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:44.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:44 smithi033 ceph-mon[29274]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:45.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:44 smithi106 ceph-mon[32716]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 51 op/s 2024-04-23T18:26:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 -- 172.21.15.33:0/2926162480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50201080e0 msgr2=0x7f50201084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 --2- 172.21.15.33:0/2926162480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50201080e0 0x7f50201084c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5010009900 tx=0x7f501002a480 comp rx=0 tx=0).stop 2024-04-23T18:26:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 -- 172.21.15.33:0/2926162480 shutdown_connections 2024-04-23T18:26:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 --2- 172.21.15.33:0/2926162480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f502006b120 0x7f502010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 --2- 172.21.15.33:0/2926162480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006a780 0x7f502006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 --2- 172.21.15.33:0/2926162480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50201080e0 0x7f50201084c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 -- 172.21.15.33:0/2926162480 >> 172.21.15.33:0/2926162480 conn(0x7f5020076c50 msgr2=0x7f5020077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 -- 172.21.15.33:0/2926162480 shutdown_connections 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.402+0000 7f50286e1640 1 -- 172.21.15.33:0/2926162480 wait complete. 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 Processor -- start 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 -- start start 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006b120 0x7f50201017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50201080e0 0x7f5020101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5020112190 con 0x7f502006a780 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5020112010 con 0x7f502006b120 2024-04-23T18:26:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f50286e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5020112310 con 0x7f50201080e0 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46374/0 (socket says 172.21.15.33:46374) 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 -- 172.21.15.33:0/4116603668 learned_addr learned my addr 172.21.15.33:0/4116603668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f5026c7e640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50201080e0 0x7f5020101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f5025c7c640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006b120 0x7f50201017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 -- 172.21.15.33:0/4116603668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50201080e0 msgr2=0x7f5020101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50201080e0 0x7f5020101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 -- 172.21.15.33:0/4116603668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006b120 msgr2=0x7f50201017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006b120 0x7f50201017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.403+0000 7f502647d640 1 -- 172.21.15.33:0/4116603668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50201a6b10 con 0x7f502006a780 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f502647d640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5010005940 tx=0x7f50100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f50177fe640 1 -- 172.21.15.33:0/4116603668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501003f030 con 0x7f502006a780 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f50177fe640 1 -- 172.21.15.33:0/4116603668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5010003cb0 con 0x7f502006a780 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50201a6da0 con 0x7f502006a780 2024-04-23T18:26:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f50177fe640 1 -- 172.21.15.33:0/4116603668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010035070 con 0x7f502006a780 2024-04-23T18:26:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.404+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201a7280 con 0x7f502006a780 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.406+0000 7f50177fe640 1 -- 172.21.15.33:0/4116603668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f5010042030 con 0x7f502006a780 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.407+0000 7f50177fe640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 0x7f500c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.407+0000 7f5025c7c640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 0x7f500c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.407+0000 7f50177fe640 1 -- 172.21.15.33:0/4116603668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f50100f0e30 con 0x7f502006a780 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.407+0000 7f502b3fc740 1 Processor -- start 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.407+0000 7f5025c7c640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 0x7f500c0a6420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f501c005fd0 tx=0x7f501c005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 -- start start 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3da1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557c3cf9970 con 0x5557c3d94620 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5557c3c964a0 con 0x5557c3d9f800 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502b3fc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557c3cf5930 con 0x5557c3d94f40 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46384/0 (socket says 172.21.15.33:46384) 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 -- 172.21.15.33:0/823209801 learned_addr learned my addr 172.21.15.33:0/823209801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f5026c7e640 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3da1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f5025c7c640 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 -- 172.21.15.33:0/823209801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 msgr2=0x5557c3d9f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 -- 172.21.15.33:0/823209801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 msgr2=0x5557c3da1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3da1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 -- 172.21.15.33:0/823209801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c3a0c210 con 0x5557c3d94620 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f502647d640 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50100b9630 tx=0x7f5010032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843063c4f90cf915 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.408+0000 7f5014ff9640 1 -- 172.21.15.33:0/823209801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010049040 con 0x5557c3d94620 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f5014ff9640 1 -- 172.21.15.33:0/823209801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50100bd360 con 0x5557c3d94620 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f5014ff9640 1 -- 172.21.15.33:0/823209801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50100dd0c0 con 0x5557c3d94620 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 -- 172.21.15.33:0/823209801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 msgr2=0x5557c3d94a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f50100b9630 tx=0x7f5010032f60 comp rx=0 tx=0).stop 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 -- 172.21.15.33:0/823209801 shutdown_connections 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3da1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 --2- 172.21.15.33:0/823209801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d94a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.409+0000 7f502b3fc740 1 -- 172.21.15.33:0/823209801 >> 172.21.15.33:0/823209801 conn(0x5557c3d90b30 msgr2=0x5557c3d90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- 172.21.15.33:0/823209801 shutdown_connections 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- 172.21.15.33:0/823209801 wait complete. 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- start start 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 Processor -- start 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3d9c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5557c3da21f0 con 0x5557c3d94620 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5557c3c940b0 con 0x5557c3d9f800 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557c3da2370 con 0x5557c3d94f40 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502b3fc740 10 client.0 _ll_register_callbacks cb 0x5557c3c81d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46396/0 (socket says 172.21.15.33:46396) 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f502647d640 1 -- 172.21.15.33:0/2449607930 learned_addr learned my addr 172.21.15.33:0/2449607930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.410+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5026c7e640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3d9c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 -- 172.21.15.33:0/2449607930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 msgr2=0x5557c3d9c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 -- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 msgr2=0x5557c3d9c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3d9c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 -- 172.21.15.33:0/2449607930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c3ad8620 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502647d640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f50100c53a0 tx=0x7f5010009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6b5ba8a6f265d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010049040 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f50100bd0d0 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50100bd360 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50100bcc60 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502b3fc740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557c3da21f0 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.411+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50100bd360 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50100eece0 con 0x5557c3d94620 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 20 client.15675 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3daf230 0x5557c3db1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 4 client.15675 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c3d07390 con 0x5557c3daf230 2024-04-23T18:26:45.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.412+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3daf230 0x5557c3db1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3daf230 0x5557c3db1620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cc6c11b065e653 server_cookie=600ce0b95ab94b3e in_seq=0 out_seq=0 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f5009ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f5009ffb640 1 -- 172.21.15.33:0/2449607930 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c3d07390 con 0x5557c3daf230 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f5009ffb640 10 client.15675 handle_command_reply: tid=0 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f502b3fc740 1 client.15675 shutdown 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 msgr2=0x5557c3d9f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.413+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f50100c53a0 tx=0x7f5010009680 comp rx=0 tx=0).stop 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 shutdown_connections 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3daf230 0x5557c3db1620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d9f800 0x5557c3d9c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d94f40 0x5557c3d9c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f5009ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2449607930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3d9f130 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 >> 172.21.15.33:0/2449607930 conn(0x5557c3d90b30 msgr2=0x5557c3d96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 shutdown_connections 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 1 -- 172.21.15.33:0/2449607930 wait complete. 2024-04-23T18:26:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.414+0000 7f502b3fc740 20 client.15675 trim_cache size 0 max 16384 2024-04-23T18:26:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 Processor -- start 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 -- start start 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 0x5557c3daafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3d9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50201afc90 con 0x5557c3d94f40 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50201b17d0 con 0x5557c3d94620 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502b3fc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50201b0360 con 0x5557c3d9f800 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46412/0 (socket says 172.21.15.33:46412) 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 -- 172.21.15.33:0/2382575664 learned_addr learned my addr 172.21.15.33:0/2382575664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5026c7e640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3d9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502647d640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 0x5557c3daafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 -- 172.21.15.33:0/2382575664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 msgr2=0x5557c3d9ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3d9ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 -- 172.21.15.33:0/2382575664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 msgr2=0x5557c3daafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 0x5557c3daafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 -- 172.21.15.33:0/2382575664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c3d94d70 con 0x5557c3d94f40 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f502647d640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 0x5557c3daafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.430+0000 7f5025c7c640 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f501c014040 tx=0x7f501c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5fd9ac56198547 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f500bfff640 1 -- 172.21.15.33:0/2382575664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c02d070 con 0x5557c3d94f40 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f500bfff640 1 -- 172.21.15.33:0/2382575664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c0277f0 con 0x5557c3d94f40 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f500bfff640 1 -- 172.21.15.33:0/2382575664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c01eb00 con 0x5557c3d94f40 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 -- 172.21.15.33:0/2382575664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 msgr2=0x5557c3d97140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f501c014040 tx=0x7f501c015000 comp rx=0 tx=0).stop 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 -- 172.21.15.33:0/2382575664 shutdown_connections 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3d9ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94f40 0x5557c3d97140 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 --2- 172.21.15.33:0/2382575664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94620 0x5557c3daafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 -- 172.21.15.33:0/2382575664 >> 172.21.15.33:0/2382575664 conn(0x5557c3d90b30 msgr2=0x5557c3d96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 -- 172.21.15.33:0/2382575664 shutdown_connections 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.431+0000 7f502b3fc740 1 -- 172.21.15.33:0/2382575664 wait complete. 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 -- start start 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 Processor -- start 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94f40 0x5557c3da2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3da3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50201b1100 con 0x5557c3d94620 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50201b0a30 con 0x5557c3d94f40 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50201adc70 con 0x5557c3d9f800 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502b3fc740 10 client.0 _ll_register_callbacks cb 0x7f50100bca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502647d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46420/0 (socket says 172.21.15.33:46420) 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f502647d640 1 -- 172.21.15.33:0/3591216980 learned_addr learned my addr 172.21.15.33:0/3591216980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f5025c7c640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94f40 0x5557c3da2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.432+0000 7f5026c7e640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3da3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 -- 172.21.15.33:0/3591216980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 msgr2=0x5557c3da3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3da3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 -- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94f40 msgr2=0x5557c3da2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94f40 0x5557c3da2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 -- 172.21.15.33:0/3591216980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c3da38b0 con 0x5557c3d94620 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502647d640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5010003fd0 tx=0x7f50100bb170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863b60312f653665 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010049070 con 0x5557c3d94620 2024-04-23T18:26:45.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5010003a00 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f501c0277f0 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50100bd360 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 10 client.15681 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5557c3da38b0 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501c0277f0 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50100eece0 con 0x5557c3d94620 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 10 client.15681 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 20 client.15681 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3e158d0 0x5557c3e17cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 4 client.15681 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.433+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557c3d01200 con 0x5557c3e158d0 2024-04-23T18:26:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.434+0000 7f5025c7c640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3e158d0 0x5557c3e17cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.434+0000 7f5009ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.434+0000 7f5025c7c640 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3e158d0 0x5557c3e17cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b304e33cbfb4ab74 server_cookie=c03fcdb6b8b92ff1 in_seq=0 out_seq=0 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f5009ffb640 1 -- 172.21.15.33:0/3591216980 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557c3d01200 con 0x5557c3e158d0 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f5009ffb640 10 client.15681 handle_command_reply: tid=0 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 client.15681 shutdown 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 msgr2=0x5557c3da7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5010003fd0 tx=0x7f50100bb170 comp rx=0 tx=0).stop 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 shutdown_connections 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5557c3e158d0 0x5557c3e17cc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557c3d9f800 0x5557c3da3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f5009ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5557c3d94f40 0x5557c3da2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 --2- 172.21.15.33:0/3591216980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5557c3d94620 0x5557c3da7800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 >> 172.21.15.33:0/3591216980 conn(0x5557c3d90b30 msgr2=0x5557c3d8f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 shutdown_connections 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.446+0000 7f502b3fc740 1 -- 172.21.15.33:0/3591216980 wait complete. 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f502b3fc740 20 client.15681 trim_cache size 0 max 16384 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:26:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 msgr2=0x7f500c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 0x7f500c0a6420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f501c005fd0 tx=0x7f501c005850 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 msgr2=0x7f5020101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5010005940 tx=0x7f50100042f0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 shutdown_connections 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f500c0a3f60 0x7f500c0a6420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50201080e0 0x7f5020101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f502006b120 0x7f50201017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 --2- 172.21.15.33:0/4116603668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502006a780 0x7f5020101270 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 >> 172.21.15.33:0/4116603668 conn(0x7f5020076c50 msgr2=0x7f50200ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 shutdown_connections 2024-04-23T18:26:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:45.447+0000 7f50286e1640 1 -- 172.21.15.33:0/4116603668 wait complete. 2024-04-23T18:26:46.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:46 smithi122 ceph-mon[32546]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-23T18:26:46.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:46 smithi033 ceph-mon[29274]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-23T18:26:47.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:46 smithi106 ceph-mon[32716]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-04-23T18:26:48.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:48 smithi122 ceph-mon[32546]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:48.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:48 smithi033 ceph-mon[29274]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:49.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:48 smithi106 ceph-mon[32716]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 79 KiB/s wr, 59 op/s 2024-04-23T18:26:50.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:50 smithi122 ceph-mon[32546]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-23T18:26:50.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:50 smithi033 ceph-mon[29274]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-23T18:26:51.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:50 smithi106 ceph-mon[32716]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-04-23T18:26:52.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:52 smithi122 ceph-mon[32546]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-04-23T18:26:52.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:52 smithi033 ceph-mon[29274]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-04-23T18:26:53.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:52 smithi106 ceph-mon[32716]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.6 KiB/s wr, 17 op/s 2024-04-23T18:26:54.452 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:26:54.452 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.65 2024-04-23T18:26:54.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 -- 172.21.15.33:0/2077091631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c102b50 msgr2=0x7f048c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 --2- 172.21.15.33:0/2077091631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c102b50 0x7f048c106b90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0478009900 tx=0x7f047802a410 comp rx=0 tx=0).stop 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 -- 172.21.15.33:0/2077091631 shutdown_connections 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 --2- 172.21.15.33:0/2077091631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c107270 0x7f048c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 --2- 172.21.15.33:0/2077091631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c102b50 0x7f048c106b90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 --2- 172.21.15.33:0/2077091631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c1021a0 0x7f048c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 -- 172.21.15.33:0/2077091631 >> 172.21.15.33:0/2077091631 conn(0x7f048c079350 msgr2=0x7f048c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 -- 172.21.15.33:0/2077091631 shutdown_connections 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.566+0000 7f0492b5d640 1 -- 172.21.15.33:0/2077091631 wait complete. 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 Processor -- start 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 -- start start 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c1021a0 0x7f048c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c19a7d0 0x7f048c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1121b0 con 0x7f048c107270 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112030 con 0x7f048c1021a0 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f0492b5d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112330 con 0x7f048c19a7d0 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f048bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f048bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37788/0 (socket says 172.21.15.33:37788) 2024-04-23T18:26:54.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f048bfff640 1 -- 172.21.15.33:0/1200768544 learned_addr learned my addr 172.21.15.33:0/1200768544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f04908f9640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c1021a0 0x7f048c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.567+0000 7f04910fa640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c19a7d0 0x7f048c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 -- 172.21.15.33:0/1200768544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c19a7d0 msgr2=0x7f048c19ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c19a7d0 0x7f048c19ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 -- 172.21.15.33:0/1200768544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c1021a0 msgr2=0x7f048c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c1021a0 0x7f048c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 -- 172.21.15.33:0/1200768544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c19b360 con 0x7f048c107270 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f048bfff640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0478006010 tx=0x7f0478004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047803f030 con 0x7f048c107270 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0478003aa0 con 0x7f048c107270 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c19b5f0 con 0x7f048c107270 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478035070 con 0x7f048c107270 2024-04-23T18:26:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.568+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a77a0 con 0x7f048c107270 2024-04-23T18:26:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.570+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c103670 con 0x7f048c107270 2024-04-23T18:26:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.571+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0478042030 con 0x7f048c107270 2024-04-23T18:26:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.571+0000 7f0489ffb640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 0x7f047c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.572+0000 7f04908f9640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 0x7f047c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.572+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f04780f0d10 con 0x7f048c107270 2024-04-23T18:26:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.572+0000 7f04908f9640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 0x7f047c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0480009bd0 tx=0x7f0480005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.577+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04780bcbf0 con 0x7f048c107270 2024-04-23T18:26:54.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:54 smithi033 ceph-mon[29274]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:26:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.812+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f048c102b50 con 0x7f048c107270 2024-04-23T18:26:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.813+0000 7f0489ffb640 1 -- 172.21.15.33:0/1200768544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f04780c0ef0 con 0x7f048c107270 2024-04-23T18:26:54.806 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:26:54.806 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:26:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:26:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 msgr2=0x7f047c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 0x7f047c0a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0480009bd0 tx=0x7f0480005f90 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 msgr2=0x7f048c1a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0478006010 tx=0x7f0478004060 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 shutdown_connections 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f047c0a3e40 0x7f047c0a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f048c19a7d0 0x7f048c19ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c107270 0x7f048c1a0ba0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 --2- 172.21.15.33:0/1200768544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f048c1021a0 0x7f048c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 >> 172.21.15.33:0/1200768544 conn(0x7f048c079350 msgr2=0x7f048c100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 shutdown_connections 2024-04-23T18:26:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.816+0000 7f0492b5d640 1 -- 172.21.15.33:0/1200768544 wait complete. 2024-04-23T18:26:54.820 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 81.65 --await-for 42.0' 2024-04-23T18:26:54.820 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 81.65 --await-for 42.0 2024-04-23T18:26:54.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:54 smithi122 ceph-mon[32546]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 -- 172.21.15.33:0/3987639153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 msgr2=0x7fc4a810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3987639153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a810ee40 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009ca0 tx=0x7fc4a002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 -- 172.21.15.33:0/3987639153 shutdown_connections 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3987639153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a810ee40 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3987639153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a8102b50 0x7fc4a8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3987639153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 0x7fc4a8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 -- 172.21.15.33:0/3987639153 >> 172.21.15.33:0/3987639153 conn(0x7fc4a8079350 msgr2=0x7fc4a8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 -- 172.21.15.33:0/3987639153 shutdown_connections 2024-04-23T18:26:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.966+0000 7fc4ad672640 1 -- 172.21.15.33:0/3987639153 wait complete. 2024-04-23T18:26:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 Processor -- start 2024-04-23T18:26:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 -- start start 2024-04-23T18:26:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 0x7fc4a8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a819cff0 0x7fc4a819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81121e0 con 0x7fc4a8107270 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8112060 con 0x7fc4a81021a0 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4ad672640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8112360 con 0x7fc4a819cff0 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37802/0 (socket says 172.21.15.33:37802) 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4a67fc640 1 -- 172.21.15.33:0/3674037741 learned_addr learned my addr 172.21.15.33:0/3674037741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 0x7fc4a8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.967+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a819cff0 0x7fc4a819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 -- 172.21.15.33:0/3674037741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a819cff0 msgr2=0x7fc4a819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a819cff0 0x7fc4a819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 -- 172.21.15.33:0/3674037741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 msgr2=0x7fc4a8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 0x7fc4a8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 -- 172.21.15.33:0/3674037741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a819dae0 con 0x7fc4a8107270 2024-04-23T18:26:54.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d740 tx=0x7fc49c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc48ffff640 1 -- 172.21.15.33:0/3674037741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c017070 con 0x7fc4a8107270 2024-04-23T18:26:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a819ddd0 con 0x7fc4a8107270 2024-04-23T18:26:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc48ffff640 1 -- 172.21.15.33:0/3674037741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc49c00de50 con 0x7fc4a8107270 2024-04-23T18:26:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc48ffff640 1 -- 172.21.15.33:0/3674037741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c004ce0 con 0x7fc4a8107270 2024-04-23T18:26:54.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.968+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a55c0 con 0x7fc4a8107270 2024-04-23T18:26:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.971+0000 7fc48ffff640 1 -- 172.21.15.33:0/3674037741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fc49c004e80 con 0x7fc4a8107270 2024-04-23T18:26:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.971+0000 7fc4b0387740 1 Processor -- start 2024-04-23T18:26:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.971+0000 7fc48ffff640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 0x7fc4900a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.971+0000 7fc4b0387740 1 -- start start 2024-04-23T18:26:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc48ffff640 1 -- 172.21.15.33:0/3674037741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc49c0d02f0 con 0x7fc4a8107270 2024-04-23T18:26:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e484f0 0x55c105e48970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c105d9f370 con 0x55c105e43bd0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 0x7fc4900a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c105d3c2e0 con 0x55c105e484f0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4b0387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c105d9b330 con 0x55c105e44580 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37804/0 (socket says 172.21.15.33:37804) 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 -- 172.21.15.33:0/1493631191 learned_addr learned my addr 172.21.15.33:0/1493631191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e47ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e484f0 0x55c105e48970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 -- 172.21.15.33:0/1493631191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 msgr2=0x55c105e47ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e47ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 -- 172.21.15.33:0/1493631191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e484f0 msgr2=0x55c105e48970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e484f0 0x55c105e48970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 -- 172.21.15.33:0/1493631191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c105b7dd80 con 0x55c105e43bd0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 0x7fc4900a6400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006010 tx=0x7fc4a0005fa0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.972+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc498009cf0 tx=0x7fc498011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6547dcd90cfe1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc48d7fa640 1 -- 172.21.15.33:0/1493631191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498004230 con 0x55c105e43bd0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc48d7fa640 1 -- 172.21.15.33:0/1493631191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980043d0 con 0x55c105e43bd0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc48d7fa640 1 -- 172.21.15.33:0/1493631191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498019880 con 0x55c105e43bd0 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 -- 172.21.15.33:0/1493631191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 msgr2=0x55c105e43fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc498009cf0 tx=0x7fc498011810 comp rx=0 tx=0).stop 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 -- 172.21.15.33:0/1493631191 shutdown_connections 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e484f0 0x55c105e48970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1493631191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e43bd0 0x55c105e43fb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 -- 172.21.15.33:0/1493631191 >> 172.21.15.33:0/1493631191 conn(0x55c105e35e70 msgr2=0x55c105e36250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 -- 172.21.15.33:0/1493631191 shutdown_connections 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.973+0000 7fc4b0387740 1 -- 172.21.15.33:0/1493631191 wait complete. 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 -- start start 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 Processor -- start 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c105e506a0 con 0x55c105e484f0 2024-04-23T18:26:54.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c105d39fb0 con 0x55c105e43bd0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c105e50820 con 0x55c105e44580 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37820/0 (socket says 172.21.15.33:37820) 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1315006057 learned_addr learned my addr 172.21.15.33:0/1315006057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4b0387740 10 client.0 _ll_register_callbacks cb 0x55c105d399a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.974+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1315006057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 msgr2=0x55c105e4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 msgr2=0x55c105e4d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e4d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1315006057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c105bde510 con 0x55c105e484f0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0031fc0 tx=0x7fc4a002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5c9e7fc3e316c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc48aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0042040 con 0x55c105e484f0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc4a0004040 con 0x55c105e484f0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.975+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4980043d0 con 0x55c105e484f0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.976+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0003e40 con 0x55c105e484f0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.976+0000 7fc4b0387740 10 client.15699 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.976+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c105a71670 con 0x55c105e484f0 2024-04-23T18:26:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.976+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980043d0 con 0x55c105e484f0 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4a002fc30 con 0x55c105e484f0 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 10 client.15699 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 10 client.15699 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 20 client.15699 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105e5dbe0 0x55c105e60010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 4 client.15699 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c105dacd90 con 0x55c105e5dbe0 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105e5dbe0 0x55c105e60010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105e5dbe0 0x55c105e60010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983a7ed9efd5e90 server_cookie=51c7930bc3f0fff4 in_seq=0 out_seq=0 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.977+0000 7fc48aff5640 10 client.15699 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.978+0000 7fc48aff5640 1 -- 172.21.15.33:0/1315006057 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c105dacd90 con 0x55c105e5dbe0 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.978+0000 7fc48aff5640 10 client.15699 handle_command_reply: tid=0 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.978+0000 7fc4b0387740 1 client.15699 shutdown 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.978+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 msgr2=0x55c105e4dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.978+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0031fc0 tx=0x7fc4a002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 shutdown_connections 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105e5dbe0 0x55c105e60010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4dc00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1315006057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e4d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 >> 172.21.15.33:0/1315006057 conn(0x55c105e35e70 msgr2=0x55c105e43260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc48aff5640 0 client.15699 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 shutdown_connections 2024-04-23T18:26:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 1 -- 172.21.15.33:0/1315006057 wait complete. 2024-04-23T18:26:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:54.979+0000 7fc4b0387740 20 client.15699 trim_cache size 0 max 16384 2024-04-23T18:26:55.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.012+0000 7fc4b0387740 1 Processor -- start 2024-04-23T18:26:55.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.012+0000 7fc4b0387740 1 -- start start 2024-04-23T18:26:55.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e43bd0 0x55c105e59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 0x55c105e4ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81aba10 con 0x55c105e484f0 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ad550 con 0x55c105e44580 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4b0387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ac0e0 con 0x55c105e43bd0 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37830/0 (socket says 172.21.15.33:37830) 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1786388588 learned_addr learned my addr 172.21.15.33:0/1786388588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e43bd0 0x55c105e59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 0x55c105e4ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1786388588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e43bd0 msgr2=0x55c105e59560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e43bd0 0x55c105e59560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1786388588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 msgr2=0x55c105e4ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 0x55c105e4ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 -- 172.21.15.33:0/1786388588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c105e35450 con 0x55c105e484f0 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 0x55c105e4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.013+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005e30 tx=0x7fc4a00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f30c71859fd7501 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.014+0000 7fc48cff9640 1 -- 172.21.15.33:0/1786388588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0042040 con 0x55c105e484f0 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.014+0000 7fc48cff9640 1 -- 172.21.15.33:0/1786388588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a005f020 con 0x55c105e484f0 2024-04-23T18:26:55.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.014+0000 7fc48cff9640 1 -- 172.21.15.33:0/1786388588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a002fd70 con 0x55c105e484f0 2024-04-23T18:26:55.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- 172.21.15.33:0/1786388588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 msgr2=0x55c105e4bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005e30 tx=0x7fc4a00095c0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- 172.21.15.33:0/1786388588 shutdown_connections 2024-04-23T18:26:55.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e4bfd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e44580 0x55c105e4ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- 172.21.15.33:0/1786388588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e43bd0 0x55c105e59560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- 172.21.15.33:0/1786388588 >> 172.21.15.33:0/1786388588 conn(0x55c105e35e70 msgr2=0x55c105e47820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- 172.21.15.33:0/1786388588 shutdown_connections 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- 172.21.15.33:0/1786388588 wait complete. 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- start start 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 Processor -- start 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e65d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e55a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ace80 con 0x55c105e484f0 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ac7b0 con 0x55c105e43bd0 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.015+0000 7fc4b0387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81a99f0 con 0x55c105e44580 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4b0387740 10 client.0 _ll_register_callbacks cb 0x7fc4a0031ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:55.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37842/0 (socket says 172.21.15.33:37842) 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 -- 172.21.15.33:0/145252647 learned_addr learned my addr 172.21.15.33:0/145252647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a67fc640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e55a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e65d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 -- 172.21.15.33:0/145252647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 msgr2=0x55c105e55a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e55a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 -- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 msgr2=0x55c105e65d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e65d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 -- 172.21.15.33:0/145252647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c105e64b90 con 0x55c105e484f0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4a77fe640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0061040 tx=0x7fc4a00309b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1abbc705d151f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc48aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0068070 con 0x55c105e484f0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc4a0030d60 con 0x55c105e484f0 2024-04-23T18:26:55.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.016+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4a005f020 con 0x55c105e484f0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc4a0074050 con 0x55c105e484f0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c105ec46f0 con 0x55c105e484f0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4a004d070 con 0x55c105e484f0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc4a00041e0 con 0x55c105e484f0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 10 client.15705 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 20 client.15705 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105ec4d40 0x55c105ec7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 4 client.15705 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.65, "await_for": 42.0}] 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.017+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.65, "await_for": 42.0}) -- 0x7fc4a005f020 con 0x55c105ec4d40 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.018+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105ec4d40 0x55c105ec7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:55.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.018+0000 7fc4a6ffd640 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105ec4d40 0x55c105ec7160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaa2d91aa82ed26 server_cookie=5c616eb88fd6a26 in_seq=0 out_seq=0 2024-04-23T18:26:55.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.018+0000 7fc48aff5640 10 client.15705 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.032+0000 7fc48aff5640 1 -- 172.21.15.33:0/145252647 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7fc4a005f020 con 0x55c105ec4d40 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.032+0000 7fc48aff5640 10 client.15705 handle_command_reply: tid=0 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.032+0000 7fc4b0387740 1 client.15705 shutdown 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.032+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 msgr2=0x55c105e55fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0061040 tx=0x7fc4a00309b0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 shutdown_connections 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c105ec4d40 0x55c105ec7160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c105e484f0 0x55c105e55fd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc48aff5640 0 client.15705 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c105e44580 0x55c105e55a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 --2- 172.21.15.33:0/145252647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c105e43bd0 0x55c105e65d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 >> 172.21.15.33:0/145252647 conn(0x55c105e35e70 msgr2=0x55c105e4ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:55.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 shutdown_connections 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 1 -- 172.21.15.33:0/145252647 wait complete. 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4b0387740 20 client.15705 trim_cache size 0 max 16384 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 28, 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 2, 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "4f4e66b4": { 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 2, 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:26:55.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 81.7, 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:26:55.028 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:26:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 msgr2=0x7fc4900a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 0x7fc4900a6400 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006010 tx=0x7fc4a0005fa0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 msgr2=0x7fc4a8198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.033+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d740 tx=0x7fc49c00dc00 comp rx=0 tx=0).stop 2024-04-23T18:26:55.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 shutdown_connections 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc4900a3f40 0x7fc4900a6400 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a819cff0 0x7fc4a819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4a8107270 0x7fc4a8198cc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 --2- 172.21.15.33:0/3674037741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4a81021a0 0x7fc4a8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 >> 172.21.15.33:0/3674037741 conn(0x7fc4a8079350 msgr2=0x7fc4a8100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 shutdown_connections 2024-04-23T18:26:55.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:26:55.034+0000 7fc4ad672640 1 -- 172.21.15.33:0/3674037741 wait complete. 2024-04-23T18:26:55.041 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4f4e66b4', quiesce took 0.6 seconds. Will release after: 21.65 2024-04-23T18:26:55.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:54 smithi106 ceph-mon[32716]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:26:55.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:55 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1200768544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:55.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:55 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1200768544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:56.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:55 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1200768544' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:26:56.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:56 smithi122 ceph-mon[32546]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:26:56.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:56 smithi033 ceph-mon[29274]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:26:57.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:56 smithi106 ceph-mon[32716]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-23T18:26:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- 172.21.15.33:0/3842129905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f301010f0 msgr2=0x7f3f301014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 --2- 172.21.15.33:0/3842129905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f301010f0 0x7f3f301014d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009900 tx=0x7f3f2c02a410 comp rx=0 tx=0).stop 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- 172.21.15.33:0/3842129905 shutdown_connections 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 --2- 172.21.15.33:0/3842129905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f30106170 0x7f3f3010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 --2- 172.21.15.33:0/3842129905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f30101aa0 0x7f3f30105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 --2- 172.21.15.33:0/3842129905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f301010f0 0x7f3f301014d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- 172.21.15.33:0/3842129905 >> 172.21.15.33:0/3842129905 conn(0x7f3f30077fd0 msgr2=0x7f3f300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- 172.21.15.33:0/3842129905 shutdown_connections 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- 172.21.15.33:0/3842129905 wait complete. 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 Processor -- start 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.539+0000 7f3f387e2640 1 -- start start 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 0x7f3f301a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3019a790 0x7f3f3019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301110b0 con 0x7f3f30106170 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30110f30 con 0x7f3f3019a790 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f387e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f30111230 con 0x7f3f301010f0 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38326/0 (socket says 172.21.15.33:38326) 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1536381835 learned_addr learned my addr 172.21.15.33:0/1536381835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3019a790 0x7f3f3019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f3657e640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 0x7f3f301a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1536381835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 msgr2=0x7f3f301a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 0x7f3f301a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1536381835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3019a790 msgr2=0x7f3f3019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3019a790 0x7f3f3019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1536381835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3019b320 con 0x7f3f30106170 2024-04-23T18:26:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f3657e640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 0x7f3f301a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.540+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005fd0 tx=0x7f3f2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.541+0000 7f3f277fe640 1 -- 172.21.15.33:0/1536381835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20017070 con 0x7f3f30106170 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.541+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3019b610 con 0x7f3f30106170 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.541+0000 7f3f277fe640 1 -- 172.21.15.33:0/1536381835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f20009c30 con 0x7f3f30106170 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.541+0000 7f3f277fe640 1 -- 172.21.15.33:0/1536381835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20003d30 con 0x7f3f30106170 2024-04-23T18:26:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.541+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f301a7840 con 0x7f3f30106170 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.544+0000 7f3f277fe640 1 -- 172.21.15.33:0/1536381835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3f20003ed0 con 0x7f3f30106170 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.544+0000 7f3f3b5d3740 1 Processor -- start 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.544+0000 7f3f3b5d3740 1 -- start start 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562917123970 con 0x5629171d2010 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5629170c04a0 con 0x5629171c6e30 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56291711f930 con 0x5629171c7750 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f277fe640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 0x7f3f1c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f36d7f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 0x7f3f1c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f277fe640 1 -- 172.21.15.33:0/1536381835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3f200d0440 con 0x7f3f30106170 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38328/0 (socket says 172.21.15.33:38328) 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3477398161 learned_addr learned my addr 172.21.15.33:0/3477398161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.545+0000 7f3f3657e640 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3477398161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 msgr2=0x5629171d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3477398161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 msgr2=0x5629171c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3477398161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562916e36210 con 0x5629171d2010 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 0x7f3f1c0a63a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005c40 tx=0x7f3f28005bd0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3f2009c130 tx=0x7f3f20009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9317989d9b3ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f24ff9640 1 -- 172.21.15.33:0/3477398161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f200a4e10 con 0x5629171d2010 2024-04-23T18:26:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f24ff9640 1 -- 172.21.15.33:0/3477398161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2009a4a0 con 0x5629171d2010 2024-04-23T18:26:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.546+0000 7f3f24ff9640 1 -- 172.21.15.33:0/3477398161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f200cdf60 con 0x5629171d2010 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3477398161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 msgr2=0x5629171d4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3f2009c130 tx=0x7f3f20009f30 comp rx=0 tx=0).stop 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3477398161 shutdown_connections 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d4400 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3477398161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3477398161 >> 172.21.15.33:0/3477398161 conn(0x5629171c3340 msgr2=0x5629171c3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3477398161 shutdown_connections 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3477398161 wait complete. 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.547+0000 7f3f3b5d3740 1 -- start start 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 Processor -- start 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c6e30 0x5629171d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171d2010 0x5629171cf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5629171d4b80 con 0x5629171c7750 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5629170be0b0 con 0x5629171d2010 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5629171d4d00 con 0x5629171c6e30 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38334/0 (socket says 172.21.15.33:38334) 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1617346973 learned_addr learned my addr 172.21.15.33:0/1617346973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f3b5d3740 10 client.0 _ll_register_callbacks cb 0x5629170abd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171d2010 0x5629171cf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1617346973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c6e30 msgr2=0x5629171d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c6e30 0x5629171d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171d2010 msgr2=0x5629171cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171d2010 0x5629171cf2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.548+0000 7f3f35d7d640 1 -- 172.21.15.33:0/1617346973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562916f02620 con 0x5629171c7750 2024-04-23T18:26:58.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3f20098600 tx=0x7f3f2009c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414aa60e128e739e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f200a0300 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3f200cdf10 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f2009a4a0 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f200a4e10 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f3b5d3740 10 client.15717 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629171e0b10 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f200041c0 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.549+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3f200cf070 con 0x5629171c7750 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 10 client.15717 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 20 client.15717 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5629171e1e60 0x5629171e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 4 client.15717 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3f2009a4a0 con 0x5629171e1e60 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3657e640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5629171e1e60 0x5629171e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f19ffb640 10 client.15717 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.550+0000 7f3f3657e640 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5629171e1e60 0x5629171e4250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4400fb642b64fb server_cookie=df39d21a1d3dc0a0 in_seq=0 out_seq=0 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f19ffb640 1 -- 172.21.15.33:0/1617346973 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3f2009a4a0 con 0x5629171e1e60 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f19ffb640 10 client.15717 handle_command_reply: tid=0 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 client.15717 shutdown 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 msgr2=0x5629171cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3f20098600 tx=0x7f3f2009c0a0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 shutdown_connections 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5629171e1e60 0x5629171e4250 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f19ffb640 0 client.15717 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171d2010 0x5629171cf2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c7750 0x5629171cecc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/1617346973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c6e30 0x5629171d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.551+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 >> 172.21.15.33:0/1617346973 conn(0x5629171c3340 msgr2=0x5629171b9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.552+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 shutdown_connections 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.552+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/1617346973 wait complete. 2024-04-23T18:26:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.552+0000 7f3f3b5d3740 20 client.15717 trim_cache size 0 max 16384 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 Processor -- start 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 -- start start 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c7750 0x5629171d1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171d2010 0x5629171e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301aba20 con 0x5629171c6e30 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ad560 con 0x5629171c7750 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ac0f0 con 0x5629171d2010 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3657e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c7750 0x5629171d1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3657e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38346/0 (socket says 172.21.15.33:38346) 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.581+0000 7f3f3657e640 1 -- 172.21.15.33:0/2937780701 learned_addr learned my addr 172.21.15.33:0/2937780701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171d2010 0x5629171e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 -- 172.21.15.33:0/2937780701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171d2010 msgr2=0x5629171e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171d2010 0x5629171e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 -- 172.21.15.33:0/2937780701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c7750 msgr2=0x5629171d1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c7750 0x5629171d1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 -- 172.21.15.33:0/2937780701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629171d4400 con 0x5629171c6e30 2024-04-23T18:26:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f3657e640 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009900 tx=0x7f3f2c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea56ae5b984afb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f1bfff640 1 -- 172.21.15.33:0/2937780701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c035040 con 0x5629171c6e30 2024-04-23T18:26:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f1bfff640 1 -- 172.21.15.33:0/2937780701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2009a4a0 con 0x5629171c6e30 2024-04-23T18:26:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.582+0000 7f3f1bfff640 1 -- 172.21.15.33:0/2937780701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c004920 con 0x5629171c6e30 2024-04-23T18:26:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/2937780701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 msgr2=0x5629171dda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009900 tx=0x7f3f2c031400 comp rx=0 tx=0).stop 2024-04-23T18:26:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/2937780701 shutdown_connections 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171d2010 0x5629171e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c7750 0x5629171d1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/2937780701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171c6e30 0x5629171dda40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.583+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/2937780701 >> 172.21.15.33:0/2937780701 conn(0x5629171c3340 msgr2=0x5629171c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.584+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/2937780701 shutdown_connections 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.584+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/2937780701 wait complete. 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.584+0000 7f3f3b5d3740 1 -- start start 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.584+0000 7f3f3b5d3740 1 Processor -- start 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.584+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ace90 con 0x5629171d2010 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301ac7c0 con 0x5629171c6e30 2024-04-23T18:26:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f301a9a00 con 0x5629171c7750 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3657e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3b5d3740 10 client.0 _ll_register_callbacks cb 0x7f3f20009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3657e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171d59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:41370/0 (socket says 172.21.15.33:41370) 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f3657e640 1 -- 172.21.15.33:0/3353086018 learned_addr learned my addr 172.21.15.33:0/3353086018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f36d7f640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 msgr2=0x5629171d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3353086018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 msgr2=0x5629171d6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.585+0000 7f3f35d7d640 1 -- 172.21.15.33:0/3353086018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629171cfc60 con 0x5629171c7750 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f35d7d640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d5ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f200ce1a0 tx=0x7f3f20005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855633b8a764b125 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f20098440 con 0x5629171c7750 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f3f200980f0 con 0x5629171c7750 2024-04-23T18:26:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f2009a4a0 con 0x5629171c7750 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3f200cd960 con 0x5629171c7750 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f3b5d3740 10 client.25330 fetch_fsmap learned FSMap version 28 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562917248270 con 0x5629171c7750 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.586+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f2009a4a0 con 0x5629171c7750 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3f200cf040 con 0x5629171c7750 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 10 client.25330 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 10 client.25330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 20 client.25330 populate_metadata read hostname 'smithi033' 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562917248950 0x56291724ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:26:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 4 client.25330 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:26:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.587+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562917131390 con 0x562917248950 2024-04-23T18:26:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.588+0000 7f3f3657e640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562917248950 0x56291724ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:26:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.588+0000 7f3f3657e640 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562917248950 0x56291724ad70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49cc5884e65ef7be server_cookie=fd862442ed589a8e in_seq=0 out_seq=0 2024-04-23T18:26:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.588+0000 7f3f19ffb640 10 client.25330 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f19ffb640 1 -- 172.21.15.33:0/3353086018 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562917131390 con 0x562917248950 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f19ffb640 10 client.25330 handle_command_reply: tid=0 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 client.25330 shutdown 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 msgr2=0x5629171d5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d5ef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f200ce1a0 tx=0x7f3f20005ab0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 shutdown_connections 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562917248950 0x56291724ad70 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629171d2010 0x5629171d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f19ffb640 0 client.25330 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629171c7750 0x5629171d5ef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 --2- 172.21.15.33:0/3353086018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5629171c6e30 0x5629171d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 >> 172.21.15.33:0/3353086018 conn(0x5629171c3340 msgr2=0x5629171dca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 shutdown_connections 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 1 -- 172.21.15.33:0/3353086018 wait complete. 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.596+0000 7f3f3b5d3740 20 client.25330 trim_cache size 0 max 16384 2024-04-23T18:26:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:26:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:26:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:26:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:26:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 msgr2=0x7f3f1c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 0x7f3f1c0a63a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3f28005c40 tx=0x7f3f28005bd0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 msgr2=0x7f3f301a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:26:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3f20005fd0 tx=0x7f3f2000dde0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 shutdown_connections 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3f1c0a3ee0 0x7f3f1c0a63a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3019a790 0x7f3f3019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f30106170 0x7f3f301a0b60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 --2- 172.21.15.33:0/1536381835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f301010f0 0x7f3f301a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.597+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 >> 172.21.15.33:0/1536381835 conn(0x7f3f30077fd0 msgr2=0x7f3f30100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.598+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 shutdown_connections 2024-04-23T18:26:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:26:58.598+0000 7f3f387e2640 1 -- 172.21.15.33:0/1536381835 wait complete. 2024-04-23T18:26:58.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:26:58 smithi122 ceph-mon[32546]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-04-23T18:26:58.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:26:58 smithi033 ceph-mon[29274]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-04-23T18:26:59.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:26:58 smithi106 ceph-mon[32716]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-04-23T18:27:00.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-mon[29274]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:00.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:27:00.450+0000 7f052afe2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:27:00.450+0000 7f052afe2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.730 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:27:00.450+0000 7f052afe2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:27:00.450+0000 7f052afe2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 18:27:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x[29503]: 2024-04-23T18:27:00.450+0000 7f052afe2700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-23T18:27:00.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:00 smithi122 ceph-mon[32546]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:01.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:00 smithi106 ceph-mon[32716]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:02.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:02 smithi122 ceph-mon[32546]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-23T18:27:02.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:02 smithi122 ceph-mon[32546]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-23T18:27:02.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:02 smithi033 ceph-mon[29274]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-23T18:27:02.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:02 smithi033 ceph-mon[29274]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-23T18:27:03.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:02 smithi106 ceph-mon[32716]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-23T18:27:03.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:02 smithi106 ceph-mon[32716]: mgrmap e25: x(active, since 32s), standbys: z, y 2024-04-23T18:27:04.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:04 smithi122 ceph-mon[32546]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:27:04.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:04 smithi033 ceph-mon[29274]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:27:05.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:04 smithi106 ceph-mon[32716]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:27:06.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:06 smithi122 ceph-mon[32546]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T18:27:06.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:06 smithi033 ceph-mon[29274]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T18:27:07.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:06 smithi106 ceph-mon[32716]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-23T18:27:08.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:08 smithi033 ceph-mon[29274]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:27:09.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:08 smithi106 ceph-mon[32716]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:27:09.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:08 smithi122 ceph-mon[32546]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-04-23T18:27:10.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:10 smithi033 ceph-mon[29274]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-23T18:27:11.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:10 smithi106 ceph-mon[32716]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-23T18:27:11.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:10 smithi122 ceph-mon[32546]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-23T18:27:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 -- 172.21.15.33:0/3663934277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f651410c560 msgr2=0x7f651410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 --2- 172.21.15.33:0/3663934277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f651410c560 0x7f651410e9b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f650c009ca0 tx=0x7f650c02a950 comp rx=0 tx=0).stop 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 -- 172.21.15.33:0/3663934277 shutdown_connections 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 --2- 172.21.15.33:0/3663934277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f651410c560 0x7f651410e9b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 --2- 172.21.15.33:0/3663934277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103b40 0x7f651410c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 --2- 172.21.15.33:0/3663934277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6514103220 0x7f6514103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 -- 172.21.15.33:0/3663934277 >> 172.21.15.33:0/3663934277 conn(0x7f6514077fd0 msgr2=0x7f65140783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 -- 172.21.15.33:0/3663934277 shutdown_connections 2024-04-23T18:27:11.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.738+0000 7f65196b9640 1 -- 172.21.15.33:0/3663934277 wait complete. 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.739+0000 7f65196b9640 1 Processor -- start 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.739+0000 7f65196b9640 1 -- start start 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.739+0000 7f65196b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.739+0000 7f65196b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.739+0000 7f65196b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f651410c560 0x7f651419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f65196b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65141121f0 con 0x7f6514103b40 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f65196b9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6514112070 con 0x7f6514103220 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f65196b9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6514112370 con 0x7f651410c560 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f650bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:40554/0 (socket says 172.21.15.33:40554) 2024-04-23T18:27:11.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 -- 172.21.15.33:0/3928109540 learned_addr learned my addr 172.21.15.33:0/3928109540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f650bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35558/0 (socket says 172.21.15.33:35558) 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f65137fe640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f651410c560 0x7f651419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 -- 172.21.15.33:0/3928109540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f651410c560 msgr2=0x7f651419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f651410c560 0x7f651419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 -- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 msgr2=0x7f65141989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 -- 172.21.15.33:0/3928109540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f651419d4a0 con 0x7f6514103220 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f650bfff640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.740+0000 7f6512ffd640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f650400e9f0 tx=0x7f650400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.741+0000 7f6510ff9640 1 -- 172.21.15.33:0/3928109540 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f650400cd80 con 0x7f6514103220 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.741+0000 7f6510ff9640 1 -- 172.21.15.33:0/3928109540 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65040054d0 con 0x7f6514103220 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.741+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f651419d790 con 0x7f6514103220 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.741+0000 7f6510ff9640 1 -- 172.21.15.33:0/3928109540 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6504010690 con 0x7f6514103220 2024-04-23T18:27:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.741+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65141a5020 con 0x7f6514103220 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.744+0000 7f6510ff9640 1 -- 172.21.15.33:0/3928109540 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6504004e90 con 0x7f6514103220 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.744+0000 7f651c3cd740 1 Processor -- start 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.744+0000 7f651c3cd740 1 -- start start 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 0x55a45423bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a454193970 con 0x55a45422e620 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a4541304a0 con 0x55a454239800 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f651c3cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a45418f930 con 0x55a45422ef40 2024-04-23T18:27:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f65137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f65137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35564/0 (socket says 172.21.15.33:35564) 2024-04-23T18:27:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f65137fe640 1 -- 172.21.15.33:0/354028656 learned_addr learned my addr 172.21.15.33:0/354028656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.745+0000 7f650bfff640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 0x55a45423bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f6512ffd640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 -- 172.21.15.33:0/354028656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 msgr2=0x55a4542392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 -- 172.21.15.33:0/354028656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 msgr2=0x55a45422ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 -- 172.21.15.33:0/354028656 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a453ea6210 con 0x55a454239800 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f65137fe640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f6510ff9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 0x7f65000a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f650bfff640 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 0x55a45423bbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009f20 tx=0x7f64fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b8295a12665b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f65137fe640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 0x7f65000a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f65097fa640 1 -- 172.21.15.33:0/354028656 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004030 con 0x55a454239800 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f65097fa640 1 -- 172.21.15.33:0/354028656 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc005020 con 0x55a454239800 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.746+0000 7f65097fa640 1 -- 172.21.15.33:0/354028656 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc004c80 con 0x55a454239800 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f6510ff9640 1 -- 172.21.15.33:0/3928109540 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f65040d0e80 con 0x7f6514103220 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 -- 172.21.15.33:0/354028656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 msgr2=0x55a45423bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 0x55a45423bbf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009f20 tx=0x7f64fc0118d0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f65137fe640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 0x7f65000a64b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f650c009620 tx=0x7f650c004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 -- 172.21.15.33:0/354028656 shutdown_connections 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a454239800 0x55a45423bbf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 --2- 172.21.15.33:0/354028656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422e620 0x55a45422ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.747+0000 7f651c3cd740 1 -- 172.21.15.33:0/354028656 >> 172.21.15.33:0/354028656 conn(0x55a45422ab30 msgr2=0x55a45422af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.748+0000 7f651c3cd740 1 -- 172.21.15.33:0/354028656 shutdown_connections 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.748+0000 7f651c3cd740 1 -- 172.21.15.33:0/354028656 wait complete. 2024-04-23T18:27:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.748+0000 7f651c3cd740 1 -- start start 2024-04-23T18:27:11.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.748+0000 7f651c3cd740 1 Processor -- start 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454239120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a454239800 0x55a454236950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35570/0 (socket says 172.21.15.33:35570) 2024-04-23T18:27:11.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 -- 172.21.15.33:0/1847448820 learned_addr learned my addr 172.21.15.33:0/1847448820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a45423c430 con 0x55a45422ef40 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a45412e0b0 con 0x55a45422e620 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f6512ffd640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454239120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a45423c5b0 con 0x55a454239800 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f651c3cd740 10 client.0 _ll_register_callbacks cb 0x55a45411bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 -- 172.21.15.33:0/1847448820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a454239800 msgr2=0x55a454236950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a454239800 0x55a454236950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 -- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 msgr2=0x55a454239120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454239120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f650bfff640 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a453f72620 con 0x55a45422ef40 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.749+0000 7f6512ffd640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454239120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f650bfff640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f64fc011dd0 tx=0x7f64fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced437e862313a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f64fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0048e0 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f64fc016040 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc005020 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f64fc004a80 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f651c3cd740 10 client.15741 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.750+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4542480e0 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.751+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc004c20 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.751+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f64fc003760 con 0x55a45422ef40 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.751+0000 7f651c3cd740 10 client.15741 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.751+0000 7f651c3cd740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.751+0000 7f651c3cd740 20 client.15741 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a454249550 0x55a45424b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f651c3cd740 4 client.15741 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f64fc005020 con 0x55a454249550 2024-04-23T18:27:11.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f6512ffd640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a454249550 0x55a45424b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f6512ffd640 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a454249550 0x55a45424b940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37ed7d19b1ab342f server_cookie=797d743fd0c21f5d in_seq=0 out_seq=0 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.752+0000 7f64fa7fc640 10 client.15741 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f64fa7fc640 1 -- 172.21.15.33:0/1847448820 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f64fc005020 con 0x55a454249550 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f64fa7fc640 10 client.15741 handle_command_reply: tid=0 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f651c3cd740 1 client.15741 shutdown 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 msgr2=0x55a454236410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f64fc011dd0 tx=0x7f64fc009c00 comp rx=0 tx=0).stop 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.753+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 shutdown_connections 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a454249550 0x55a45424b940 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a454239800 0x55a454236950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f64fa7fc640 0 client.15741 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a45422ef40 0x55a454236410 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1847448820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454239120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 >> 172.21.15.33:0/1847448820 conn(0x55a45422ab30 msgr2=0x55a4542308b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 shutdown_connections 2024-04-23T18:27:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 1 -- 172.21.15.33:0/1847448820 wait complete. 2024-04-23T18:27:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.754+0000 7f651c3cd740 20 client.15741 trim_cache size 0 max 16384 2024-04-23T18:27:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.784+0000 7f651c3cd740 1 Processor -- start 2024-04-23T18:27:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.784+0000 7f651c3cd740 1 -- start start 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.784+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454238c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.784+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f651c3cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ab490 con 0x55a454239800 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f651c3cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f65141acfd0 con 0x55a45422e620 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f65137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f651c3cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65141abb60 con 0x55a45422ef40 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f65137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35580/0 (socket says 172.21.15.33:35580) 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f65137fe640 1 -- 172.21.15.33:0/1305923424 learned_addr learned my addr 172.21.15.33:0/1305923424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f6512ffd640 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454238c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 -- 172.21.15.33:0/1305923424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 msgr2=0x55a454238c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454238c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 -- 172.21.15.33:0/1305923424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 msgr2=0x55a454251610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f650bfff640 1 -- 172.21.15.33:0/1305923424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a454231fe0 con 0x55a45422ef40 2024-04-23T18:27:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.785+0000 7f65137fe640 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.786+0000 7f650bfff640 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542510d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003b80 tx=0x7f64fc004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61549950ecea2ed9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.786+0000 7f6508ff9640 1 -- 172.21.15.33:0/1305923424 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0035f0 con 0x55a45422ef40 2024-04-23T18:27:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.786+0000 7f6508ff9640 1 -- 172.21.15.33:0/1305923424 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc016070 con 0x55a45422ef40 2024-04-23T18:27:11.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.786+0000 7f6508ff9640 1 -- 172.21.15.33:0/1305923424 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00f9b0 con 0x55a45422ef40 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- 172.21.15.33:0/1305923424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 msgr2=0x55a4542510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542510d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64fc003b80 tx=0x7f64fc004250 comp rx=0 tx=0).stop 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- 172.21.15.33:0/1305923424 shutdown_connections 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a454251610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a4542510d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 --2- 172.21.15.33:0/1305923424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a454238c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- 172.21.15.33:0/1305923424 >> 172.21.15.33:0/1305923424 conn(0x55a45422ab30 msgr2=0x55a45424b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- 172.21.15.33:0/1305923424 shutdown_connections 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- 172.21.15.33:0/1305923424 wait complete. 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.787+0000 7f651c3cd740 1 -- start start 2024-04-23T18:27:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 Processor -- start 2024-04-23T18:27:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a45423ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a4542ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ac900 con 0x55a454239800 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f65141ac230 con 0x55a45422e620 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f651c3cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65141a9520 con 0x55a45422ef40 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f6512ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f6512ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:40608/0 (socket says 172.21.15.33:40608) 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.788+0000 7f6512ffd640 1 -- 172.21.15.33:0/2322322738 learned_addr learned my addr 172.21.15.33:0/2322322738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f65137fe640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a4542ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f650bfff640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a45423ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f651c3cd740 10 client.0 _ll_register_callbacks cb 0x7f64fc004760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 -- 172.21.15.33:0/2322322738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 msgr2=0x55a45423ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a45423ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 -- 172.21.15.33:0/2322322738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 msgr2=0x55a4542ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a4542ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 -- 172.21.15.33:0/2322322738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4542ade80 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f650bfff640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a45423ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f65137fe640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a4542ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f6512ffd640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce400 tx=0x7f65040bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80f1962ee5ed033 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f64fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b6070 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f6504099260 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.789+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc016070 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f650409d2f0 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 10 client.25370 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a4542af110 con 0x55a45422e620 2024-04-23T18:27:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f650409cbf0 con 0x55a45422e620 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65040a53d0 con 0x55a45422e620 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 10 client.25370 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 20 client.25370 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a4542af810 0x55a4542b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 4 client.25370 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f64fc016070 con 0x55a4542af810 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.790+0000 7f650bfff640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a4542af810 0x55a4542b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.791+0000 7f650bfff640 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a4542af810 0x55a4542b1c20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4df0832e9c9b57 server_cookie=ecca26f464622533 in_seq=0 out_seq=0 2024-04-23T18:27:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.791+0000 7f64fa7fc640 10 client.25370 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f64fa7fc640 1 -- 172.21.15.33:0/2322322738 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f64fc016070 con 0x55a4542af810 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f64fa7fc640 10 client.25370 handle_command_reply: tid=0 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 client.25370 shutdown 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 msgr2=0x55a45423e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f65040ce400 tx=0x7f65040bd040 comp rx=0 tx=0).stop 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 shutdown_connections 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a4542af810 0x55a4542b1c20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a454239800 0x55a4542ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f64fa7fc640 0 client.25370 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a45422ef40 0x55a45423ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 --2- 172.21.15.33:0/2322322738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a45422e620 0x55a45423e900 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 >> 172.21.15.33:0/2322322738 conn(0x55a45422ab30 msgr2=0x55a454229420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 shutdown_connections 2024-04-23T18:27:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 1 -- 172.21.15.33:0/2322322738 wait complete. 2024-04-23T18:27:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.797+0000 7f651c3cd740 20 client.25370 trim_cache size 0 max 16384 2024-04-23T18:27:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:27:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:27:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:27:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 msgr2=0x7f65000a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 0x7f65000a64b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f650c009620 tx=0x7f650c004a40 comp rx=0 tx=0).stop 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 msgr2=0x7f65141984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f650400e9f0 tx=0x7f650400edf0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 shutdown_connections 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65000a3ff0 0x7f65000a64b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f651410c560 0x7f651419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6514103b40 0x7f65141989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 --2- 172.21.15.33:0/3928109540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6514103220 0x7f65141984b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 >> 172.21.15.33:0/3928109540 conn(0x7f6514077fd0 msgr2=0x7f65140fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 shutdown_connections 2024-04-23T18:27:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:11.798+0000 7f65196b9640 1 -- 172.21.15.33:0/3928109540 wait complete. 2024-04-23T18:27:12.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:12 smithi033 ceph-mon[29274]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-23T18:27:13.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:12 smithi106 ceph-mon[32716]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-23T18:27:13.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:12 smithi122 ceph-mon[32546]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 30 op/s 2024-04-23T18:27:14.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:14 smithi033 ceph-mon[29274]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-23T18:27:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:14 smithi106 ceph-mon[32716]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-23T18:27:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:14 smithi122 ceph-mon[32546]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-23T18:27:16.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:16.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 -- 172.21.15.33:0/1947613513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 msgr2=0x7fda2c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 --2- 172.21.15.33:0/1947613513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c10dd40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fda24009ca0 tx=0x7fda2402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 -- 172.21.15.33:0/1947613513 shutdown_connections 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 --2- 172.21.15.33:0/1947613513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c10dd40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 --2- 172.21.15.33:0/1947613513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101a50 0x7fda2c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 --2- 172.21.15.33:0/1947613513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.813+0000 7fda339bb640 1 -- 172.21.15.33:0/1947613513 >> 172.21.15.33:0/1947613513 conn(0x7fda2c077f50 msgr2=0x7fda2c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.814+0000 7fda339bb640 1 -- 172.21.15.33:0/1947613513 shutdown_connections 2024-04-23T18:27:16.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.814+0000 7fda339bb640 1 -- 172.21.15.33:0/1947613513 wait complete. 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.814+0000 7fda339bb640 1 Processor -- start 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.814+0000 7fda339bb640 1 -- start start 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c19cf80 0x7fda2c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1110e0 con 0x7fda2c106170 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c110f60 con 0x7fda2c1010a0 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda339bb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c111260 con 0x7fda2c19cf80 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59808/0 (socket says 172.21.15.33:59808) 2024-04-23T18:27:16.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 -- 172.21.15.33:0/3056494213 learned_addr learned my addr 172.21.15.33:0/3056494213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda31f58640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c19cf80 0x7fda2c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda31757640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 -- 172.21.15.33:0/3056494213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c19cf80 msgr2=0x7fda2c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c19cf80 0x7fda2c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 -- 172.21.15.33:0/3056494213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 msgr2=0x7fda2c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 -- 172.21.15.33:0/3056494213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c19da70 con 0x7fda2c106170 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda31757640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.815+0000 7fda30f56640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00e9f0 tx=0x7fda1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.816+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c00ccc0 con 0x7fda2c106170 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.816+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda1c0054d0 con 0x7fda2c106170 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.816+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c19dd60 con 0x7fda2c106170 2024-04-23T18:27:16.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.816+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1c010430 con 0x7fda2c106170 2024-04-23T18:27:16.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.816+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c102d80 con 0x7fda2c106170 2024-04-23T18:27:16.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.817+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda2c068d70 con 0x7fda2c106170 2024-04-23T18:27:16.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.819+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fda1c0046c0 con 0x7fda2c106170 2024-04-23T18:27:16.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.820+0000 7fda227fc640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 0x7fda180a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:16.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.820+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fda1c0d0e20 con 0x7fda2c106170 2024-04-23T18:27:16.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.820+0000 7fda31757640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 0x7fda180a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:16.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.821+0000 7fda31757640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 0x7fda180a6490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda28009b50 tx=0x7fda28005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:16.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:16.828+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda1c09d290 con 0x7fda2c106170 2024-04-23T18:27:16.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:16 smithi033 ceph-mon[29274]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 767 B/s wr, 23 op/s 2024-04-23T18:27:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:16 smithi106 ceph-mon[32716]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 767 B/s wr, 23 op/s 2024-04-23T18:27:17.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.169+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda2c101a50 con 0x7fda2c106170 2024-04-23T18:27:17.247 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:16 smithi122 ceph-mon[32546]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 767 B/s wr, 23 op/s 2024-04-23T18:27:17.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.170+0000 7fda227fc640 1 -- 172.21.15.33:0/3056494213 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7fda1c09cb90 con 0x7fda2c106170 2024-04-23T18:27:17.248 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:27:17.248 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 msgr2=0x7fda180a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 0x7fda180a6490 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fda28009b50 tx=0x7fda28005f90 comp rx=0 tx=0).stop 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 msgr2=0x7fda2c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fda1c00e9f0 tx=0x7fda1c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 shutdown_connections 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fda180a3fd0 0x7fda180a6490 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c19cf80 0x7fda2c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda2c106170 0x7fda2c198c00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 --2- 172.21.15.33:0/3056494213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda2c1010a0 0x7fda2c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 >> 172.21.15.33:0/3056494213 conn(0x7fda2c077f50 msgr2=0x7fda2c069440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 shutdown_connections 2024-04-23T18:27:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.173+0000 7fda339bb640 1 -- 172.21.15.33:0/3056494213 wait complete. 2024-04-23T18:27:17.251 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 4f4e66b4 --release --await' 2024-04-23T18:27:17.252 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 4f4e66b4 --release --await 2024-04-23T18:27:17.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 -- 172.21.15.33:0/187254838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 msgr2=0x7f50a810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 --2- 172.21.15.33:0/187254838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a810dd40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009ca0 tx=0x7f50a002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 -- 172.21.15.33:0/187254838 shutdown_connections 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 --2- 172.21.15.33:0/187254838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a810dd40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 --2- 172.21.15.33:0/187254838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a8101aa0 0x7f50a8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 --2- 172.21.15.33:0/187254838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a81010f0 0x7f50a81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 -- 172.21.15.33:0/187254838 >> 172.21.15.33:0/187254838 conn(0x7f50a8077fb0 msgr2=0x7f50a80783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 -- 172.21.15.33:0/187254838 shutdown_connections 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.399+0000 7f50b07f7640 1 -- 172.21.15.33:0/187254838 wait complete. 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 Processor -- start 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 -- start start 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 0x7f50a8198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 0x7f50a819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81110c0 con 0x7f50a8106170 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8110f40 con 0x7f50a81010f0 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50b07f7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8111240 con 0x7f50a819cfe0 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59836/0 (socket says 172.21.15.33:59836) 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 -- 172.21.15.33:0/2956439809 learned_addr learned my addr 172.21.15.33:0/2956439809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50ae593640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 0x7f50a8198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50aed94640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 0x7f50a819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 -- 172.21.15.33:0/2956439809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 msgr2=0x7f50a819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 0x7f50a819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 -- 172.21.15.33:0/2956439809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 msgr2=0x7f50a8198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 0x7f50a8198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.400+0000 7f50add92640 1 -- 172.21.15.33:0/2956439809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a819dad0 con 0x7f50a8106170 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f50aed94640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 0x7f50a819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f50ae593640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 0x7f50a8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:17.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f50add92640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9c0 tx=0x7f509800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f509f7fe640 1 -- 172.21.15.33:0/2956439809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509800cd80 con 0x7f50a8106170 2024-04-23T18:27:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f509f7fe640 1 -- 172.21.15.33:0/2956439809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980054d0 con 0x7f50a8106170 2024-04-23T18:27:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a819ddc0 con 0x7f50a8106170 2024-04-23T18:27:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f509f7fe640 1 -- 172.21.15.33:0/2956439809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5098010430 con 0x7f50a8106170 2024-04-23T18:27:17.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.401+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81a55b0 con 0x7f50a8106170 2024-04-23T18:27:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.403+0000 7f509f7fe640 1 -- 172.21.15.33:0/2956439809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f5098004fd0 con 0x7f50a8106170 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 Processor -- start 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 -- start start 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552436750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 0x559552445350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552447c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55955239b6d0 con 0x559552445890 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5595523381a0 con 0x559552436370 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.404+0000 7f50b35e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559552397690 con 0x559552436c90 2024-04-23T18:27:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f509f7fe640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 0x7f50940a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552447c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552447c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59846/0 (socket says 172.21.15.33:59846) 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 0x559552445350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50add92640 1 -- 172.21.15.33:0/1922034256 learned_addr learned my addr 172.21.15.33:0/1922034256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50aed94640 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552436750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50aed94640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 0x7f50940a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f509f7fe640 1 -- 172.21.15.33:0/2956439809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f50980d0ef0 con 0x7f50a8106170 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 -- 172.21.15.33:0/1922034256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 msgr2=0x559552436750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552436750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 -- 172.21.15.33:0/1922034256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 msgr2=0x559552447c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552447c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 -- 172.21.15.33:0/1922034256 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595523425b0 con 0x559552436c90 2024-04-23T18:27:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50ae593640 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 0x559552445350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50a40103d0 tx=0x7f50a4010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12056c9dfc2758d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.405+0000 7f50aed94640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 0x7f50940a6390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005fa0 tx=0x7f50a00095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.406+0000 7f509cff9640 1 -- 172.21.15.33:0/1922034256 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4004220 con 0x559552436c90 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.406+0000 7f509cff9640 1 -- 172.21.15.33:0/1922034256 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a40043c0 con 0x559552436c90 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.406+0000 7f509cff9640 1 -- 172.21.15.33:0/1922034256 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4011a50 con 0x559552436c90 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.406+0000 7f50b35e9740 1 -- 172.21.15.33:0/1922034256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 msgr2=0x559552445350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.406+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 0x559552445350 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50a40103d0 tx=0x7f50a4010890 comp rx=0 tx=0).stop 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 -- 172.21.15.33:0/1922034256 shutdown_connections 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552447c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436c90 0x559552445350 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1922034256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552436750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 -- 172.21.15.33:0/1922034256 >> 172.21.15.33:0/1922034256 conn(0x5595524327b0 msgr2=0x559552432b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 -- 172.21.15.33:0/1922034256 shutdown_connections 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 -- 172.21.15.33:0/1922034256 wait complete. 2024-04-23T18:27:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 -- start start 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.407+0000 7f50b35e9740 1 Processor -- start 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552445260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552445890 0x559552442b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559552448770 con 0x559552436c90 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559552335b20 con 0x559552436370 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5595524488f0 con 0x559552445890 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59852/0 (socket says 172.21.15.33:59852) 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 -- 172.21.15.33:0/3642768926 learned_addr learned my addr 172.21.15.33:0/3642768926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50b35e9740 10 client.0 _ll_register_callbacks cb 0x559552335530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50ae593640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552445260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50aed94640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552445890 0x559552442b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 -- 172.21.15.33:0/3642768926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552445890 msgr2=0x559552442b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552445890 0x559552442b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 -- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 msgr2=0x559552445260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552445260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 -- 172.21.15.33:0/3642768926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595520ad920 con 0x559552436c90 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f50add92640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f509809c960 tx=0x7f509809c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655ba345310b3128 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f5091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.408+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50980bd2c0 con 0x559552436c90 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f50980a5440 con 0x559552436c90 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a40043c0 con 0x559552436c90 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f509809cc60 con 0x559552436c90 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 10 client.15765 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559552179d30 con 0x559552436c90 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50980a1010 con 0x559552436c90 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50980989b0 con 0x559552436c90 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 10 client.15765 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 10 client.15765 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 20 client.15765 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x559552457f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 4 client.15765 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50a40043c0 con 0x559552455b20 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.409+0000 7f50ae593640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x559552457f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.410+0000 7f50ae593640 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x559552457f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4098f4ab716b8349 server_cookie=192c67eb0003a667 in_seq=0 out_seq=0 2024-04-23T18:27:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.410+0000 7f5091ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f5091ffb640 1 -- 172.21.15.33:0/3642768926 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f50a40043c0 con 0x559552455b20 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f5091ffb640 10 client.15765 handle_command_reply: tid=0 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 client.15765 shutdown 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 msgr2=0x559552442550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f509809c960 tx=0x7f509809c680 comp rx=0 tx=0).stop 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 shutdown_connections 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x559552457f60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f5091ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552445890 0x559552442b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552436c90 0x559552442550 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3642768926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436370 0x559552445260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 >> 172.21.15.33:0/3642768926 conn(0x5595524327b0 msgr2=0x559552438560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 shutdown_connections 2024-04-23T18:27:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 1 -- 172.21.15.33:0/3642768926 wait complete. 2024-04-23T18:27:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.411+0000 7f50b35e9740 20 client.15765 trim_cache size 0 max 16384 2024-04-23T18:27:17.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.445+0000 7f50b35e9740 1 Processor -- start 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.445+0000 7f50b35e9740 1 -- start start 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x5595524416b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x559552431850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81aba20 con 0x559552445890 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ad560 con 0x559552436c90 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50b35e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ac0f0 con 0x559552436370 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59856/0 (socket says 172.21.15.33:59856) 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 -- 172.21.15.33:0/1552733540 learned_addr learned my addr 172.21.15.33:0/1552733540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50ae593640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x5595524416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50add92640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x559552431850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 -- 172.21.15.33:0/1552733540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 msgr2=0x5595524416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x5595524416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 -- 172.21.15.33:0/1552733540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 msgr2=0x559552431850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x559552431850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 -- 172.21.15.33:0/1552733540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559552437a00 con 0x559552445890 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50add92640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x559552431850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.446+0000 7f50aed94640 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f50a0035000 tx=0x7f50a003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b864598eaa3808 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.447+0000 7f5093fff640 1 -- 172.21.15.33:0/1552733540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0051070 con 0x559552445890 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.447+0000 7f5093fff640 1 -- 172.21.15.33:0/1552733540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0036040 con 0x559552445890 2024-04-23T18:27:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.447+0000 7f5093fff640 1 -- 172.21.15.33:0/1552733540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a002faa0 con 0x559552445890 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.447+0000 7f50b35e9740 1 -- 172.21.15.33:0/1552733540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 msgr2=0x559552444d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.447+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f50a0035000 tx=0x7f50a003b000 comp rx=0 tx=0).stop 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 -- 172.21.15.33:0/1552733540 shutdown_connections 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x559552444d90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x559552431850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- 172.21.15.33:0/1552733540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x5595524416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 -- 172.21.15.33:0/1552733540 >> 172.21.15.33:0/1552733540 conn(0x5595524327b0 msgr2=0x559552431140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 -- 172.21.15.33:0/1552733540 shutdown_connections 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 -- 172.21.15.33:0/1552733540 wait complete. 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 -- start start 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 Processor -- start 2024-04-23T18:27:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x55955243f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x55955243f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.448+0000 7f50b35e9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50b35e9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ace90 con 0x559552445890 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50b35e9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81ac7c0 con 0x559552436c90 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50b35e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50a81a9a00 con 0x559552436370 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59864/0 (socket says 172.21.15.33:59864) 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 -- 172.21.15.33:0/3190081085 learned_addr learned my addr 172.21.15.33:0/3190081085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50ae593640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x55955243f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50add92640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x55955243f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50b35e9740 10 client.0 _ll_register_callbacks cb 0x7f509809c7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 -- 172.21.15.33:0/3190081085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 msgr2=0x55955243f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x55955243f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 -- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 msgr2=0x55955243f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x55955243f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 -- 172.21.15.33:0/3190081085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55955244c1b0 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50ae593640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x55955243f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50aed94640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50a0006010 tx=0x7f50a0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c198f4c13b8a4a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f5091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0051070 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f50a0004570 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.449+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50a0036040 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50a00318e0 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 10 client.15771 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5595524bc5a0 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50a0036040 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50a004a040 con 0x559552445890 2024-04-23T18:27:17.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 10 client.15771 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 10 client.15771 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 20 client.15771 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x5595524bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 4 client.15771 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4f4e66b4", "release": true, "await": true}] 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4f4e66b4", "release": true, "await": true}) -- 0x559552359500 con 0x559552455b20 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.450+0000 7f50ae593640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x5595524bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.451+0000 7f5091ffb640 10 client.15771 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:17.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.451+0000 7f50ae593640 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x5595524bec50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=870c4ae1d3e15fc0 server_cookie=98d43e064919a9b0 in_seq=0 out_seq=0 2024-04-23T18:27:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.453+0000 7f5091ffb640 1 -- 172.21.15.33:0/3190081085 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x559552359500 con 0x559552455b20 2024-04-23T18:27:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.453+0000 7f5091ffb640 10 client.15771 handle_command_reply: tid=0 2024-04-23T18:27:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.453+0000 7f50b35e9740 1 client.15771 shutdown 2024-04-23T18:27:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.453+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 msgr2=0x55955243fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.453+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f50a0006010 tx=0x7f50a0009360 comp rx=0 tx=0).stop 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 shutdown_connections 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559552455b20 0x5595524bec50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559552445890 0x55955243fea0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559552436c90 0x55955243f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 --2- 172.21.15.33:0/3190081085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559552436370 0x55955243f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 >> 172.21.15.33:0/3190081085 conn(0x5595524327b0 msgr2=0x5595524319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f5091ffb640 0 client.15771 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 shutdown_connections 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 1 -- 172.21.15.33:0/3190081085 wait complete. 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b35e9740 20 client.15771 trim_cache size 0 max 16384 2024-04-23T18:27:17.450 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 28, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 4, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "4f4e66b4": { 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 4, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 81.7, 2024-04-23T18:27:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:17.452 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:27:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 msgr2=0x7f50940a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 0x7f50940a6390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f50a0005fa0 tx=0x7f50a00095c0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 msgr2=0x7f50a8198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.454+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f509800e9c0 tx=0x7f509800ee80 comp rx=0 tx=0).stop 2024-04-23T18:27:17.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 shutdown_connections 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f50940a3ed0 0x7f50940a6390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a819cfe0 0x7f50a819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50a8106170 0x7f50a8198cb0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 --2- 172.21.15.33:0/2956439809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50a81010f0 0x7f50a8198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 >> 172.21.15.33:0/2956439809 conn(0x7f50a8077fb0 msgr2=0x7f50a806a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 shutdown_connections 2024-04-23T18:27:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.455+0000 7f50b07f7640 1 -- 172.21.15.33:0/2956439809 wait complete. 2024-04-23T18:27:17.459 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4f4e66b4', seconds elapsed: 23.0 2024-04-23T18:27:17.459 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.65 2024-04-23T18:27:17.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- 172.21.15.33:0/1176482119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 msgr2=0x7f277c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 --2- 172.21.15.33:0/1176482119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c10ee60 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2774009ca0 tx=0x7f277402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- 172.21.15.33:0/1176482119 shutdown_connections 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 --2- 172.21.15.33:0/1176482119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c10ee60 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 --2- 172.21.15.33:0/1176482119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c101080 0x7f277c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 --2- 172.21.15.33:0/1176482119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1006d0 0x7f277c100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- 172.21.15.33:0/1176482119 >> 172.21.15.33:0/1176482119 conn(0x7f277c0773f0 msgr2=0x7f277c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- 172.21.15.33:0/1176482119 shutdown_connections 2024-04-23T18:27:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- 172.21.15.33:0/1176482119 wait complete. 2024-04-23T18:27:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 Processor -- start 2024-04-23T18:27:17.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.573+0000 7f2782805640 1 -- start start 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c1006d0 0x7f277c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1985d0 0x7f277c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112250 con 0x7f277c1072e0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f277c1120d0 con 0x7f277c1985d0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2782805640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f277c1123d0 con 0x7f277c1006d0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59894/0 (socket says 172.21.15.33:59894) 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 -- 172.21.15.33:0/2206834090 learned_addr learned my addr 172.21.15.33:0/2206834090 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277bfff640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c1006d0 0x7f277c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f2780da2640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1985d0 0x7f277c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 -- 172.21.15.33:0/2206834090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c1006d0 msgr2=0x7f277c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c1006d0 0x7f277c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 -- 172.21.15.33:0/2206834090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1985d0 msgr2=0x7f277c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1985d0 0x7f277c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.574+0000 7f277b7fe640 1 -- 172.21.15.33:0/2206834090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c1990f0 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f277b7fe640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f276c00e9c0 tx=0x7f276c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c00ccc0 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f276c0054d0 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c010430 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c1993e0 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.575+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1018c0 con 0x7f277c1072e0 2024-04-23T18:27:17.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.576+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277c199b00 con 0x7f277c1072e0 2024-04-23T18:27:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.577+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f276c005020 con 0x7f277c1072e0 2024-04-23T18:27:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.578+0000 7f27797fa640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 0x7f27640a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.578+0000 7f277bfff640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 0x7f27640a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:17.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.578+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f276c0d0d40 con 0x7f277c1072e0 2024-04-23T18:27:17.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.579+0000 7f277bfff640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 0x7f27640a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2770005d00 tx=0x7f277000b410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.587+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f276c09d1b0 con 0x7f277c1072e0 2024-04-23T18:27:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.894+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f277c101080 con 0x7f277c1072e0 2024-04-23T18:27:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.895+0000 7f27797fa640 1 -- 172.21.15.33:0/2206834090 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f276c09cab0 con 0x7f277c1072e0 2024-04-23T18:27:17.888 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:27:17.888 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:27:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:27:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 msgr2=0x7f27640a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 0x7f27640a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2770005d00 tx=0x7f277000b410 comp rx=0 tx=0).stop 2024-04-23T18:27:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 msgr2=0x7f277c19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f276c00e9c0 tx=0x7f276c00ee80 comp rx=0 tx=0).stop 2024-04-23T18:27:17.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 shutdown_connections 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27640a3f50 0x7f27640a6410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f277c1985d0 0x7f277c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f277c1072e0 0x7f277c19e960 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 --2- 172.21.15.33:0/2206834090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f277c1006d0 0x7f277c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.898+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 >> 172.21.15.33:0/2206834090 conn(0x7f277c0773f0 msgr2=0x7f277c105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.899+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 shutdown_connections 2024-04-23T18:27:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:17.899+0000 7f2782805640 1 -- 172.21.15.33:0/2206834090 wait complete. 2024-04-23T18:27:17.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 81.65 --await-for 42.0' 2024-04-23T18:27:17.903 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 81.65 --await-for 42.0 2024-04-23T18:27:17.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3056494213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 -- 172.21.15.33:0/807943599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1124101a50 msgr2=0x7f1124105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 --2- 172.21.15.33:0/807943599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1124101a50 0x7f1124105a90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1114009900 tx=0x7f111402a410 comp rx=0 tx=0).stop 2024-04-23T18:27:18.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 -- 172.21.15.33:0/807943599 shutdown_connections 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 --2- 172.21.15.33:0/807943599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1124106170 0x7f112410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 --2- 172.21.15.33:0/807943599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1124101a50 0x7f1124105a90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 --2- 172.21.15.33:0/807943599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11241010a0 0x7f1124101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 -- 172.21.15.33:0/807943599 >> 172.21.15.33:0/807943599 conn(0x7f1124077f50 msgr2=0x7f1124078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 -- 172.21.15.33:0/807943599 shutdown_connections 2024-04-23T18:27:18.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.035+0000 7f112c06b640 1 -- 172.21.15.33:0/807943599 wait complete. 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.036+0000 7f112c06b640 1 Processor -- start 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.036+0000 7f112c06b640 1 -- start start 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 0x7f1124197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f112419bf30 0x7f112419c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1124111090 con 0x7f11241010a0 2024-04-23T18:27:18.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1124110f10 con 0x7f1124106170 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112c06b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1124111210 con 0x7f112419bf30 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59902/0 (socket says 172.21.15.33:59902) 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 -- 172.21.15.33:0/2206479368 learned_addr learned my addr 172.21.15.33:0/2206479368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f112a608640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f112419bf30 0x7f112419c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129606640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 0x7f1124197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 -- 172.21.15.33:0/2206479368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f112419bf30 msgr2=0x7f112419c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f112419bf30 0x7f112419c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 -- 172.21.15.33:0/2206479368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 msgr2=0x7f1124197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 0x7f1124197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 -- 172.21.15.33:0/2206479368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f112419ca20 con 0x7f11241010a0 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129606640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 0x7f1124197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.037+0000 7f1129e07640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f112000e9f0 tx=0x7f112000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.038+0000 7f111affd640 1 -- 172.21.15.33:0/2206479368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112000cd80 con 0x7f11241010a0 2024-04-23T18:27:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.038+0000 7f111affd640 1 -- 172.21.15.33:0/2206479368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11200108e0 con 0x7f11241010a0 2024-04-23T18:27:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.038+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f112419cd10 con 0x7f11241010a0 2024-04-23T18:27:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.038+0000 7f111affd640 1 -- 172.21.15.33:0/2206479368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120018980 con 0x7f11241010a0 2024-04-23T18:27:18.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.038+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11241a4520 con 0x7f11241010a0 2024-04-23T18:27:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.040+0000 7f111affd640 1 -- 172.21.15.33:0/2206479368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f1120010430 con 0x7f11241010a0 2024-04-23T18:27:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f111affd640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 0x7f11100a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 Processor -- start 2024-04-23T18:27:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f1129606640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 0x7f11100a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f111affd640 1 -- 172.21.15.33:0/2206479368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f11200d0e10 con 0x7f11241010a0 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 -- start start 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a4fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a3599a3370 con 0x55a359a48eb0 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a3599402e0 con 0x55a359a43d10 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f112ed7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a35999f330 con 0x55a359a43370 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.041+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129606640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 0x7f11100a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1114000c00 tx=0x7f1114009610 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59918/0 (socket says 172.21.15.33:59918) 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 -- 172.21.15.33:0/2069276229 learned_addr learned my addr 172.21.15.33:0/2069276229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129606640 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f112a608640 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a4fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 -- 172.21.15.33:0/2069276229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 msgr2=0x55a359a437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 -- 172.21.15.33:0/2069276229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 msgr2=0x55a359a4fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a4fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 -- 172.21.15.33:0/2069276229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a359781d80 con 0x55a359a48eb0 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f1129e07640 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1120000c00 tx=0x7f11200107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ae295e9b849ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f110ffff640 1 -- 172.21.15.33:0/2069276229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11200a0f30 con 0x55a359a48eb0 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f110ffff640 1 -- 172.21.15.33:0/2069276229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11200a57f0 con 0x55a359a48eb0 2024-04-23T18:27:18.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.042+0000 7f110ffff640 1 -- 172.21.15.33:0/2069276229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120018d80 con 0x55a359a48eb0 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 -- 172.21.15.33:0/2069276229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 msgr2=0x55a359a49290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1120000c00 tx=0x7f11200107e0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 -- 172.21.15.33:0/2069276229 shutdown_connections 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a4fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2069276229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a49290 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.043+0000 7f112ed7e740 1 -- 172.21.15.33:0/2069276229 >> 172.21.15.33:0/2069276229 conn(0x55a359a39f10 msgr2=0x55a359a3a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 -- 172.21.15.33:0/2069276229 shutdown_connections 2024-04-23T18:27:18.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 -- 172.21.15.33:0/2069276229 wait complete. 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 -- start start 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 Processor -- start 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a48eb0 0x55a359a4dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.044+0000 7f112ed7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a359a502d0 con 0x55a359a43d10 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f112ed7e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a35993dfb0 con 0x55a359a48eb0 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f112ed7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a359a50450 con 0x55a359a43370 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f112ed7e740 10 client.0 _ll_register_callbacks cb 0x55a35993d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129606640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:35372/0 (socket says 172.21.15.33:35372) 2024-04-23T18:27:18.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129606640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59932/0 (socket says 172.21.15.33:59932) 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 -- 172.21.15.33:0/369573585 learned_addr learned my addr 172.21.15.33:0/369573585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f112a608640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a48eb0 0x55a359a4dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 -- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a48eb0 msgr2=0x55a359a4dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a48eb0 0x55a359a4dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 -- 172.21.15.33:0/369573585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 msgr2=0x55a359a4d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 -- 172.21.15.33:0/369573585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3597e2510 con 0x55a359a43370 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129606640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f1129e07640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f112000c6a0 tx=0x7f112000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33f11ddce5fb7176 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f110d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.045+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11200a0f30 con 0x55a359a43370 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f1120018b20 con 0x55a359a43370 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11200a57f0 con 0x55a359a43370 2024-04-23T18:27:18.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1120018d20 con 0x55a359a43370 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 10 client.25393 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a359675670 con 0x55a359a43370 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11200988d0 con 0x55a359a43370 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f112009ba40 con 0x55a359a43370 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 10 client.25393 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 10 client.25393 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 20 client.25393 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359a5d710 0x55a359a5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 4 client.25393 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.046+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f11200a57f0 con 0x55a359a5d710 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.047+0000 7f1129606640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359a5d710 0x55a359a5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.047+0000 7f110d7fa640 10 client.25393 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.047+0000 7f1129606640 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359a5d710 0x55a359a5fb00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b692384b992b7cf server_cookie=3237cad210eacd83 in_seq=0 out_seq=0 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f110d7fa640 1 -- 172.21.15.33:0/369573585 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f11200a57f0 con 0x55a359a5d710 2024-04-23T18:27:18.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f110d7fa640 10 client.25393 handle_command_reply: tid=0 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 client.25393 shutdown 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 msgr2=0x55a359a4d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f112000c6a0 tx=0x7f112000cf20 comp rx=0 tx=0).stop 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 shutdown_connections 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359a5d710 0x55a359a5fb00 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a48eb0 0x55a359a4dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43d10 0x55a359a4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f110d7fa640 0 client.25393 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 --2- 172.21.15.33:0/369573585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4d130 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 >> 172.21.15.33:0/369573585 conn(0x55a359a39f10 msgr2=0x55a359a39600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 shutdown_connections 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 1 -- 172.21.15.33:0/369573585 wait complete. 2024-04-23T18:27:18.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.048+0000 7f112ed7e740 20 client.25393 trim_cache size 0 max 16384 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 Processor -- start 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 -- start start 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a58db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11241aba90 con 0x55a359a48eb0 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11241ad5d0 con 0x55a359a43d10 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112ed7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11241ac160 con 0x55a359a43370 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112a608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112a608640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59942/0 (socket says 172.21.15.33:59942) 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f112a608640 1 -- 172.21.15.33:0/456652268 learned_addr learned my addr 172.21.15.33:0/456652268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:18.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f1129e07640 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.083+0000 7f1129606640 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a58db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 -- 172.21.15.33:0/456652268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 msgr2=0x55a359a4ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 -- 172.21.15.33:0/456652268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 msgr2=0x55a359a58db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a58db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 -- 172.21.15.33:0/456652268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a359a50330 con 0x55a359a48eb0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f112a608640 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f111c009370 tx=0x7f111c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a47e2c56211092 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f110f7fe640 1 -- 172.21.15.33:0/456652268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c020070 con 0x55a359a48eb0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f110f7fe640 1 -- 172.21.15.33:0/456652268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f111c01baa0 con 0x55a359a48eb0 2024-04-23T18:27:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.084+0000 7f110f7fe640 1 -- 172.21.15.33:0/456652268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111c010b20 con 0x55a359a48eb0 2024-04-23T18:27:18.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 -- 172.21.15.33:0/456652268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 msgr2=0x55a359a45870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f111c009370 tx=0x7f111c009f90 comp rx=0 tx=0).stop 2024-04-23T18:27:18.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 -- 172.21.15.33:0/456652268 shutdown_connections 2024-04-23T18:27:18.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a48eb0 0x55a359a45870 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a58db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 --2- 172.21.15.33:0/456652268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a43370 0x55a359a4ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 -- 172.21.15.33:0/456652268 >> 172.21.15.33:0/456652268 conn(0x55a359a39f10 msgr2=0x55a359a586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 -- 172.21.15.33:0/456652268 shutdown_connections 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.085+0000 7f112ed7e740 1 -- 172.21.15.33:0/456652268 wait complete. 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 -- start start 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 Processor -- start 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a48eb0 0x55a359a60040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11241acf00 con 0x55a359a43370 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11241ac830 con 0x55a359a43d10 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11241a9a70 con 0x55a359a48eb0 2024-04-23T18:27:18.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f1129e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59958/0 (socket says 172.21.15.33:59958) 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f1129e07640 1 -- 172.21.15.33:0/2728084795 learned_addr learned my addr 172.21.15.33:0/2728084795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.086+0000 7f112ed7e740 10 client.0 _ll_register_callbacks cb 0x7f111403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f112a608640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a48eb0 0x55a359a60040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129606640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 -- 172.21.15.33:0/2728084795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a48eb0 msgr2=0x55a359a60040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a48eb0 0x55a359a60040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 -- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 msgr2=0x55a359a5fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a5fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 -- 172.21.15.33:0/2728084795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a359a60870 con 0x55a359a43370 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129606640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a5fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f1129e07640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f112009be40 tx=0x7f11200ce470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e61e80c820c2bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f110d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:18.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11200991f0 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f11200a23a0 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111c01baa0 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11200a0f30 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f112ed7e740 10 client.15795 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.087+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a359a60870 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f111c01baa0 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f11200ce840 con 0x55a359a43370 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 10 client.15795 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 10 client.15795 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 20 client.15795 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359ac4c80 0x55a359ac7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 4 client.15795 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.65, "await_for": 42.0}] 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.65, "await_for": 42.0}) -- 0x55a3599b0d90 con 0x55a359ac4c80 2024-04-23T18:27:18.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.088+0000 7f1129606640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359ac4c80 0x55a359ac7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:18.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.089+0000 7f110d7fa640 10 client.15795 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:18.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.089+0000 7f1129606640 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359ac4c80 0x55a359ac7070 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a90d7d26c50cfe7 server_cookie=d05798870c629238 in_seq=0 out_seq=0 2024-04-23T18:27:18.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f110d7fa640 1 -- 172.21.15.33:0/2728084795 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55a3599b0d90 con 0x55a359ac4c80 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f110d7fa640 10 client.15795 handle_command_reply: tid=0 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 client.15795 shutdown 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 msgr2=0x55a359a62f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f112009be40 tx=0x7f11200ce470 comp rx=0 tx=0).stop 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 shutdown_connections 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a359ac4c80 0x55a359ac7070 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f110d7fa640 0 client.15795 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a359a48eb0 0x55a359a60040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a359a43d10 0x55a359a5fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 --2- 172.21.15.33:0/2728084795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a359a43370 0x55a359a62f00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 >> 172.21.15.33:0/2728084795 conn(0x55a359a39f10 msgr2=0x55a359a39600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.098+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 shutdown_connections 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112ed7e740 1 -- 172.21.15.33:0/2728084795 wait complete. 2024-04-23T18:27:18.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112ed7e740 20 client.15795 trim_cache size 0 max 16384 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 28, 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 6, 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "532a83b8": { 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 6, 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:18.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 81.7, 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:18.094 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:18.095 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:18.095 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:27:18.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 msgr2=0x7f11100a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 0x7f11100a6410 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1114000c00 tx=0x7f1114009610 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 msgr2=0x7f1124197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f112000e9f0 tx=0x7f112000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 shutdown_connections 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11100a3f50 0x7f11100a6410 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f112419bf30 0x7f112419c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1124106170 0x7f1124197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 --2- 172.21.15.33:0/2206479368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11241010a0 0x7f1124197670 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.099+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 >> 172.21.15.33:0/2206479368 conn(0x7f1124077f50 msgr2=0x7f11240ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.100+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 shutdown_connections 2024-04-23T18:27:18.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:18.100+0000 7f112c06b640 1 -- 172.21.15.33:0/2206479368 wait complete. 2024-04-23T18:27:18.104 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '532a83b8', quiesce took 0.6 seconds. Will release after: 21.65 2024-04-23T18:27:18.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3056494213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:18.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3056494213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:18.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:18 smithi033 ceph-mon[29274]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 682 B/s wr, 20 op/s 2024-04-23T18:27:18.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2206834090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:18 smithi106 ceph-mon[32716]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 682 B/s wr, 20 op/s 2024-04-23T18:27:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:18 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2206834090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:19.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:18 smithi122 ceph-mon[32546]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 682 B/s wr, 20 op/s 2024-04-23T18:27:19.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:18 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2206834090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:20.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:20 smithi033 ceph-mon[29274]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T18:27:21.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:20 smithi106 ceph-mon[32716]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T18:27:21.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:20 smithi122 ceph-mon[32546]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-23T18:27:22.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:22 smithi033 ceph-mon[29274]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-23T18:27:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:22 smithi106 ceph-mon[32716]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-23T18:27:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:22 smithi122 ceph-mon[32546]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 13 op/s 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '57e0caf76fdb7ab8b1358588f292d08519163844'. 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-04-23T18:27:24.370 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-23T18:27:24.371 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 57e0caf76fd qa/suites/fs: add quiescer to the fs suite 2024-04-23T18:27:24.372 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-23T18:27:24.404 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-23T18:27:24.406 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:27:24.406 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 -- 172.21.15.33:0/2458832432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7684106fb0 msgr2=0x7f7684107390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 --2- 172.21.15.33:0/2458832432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7684106fb0 0x7f7684107390 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402a400 comp rx=0 tx=0).stop 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 -- 172.21.15.33:0/2458832432 shutdown_connections 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 --2- 172.21.15.33:0/2458832432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 0x7f768410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 --2- 172.21.15.33:0/2458832432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76841011a0 0x7f7684101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 --2- 172.21.15.33:0/2458832432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7684106fb0 0x7f7684107390 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 -- 172.21.15.33:0/2458832432 >> 172.21.15.33:0/2458832432 conn(0x7f7684076c50 msgr2=0x7f7684077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 -- 172.21.15.33:0/2458832432 shutdown_connections 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.893+0000 7f7689168640 1 -- 172.21.15.33:0/2458832432 wait complete. 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 Processor -- start 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 -- start start 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 0x7f7684198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7684106fb0 0x7f768419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76841121a0 con 0x7f76841011a0 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7684112020 con 0x7f7684101b60 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7689168640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7684112320 con 0x7f7684106fb0 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7682d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59962/0 (socket says 172.21.15.33:59962) 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.894+0000 7f7682d9d640 1 -- 172.21.15.33:0/3852473000 learned_addr learned my addr 172.21.15.33:0/3852473000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f767bfff640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 0x7f7684198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f768359e640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7684106fb0 0x7f768419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 -- 172.21.15.33:0/3852473000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7684106fb0 msgr2=0x7f768419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7684106fb0 0x7f768419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 -- 172.21.15.33:0/3852473000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 msgr2=0x7f7684198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 0x7f7684198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 -- 172.21.15.33:0/3852473000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768419d690 con 0x7f76841011a0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7674005940 tx=0x7f76740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7680d99640 1 -- 172.21.15.33:0/3852473000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767403f030 con 0x7f76841011a0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768419d920 con 0x7f76841011a0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7680d99640 1 -- 172.21.15.33:0/3852473000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674003cb0 con 0x7f76841011a0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7680d99640 1 -- 172.21.15.33:0/3852473000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7674035070 con 0x7f76841011a0 2024-04-23T18:27:24.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.895+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76841a5210 con 0x7f76841011a0 2024-04-23T18:27:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.897+0000 7f7680d99640 1 -- 172.21.15.33:0/3852473000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f7674042030 con 0x7f76841011a0 2024-04-23T18:27:24.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.897+0000 7f768be84740 1 Processor -- start 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 -- start start 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4041ffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c404164970 con 0x55c40420a800 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4041014a0 con 0x55c4041ff620 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768be84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c404160930 con 0x55c4041fff40 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f7680d99640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 0x7f76700a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59970/0 (socket says 172.21.15.33:59970) 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 -- 172.21.15.33:0/544998692 learned_addr learned my addr 172.21.15.33:0/544998692 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768359e640 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4041ffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f768359e640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 0x7f76700a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f7680d99640 1 -- 172.21.15.33:0/3852473000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f76740f1400 con 0x7f76841011a0 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f7682d9d640 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 -- 172.21.15.33:0/544998692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 msgr2=0x55c40420a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 -- 172.21.15.33:0/544998692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 msgr2=0x55c4041ffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4041ffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.898+0000 7f767bfff640 1 -- 172.21.15.33:0/544998692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c403e77210 con 0x55c40420a800 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.899+0000 7f767bfff640 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bc80 tx=0x7f766c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2505dff9354fd240 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.899+0000 7f768359e640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 0x7f76700a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f767c009fd0 tx=0x7f767c004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.899+0000 7f76797fa640 1 -- 172.21.15.33:0/544998692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c015070 con 0x55c40420a800 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.899+0000 7f76797fa640 1 -- 172.21.15.33:0/544998692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c01a910 con 0x55c40420a800 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.899+0000 7f76797fa640 1 -- 172.21.15.33:0/544998692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c011840 con 0x55c40420a800 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- 172.21.15.33:0/544998692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 msgr2=0x55c40420cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f766c00bc80 tx=0x7f766c00f730 comp rx=0 tx=0).stop 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- 172.21.15.33:0/544998692 shutdown_connections 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c40420cbf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- 172.21.15.33:0/544998692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4041ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- 172.21.15.33:0/544998692 >> 172.21.15.33:0/544998692 conn(0x55c4041fbb30 msgr2=0x55c4041fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- 172.21.15.33:0/544998692 shutdown_connections 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- 172.21.15.33:0/544998692 wait complete. 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- start start 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 Processor -- start 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041ff620 0x55c40420a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041fff40 0x55c404207410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c40420d430 con 0x55c40420a800 2024-04-23T18:27:24.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c4040ff0b0 con 0x55c4041fff40 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c40420d5b0 con 0x55c4041ff620 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768be84740 10 client.0 _ll_register_callbacks cb 0x55c4040ecd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.900+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59976/0 (socket says 172.21.15.33:59976) 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 -- 172.21.15.33:0/3145099795 learned_addr learned my addr 172.21.15.33:0/3145099795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041ff620 0x55c40420a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f767bfff640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041fff40 0x55c404207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 -- 172.21.15.33:0/3145099795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041ff620 msgr2=0x55c40420a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041ff620 0x55c40420a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 -- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041fff40 msgr2=0x55c404207410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041fff40 0x55c404207410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 -- 172.21.15.33:0/3145099795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c403f43620 con 0x55c40420a800 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768359e640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f767c009a00 tx=0x7f767c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f1dca4bbd5b30d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f766a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c004d20 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f767c00f980 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f766c01a910 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f767c00fb80 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768be84740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.901+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c404219370 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f766c01a910 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f767c01d040 con 0x55c40420a800 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 20 client.15807 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c40421a830 0x55c40421cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 4 client.15807 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c404172390 con 0x55c40421a830 2024-04-23T18:27:24.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c40421a830 0x55c40421cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f7682d9d640 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c40421a830 0x55c40421cc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f0b2d5fb1739f3f server_cookie=9e3f07a47f23d0ea in_seq=0 out_seq=0 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.902+0000 7f766a7fc640 10 client.15807 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f766a7fc640 1 -- 172.21.15.33:0/3145099795 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c404172390 con 0x55c40421a830 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f766a7fc640 10 client.15807 handle_command_reply: tid=0 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 client.15807 shutdown 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 msgr2=0x55c404207950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f767c009a00 tx=0x7f767c004a40 comp rx=0 tx=0).stop 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 shutdown_connections 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c40421a830 0x55c40421cc40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404207950 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041fff40 0x55c404207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f766a7fc640 0 client.15807 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:24.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 --2- 172.21.15.33:0/3145099795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041ff620 0x55c40420a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 >> 172.21.15.33:0/3145099795 conn(0x55c4041fbb30 msgr2=0x55c404201860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 shutdown_connections 2024-04-23T18:27:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.903+0000 7f768be84740 1 -- 172.21.15.33:0/3145099795 wait complete. 2024-04-23T18:27:24.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.904+0000 7f768be84740 20 client.15807 trim_cache size 0 max 16384 2024-04-23T18:27:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.918+0000 7f768be84740 1 Processor -- start 2024-04-23T18:27:24.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.918+0000 7f768be84740 1 -- start start 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.918+0000 7f768be84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.918+0000 7f768be84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c404222550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768be84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768be84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76841adb30 con 0x55c40420a800 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768be84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f76841af670 con 0x55c4041ff620 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768be84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ae200 con 0x55c4041fff40 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59980/0 (socket says 172.21.15.33:59980) 2024-04-23T18:27:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 -- 172.21.15.33:0/1804009472 learned_addr learned my addr 172.21.15.33:0/1804009472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f7682d9d640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f767bfff640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c404222550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 -- 172.21.15.33:0/1804009472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 msgr2=0x55c404222550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c404222550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 -- 172.21.15.33:0/1804009472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 msgr2=0x55c4042162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 -- 172.21.15.33:0/1804009472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c404205960 con 0x55c40420a800 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f767bfff640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c404222550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f7682d9d640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f768359e640 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f767c000f80 tx=0x7f767c011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6df96fb272b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f7678ff9640 1 -- 172.21.15.33:0/1804009472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c036070 con 0x55c40420a800 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f7678ff9640 1 -- 172.21.15.33:0/1804009472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767c01f710 con 0x55c40420a800 2024-04-23T18:27:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.919+0000 7f7678ff9640 1 -- 172.21.15.33:0/1804009472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c01fd70 con 0x55c40420a800 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- 172.21.15.33:0/1804009472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 msgr2=0x55c404222a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f767c000f80 tx=0x7f767c011fd0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- 172.21.15.33:0/1804009472 shutdown_connections 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c404222a90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c404222550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 --2- 172.21.15.33:0/1804009472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- 172.21.15.33:0/1804009472 >> 172.21.15.33:0/1804009472 conn(0x55c4041fbb30 msgr2=0x55c4041fa420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- 172.21.15.33:0/1804009472 shutdown_connections 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- 172.21.15.33:0/1804009472 wait complete. 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.920+0000 7f768be84740 1 -- start start 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 Processor -- start 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76841aefa0 con 0x55c40420a800 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f76841ae8d0 con 0x55c4041ff620 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76841abb10 con 0x55c4041fff40 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768be84740 10 client.0 _ll_register_callbacks cb 0x7f767c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59990/0 (socket says 172.21.15.33:59990) 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 -- 172.21.15.33:0/2290582954 learned_addr learned my addr 172.21.15.33:0/2290582954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f7682d9d640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f767bfff640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 -- 172.21.15.33:0/2290582954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 msgr2=0x55c40420fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 -- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 msgr2=0x55c4042128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042128d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 -- 172.21.15.33:0/2290582954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4042106e0 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f7682d9d640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f768359e640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f767c0096d0 tx=0x7f767c00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef55a0a2000def3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.921+0000 7f766a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c036040 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f767c01faf0 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f767c01f710 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f767c01fcf0 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c404280a60 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767c01f710 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f767c01d030 con 0x55c40420a800 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 20 client.15813 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:24.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c4042810c0 0x55c4042834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 4 client.15813 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.922+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c40416c200 con 0x55c4042810c0 2024-04-23T18:27:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.923+0000 7f7682d9d640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c4042810c0 0x55c4042834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.923+0000 7f7682d9d640 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c4042810c0 0x55c4042834e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31f8560485f6243 server_cookie=d5f1ee5b24eae33f in_seq=0 out_seq=0 2024-04-23T18:27:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.923+0000 7f766a7fc640 10 client.15813 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.926+0000 7f766a7fc640 1 -- 172.21.15.33:0/2290582954 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c40416c200 con 0x55c4042810c0 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.926+0000 7f766a7fc640 10 client.15813 handle_command_reply: tid=0 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.926+0000 7f768be84740 1 client.15813 shutdown 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 msgr2=0x55c4042101a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f767c0096d0 tx=0x7f767c00fee0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 shutdown_connections 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c4042810c0 0x55c4042834e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c40420a800 0x55c4042101a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4041fff40 0x55c40420fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f766a7fc640 0 client.15813 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 --2- 172.21.15.33:0/2290582954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c4041ff620 0x55c4042128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 >> 172.21.15.33:0/2290582954 conn(0x55c4041fbb30 msgr2=0x55c4041facb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 shutdown_connections 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 1 -- 172.21.15.33:0/2290582954 wait complete. 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f768be84740 20 client.15813 trim_cache size 0 max 16384 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:27:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:27:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:27:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 msgr2=0x7f76700a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 0x7f76700a6430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f767c009fd0 tx=0x7f767c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 msgr2=0x7f76841986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7674005940 tx=0x7f76740042f0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 shutdown_connections 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76700a3f70 0x7f76700a6430 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7684106fb0 0x7f768419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7684101b60 0x7f7684198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 --2- 172.21.15.33:0/3852473000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76841011a0 0x7f76841986a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 >> 172.21.15.33:0/3852473000 conn(0x7f7684076c50 msgr2=0x7f76840fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.927+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 shutdown_connections 2024-04-23T18:27:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:24.928+0000 7f7689168640 1 -- 172.21.15.33:0/3852473000 wait complete. 2024-04-23T18:27:24.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:24 smithi033 ceph-mon[29274]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:25.056 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-23T18:27:25.112 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-23T18:27:25.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:24 smithi106 ceph-mon[32716]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:25.152 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-23T18:27:25.153 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:27:25.153 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-23T18:27:25.194 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-23T18:27:25.197 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T18:27:25.197 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-23T18:27:25.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:24 smithi122 ceph-mon[32546]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:25.254 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-23T18:27:25.255 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-23T18:27:25.255 DEBUG:teuthology.orchestra.run.smithi033:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=57e0caf76fdb7ab8b1358588f292d08519163844 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T18:27:26.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:26 smithi033 ceph-mon[29274]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:27.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:26 smithi106 ceph-mon[32716]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:27.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:26 smithi122 ceph-mon[32546]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:29.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:28 smithi106 ceph-mon[32716]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:27:29.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:28 smithi122 ceph-mon[32546]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:27:29.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:28 smithi033 ceph-mon[29274]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-23T18:27:31.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:30 smithi106 ceph-mon[32716]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:30 smithi122 ceph-mon[32546]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:31.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:30 smithi033 ceph-mon[29274]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:27:33.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:32 smithi106 ceph-mon[32716]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:27:33.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:32 smithi122 ceph-mon[32546]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:27:33.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:32 smithi033 ceph-mon[29274]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-23T18:27:35.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:34 smithi106 ceph-mon[32716]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-23T18:27:35.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:34 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:34 smithi122 ceph-mon[32546]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-23T18:27:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:34 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:35.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:34 smithi033 ceph-mon[29274]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-23T18:27:35.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:34 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:27:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:36.099 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:36.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:36.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:27:36.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:27:36.132 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:27:37.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:36 smithi122 ceph-mon[32546]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:37.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:36 smithi106 ceph-mon[32716]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:37.148 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:36 smithi033 ceph-mon[29274]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-23T18:27:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- 172.21.15.33:0/1897654916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac101aa0 msgr2=0x7fa2ac105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 --2- 172.21.15.33:0/1897654916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac101aa0 0x7fa2ac105a90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009900 tx=0x7fa2a002a410 comp rx=0 tx=0).stop 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- 172.21.15.33:0/1897654916 shutdown_connections 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 --2- 172.21.15.33:0/1897654916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 0x7fa2ac10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 --2- 172.21.15.33:0/1897654916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac101aa0 0x7fa2ac105a90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 --2- 172.21.15.33:0/1897654916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- 172.21.15.33:0/1897654916 >> 172.21.15.33:0/1897654916 conn(0x7fa2ac077fd0 msgr2=0x7fa2ac0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- 172.21.15.33:0/1897654916 shutdown_connections 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- 172.21.15.33:0/1897654916 wait complete. 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 Processor -- start 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.017+0000 7fa2b4605640 1 -- start start 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 0x7fa2ac103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1110b0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac110f30 con 0x7fa2ac106170 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac111230 con 0x7fa2ac1010f0 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b23a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39622/0 (socket says 172.21.15.33:39622) 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 -- 172.21.15.33:0/2788512745 learned_addr learned my addr 172.21.15.33:0/2788512745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b23a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:38834/0 (socket says 172.21.15.33:38834) 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 0x7fa2ac103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 -- 172.21.15.33:0/2788512745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 msgr2=0x7fa2ac102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 -- 172.21.15.33:0/2788512745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 msgr2=0x7fa2ac103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 0x7fa2ac103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 -- 172.21.15.33:0/2788512745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac1a6fc0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b2ba2640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400eaa0 tx=0x7fa2a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa29f7fe640 1 -- 172.21.15.33:0/2788512745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cdb0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.018+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac1a72b0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.019+0000 7fa29f7fe640 1 -- 172.21.15.33:0/2788512745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a4005020 con 0x7fa2ac1035b0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.019+0000 7fa29f7fe640 1 -- 172.21.15.33:0/2788512745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40116d0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.019+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac1a77d0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 Processor -- start 2024-04-23T18:27:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa29f7fe640 1 -- 172.21.15.33:0/2788512745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fa2a40051d0 con 0x7fa2ac1035b0 2024-04-23T18:27:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 -- start start 2024-04-23T18:27:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557163554970 con 0x5571635ef620 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5571634f14a0 con 0x5571635eff40 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557163550930 con 0x5571635fa800 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39634/0 (socket says 172.21.15.33:39634) 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/551224129 learned_addr learned my addr 172.21.15.33:0/551224129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.022+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 0x7fa2980a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/551224129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 msgr2=0x5571635fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/551224129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 msgr2=0x5571635fa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/551224129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557163267210 con 0x5571635ef620 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b2ba2640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 0x7fa2980a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa29f7fe640 1 -- 172.21.15.33:0/2788512745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fa2a40d0c20 con 0x7fa2ac1035b0 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005ab0 tx=0x7fa2a00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feb156ba5eae806 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa29cbfe640 1 -- 172.21.15.33:0/551224129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035050 con 0x5571635ef620 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa29cbfe640 1 -- 172.21.15.33:0/551224129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0004030 con 0x5571635ef620 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.023+0000 7fa29cbfe640 1 -- 172.21.15.33:0/551224129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0005540 con 0x5571635ef620 2024-04-23T18:27:38.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- 172.21.15.33:0/551224129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 msgr2=0x5571635efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b2ba2640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 0x7fa2980a6470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40ce1a0 tx=0x7fa2a40bd040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005ab0 tx=0x7fa2a00058b0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- 172.21.15.33:0/551224129 shutdown_connections 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/551224129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x5571635efa00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- 172.21.15.33:0/551224129 >> 172.21.15.33:0/551224129 conn(0x5571635ebb30 msgr2=0x5571635ebf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- 172.21.15.33:0/551224129 shutdown_connections 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- 172.21.15.33:0/551224129 wait complete. 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 -- start start 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.024+0000 7fa2b73fa740 1 Processor -- start 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x5571635fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635fa800 0x5571635f7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5571635fd3d0 con 0x5571635eff40 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5571634ef0b0 con 0x5571635fa800 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571635fd550 con 0x5571635ef620 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b73fa740 10 client.0 _ll_register_callbacks cb 0x5571634dcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x5571635fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39644/0 (socket says 172.21.15.33:39644) 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 -- 172.21.15.33:0/1867219769 learned_addr learned my addr 172.21.15.33:0/1867219769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635fa800 0x5571635f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 -- 172.21.15.33:0/1867219769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 msgr2=0x5571635fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x5571635fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 -- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635fa800 msgr2=0x5571635f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635fa800 0x5571635f7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 -- 172.21.15.33:0/1867219769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557163333620 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8012700 tx=0x7fa2a8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e31702366e6564 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.025+0000 7fa295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004050 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fa2a80041f0 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a0004030 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2a80043f0 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557163609340 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a0004030 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa2a8003c30 con 0x5571635eff40 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 20 client.15825 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55716360a7b0 0x55716360cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 4 client.15825 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.026+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557163562390 con 0x55716360a7b0 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.027+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55716360a7b0 0x55716360cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.027+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55716360a7b0 0x55716360cba0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f01f5975773af9 server_cookie=6e8402f07faba4a0 in_seq=0 out_seq=0 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.027+0000 7fa295ffb640 10 client.15825 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa295ffb640 1 -- 172.21.15.33:0/1867219769 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557163562390 con 0x55716360a7b0 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa295ffb640 10 client.15825 handle_command_reply: tid=0 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 client.15825 shutdown 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 msgr2=0x5571635f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8012700 tx=0x7fa2a8012bc0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 shutdown_connections 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55716360a7b0 0x55716360cba0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635fa800 0x5571635f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635eff40 0x5571635f7430 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa295ffb640 0 client.15825 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/1867219769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x5571635fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 >> 172.21.15.33:0/1867219769 conn(0x5571635ebb30 msgr2=0x5571635f18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 shutdown_connections 2024-04-23T18:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 1 -- 172.21.15.33:0/1867219769 wait complete. 2024-04-23T18:27:38.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.028+0000 7fa2b73fa740 20 client.15825 trim_cache size 0 max 16384 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 Processor -- start 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 -- start start 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1afcd0 con 0x5571635ef620 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b1810 con 0x5571635eff40 2024-04-23T18:27:38.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b03a0 con 0x5571635fa800 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39656/0 (socket says 172.21.15.33:39656) 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/3248664837 learned_addr learned my addr 172.21.15.33:0/3248664837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.043+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/3248664837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 msgr2=0x5571635f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/3248664837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 msgr2=0x5571635f2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635f2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/3248664837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571635f5980 con 0x5571635ef620 2024-04-23T18:27:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009850 tx=0x7fa2a00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e120e890ee4c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa297fff640 1 -- 172.21.15.33:0/3248664837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035040 con 0x5571635ef620 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa297fff640 1 -- 172.21.15.33:0/3248664837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a00035f0 con 0x5571635ef620 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa297fff640 1 -- 172.21.15.33:0/3248664837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00315e0 con 0x5571635ef620 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b73fa740 1 -- 172.21.15.33:0/3248664837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 msgr2=0x557163606290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.044+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009850 tx=0x7fa2a00058b0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- 172.21.15.33:0/3248664837 shutdown_connections 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635fa800 0x5571635f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635f2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/3248664837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635ef620 0x557163606290 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- 172.21.15.33:0/3248664837 >> 172.21.15.33:0/3248664837 conn(0x5571635ebb30 msgr2=0x5571635f2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- 172.21.15.33:0/3248664837 shutdown_connections 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- 172.21.15.33:0/3248664837 wait complete. 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- start start 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 Processor -- start 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x557163602a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b1140 con 0x5571635fa800 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b0a70 con 0x5571635eff40 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1adcd0 con 0x5571635ef620 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b73fa740 10 client.0 _ll_register_callbacks cb 0x7fa2a8009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b1ba0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39660/0 (socket says 172.21.15.33:39660) 2024-04-23T18:27:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/2488660068 learned_addr learned my addr 172.21.15.33:0/2488660068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.045+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x557163602a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/2488660068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 msgr2=0x557163602a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x557163602a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 msgr2=0x5571635fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 -- 172.21.15.33:0/2488660068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571635feac0 con 0x5571635fa800 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b1ba0640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005a80 tx=0x7fa2a0050040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf959cf110921615 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa295ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035070 con 0x5571635fa800 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fa2a00039b0 con 0x5571635fa800 2024-04-23T18:27:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a00035f0 con 0x5571635fa800 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2a0003bb0 con 0x5571635fa800 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b73fa740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557163670c20 con 0x5571635fa800 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.046+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a00035f0 con 0x5571635fa800 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa2a00320b0 con 0x5571635fa800 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 20 client.15831 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557163670fb0 0x5571636733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 4 client.15831 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5571635f7d00 con 0x557163670fb0 2024-04-23T18:27:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557163670fb0 0x5571636733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa295ffb640 10 client.15831 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.047+0000 7fa2b23a1640 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557163670fb0 0x5571636733a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9480983bbb151937 server_cookie=ef185c2394349426 in_seq=0 out_seq=0 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa295ffb640 1 -- 172.21.15.33:0/2488660068 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5571635f7d00 con 0x557163670fb0 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa295ffb640 10 client.15831 handle_command_reply: tid=0 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 client.15831 shutdown 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 msgr2=0x5571635fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0005a80 tx=0x7fa2a0050040 comp rx=0 tx=0).stop 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 shutdown_connections 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557163670fb0 0x5571636733a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5571635fa800 0x5571635fe380 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5571635eff40 0x5571635fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa295ffb640 0 client.15831 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 --2- 172.21.15.33:0/2488660068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571635ef620 0x557163602a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 >> 172.21.15.33:0/2488660068 conn(0x5571635ebb30 msgr2=0x5571635eacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 shutdown_connections 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 1 -- 172.21.15.33:0/2488660068 wait complete. 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b73fa740 20 client.15831 trim_cache size 0 max 16384 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:27:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:27:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:27:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 msgr2=0x7fa2980a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 0x7fa2980a6470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40ce1a0 tx=0x7fa2a40bd040 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 msgr2=0x7fa2ac1a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.053+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400eaa0 tx=0x7fa2a400ef60 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 shutdown_connections 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa2980a3fb0 0x7fa2980a6470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ac1035b0 0x7fa2ac1a68c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa2ac106170 0x7fa2ac103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 --2- 172.21.15.33:0/2788512745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2ac1010f0 0x7fa2ac102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 >> 172.21.15.33:0/2788512745 conn(0x7fa2ac077fd0 msgr2=0x7fa2ac069940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 shutdown_connections 2024-04-23T18:27:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:38.054+0000 7fa2b4605640 1 -- 172.21.15.33:0/2788512745 wait complete. 2024-04-23T18:27:39.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:38 smithi106 ceph-mon[32716]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:39.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:38 smithi122 ceph-mon[32546]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:39.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:38 smithi033 ceph-mon[29274]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 511 B/s wr, 9 op/s 2024-04-23T18:27:39.755 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:27:39.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 -- 172.21.15.33:0/3870581146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 msgr2=0x7f9564101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 --2- 172.21.15.33:0/3870581146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564101480 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9560009900 tx=0x7f956002a410 comp rx=0 tx=0).stop 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 -- 172.21.15.33:0/3870581146 shutdown_connections 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 --2- 172.21.15.33:0/3870581146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9564106170 0x7f956410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 --2- 172.21.15.33:0/3870581146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564101a50 0x7f9564105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 --2- 172.21.15.33:0/3870581146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564101480 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 -- 172.21.15.33:0/3870581146 >> 172.21.15.33:0/3870581146 conn(0x7f9564077f50 msgr2=0x7f9564078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 -- 172.21.15.33:0/3870581146 shutdown_connections 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.879+0000 7f956c29a640 1 -- 172.21.15.33:0/3870581146 wait complete. 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 Processor -- start 2024-04-23T18:27:39.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 -- start start 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 0x7f9564198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956419cfd0 0x7f956419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95641110e0 con 0x7f95641010a0 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9564110f60 con 0x7f9564106170 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956c29a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9564111260 con 0x7f956419cfd0 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956a036640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956a036640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39670/0 (socket says 172.21.15.33:39670) 2024-04-23T18:27:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.880+0000 7f956a036640 1 -- 172.21.15.33:0/3947261957 learned_addr learned my addr 172.21.15.33:0/3947261957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f9569835640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 0x7f9564198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a837640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956419cfd0 0x7f956419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 -- 172.21.15.33:0/3947261957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956419cfd0 msgr2=0x7f956419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956419cfd0 0x7f956419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 -- 172.21.15.33:0/3947261957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 msgr2=0x7f9564198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 0x7f9564198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 -- 172.21.15.33:0/3947261957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419dac0 con 0x7f95641010a0 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f9569835640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 0x7f9564198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956a036640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9560005940 tx=0x7f9560004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956003f030 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419dd50 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9560003aa0 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560035070 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.881+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a5510 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.882+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9564068d70 con 0x7f95641010a0 2024-04-23T18:27:39.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.883+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9560042030 con 0x7f95641010a0 2024-04-23T18:27:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.884+0000 7f955b7fe640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 0x7f95500a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.884+0000 7f9569835640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 0x7f95500a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.884+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f95600f14b0 con 0x7f95641010a0 2024-04-23T18:27:39.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.885+0000 7f9569835640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 0x7f95500a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95540058c0 tx=0x7f9554005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:39.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:39.888+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95600bd920 con 0x7f95641010a0 2024-04-23T18:27:40.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.113+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9564101a50 con 0x7f95641010a0 2024-04-23T18:27:40.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.114+0000 7f955b7fe640 1 -- 172.21.15.33:0/3947261957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11929 (secure 0 0 0) 0x7f95600bd220 con 0x7f95641010a0 2024-04-23T18:27:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:27:40.108 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:26:28.070986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:27:40.109 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 msgr2=0x7f95500a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 0x7f95500a6390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f95540058c0 tx=0x7f9554005850 comp rx=0 tx=0).stop 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 msgr2=0x7f9564198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9560005940 tx=0x7f9560004270 comp rx=0 tx=0).stop 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 shutdown_connections 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f95500a3ed0 0x7f95500a6390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f956419cfd0 0x7f956419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9564106170 0x7f9564198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 --2- 172.21.15.33:0/3947261957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95641010a0 0x7f9564198710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 >> 172.21.15.33:0/3947261957 conn(0x7f9564077f50 msgr2=0x7f95641001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 shutdown_connections 2024-04-23T18:27:40.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.116+0000 7f956c29a640 1 -- 172.21.15.33:0/3947261957 wait complete. 2024-04-23T18:27:40.121 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 532a83b8 --release --await' 2024-04-23T18:27:40.121 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 532a83b8 --release --await 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 -- 172.21.15.33:0/1820540344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa81010a0 msgr2=0x7f7aa8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 --2- 172.21.15.33:0/1820540344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa81010a0 0x7f7aa8101480 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009900 tx=0x7f7a9802a410 comp rx=0 tx=0).stop 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 -- 172.21.15.33:0/1820540344 shutdown_connections 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 --2- 172.21.15.33:0/1820540344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa8106170 0x7f7aa810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 --2- 172.21.15.33:0/1820540344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa8101a50 0x7f7aa8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 --2- 172.21.15.33:0/1820540344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa81010a0 0x7f7aa8101480 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 -- 172.21.15.33:0/1820540344 >> 172.21.15.33:0/1820540344 conn(0x7f7aa80771d0 msgr2=0x7f7aa80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 -- 172.21.15.33:0/1820540344 shutdown_connections 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.236+0000 7f7aace69640 1 -- 172.21.15.33:0/1820540344 wait complete. 2024-04-23T18:27:40.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 Processor -- start 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 -- start start 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 0x7f7aa81a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 0x7f7aa819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81110d0 con 0x7f7aa8106170 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8110f50 con 0x7f7aa819a7e0 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aace69640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8111250 con 0x7f7aa81010a0 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39680/0 (socket says 172.21.15.33:39680) 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 -- 172.21.15.33:0/1486873263 learned_addr learned my addr 172.21.15.33:0/1486873263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aa659c640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 0x7f7aa81a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 0x7f7aa819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 -- 172.21.15.33:0/1486873263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 msgr2=0x7f7aa81a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 0x7f7aa81a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 -- 172.21.15.33:0/1486873263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 msgr2=0x7f7aa819ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 0x7f7aa819ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 -- 172.21.15.33:0/1486873263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa819b370 con 0x7f7aa8106170 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aa659c640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 0x7f7aa81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 0x7f7aa819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.237+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9f0 tx=0x7f7a9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.238+0000 7f7a9f7fe640 1 -- 172.21.15.33:0/1486873263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9000ccc0 con 0x7f7aa8106170 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.238+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa819b660 con 0x7f7aa8106170 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.238+0000 7f7a9f7fe640 1 -- 172.21.15.33:0/1486873263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a900054d0 con 0x7f7aa8106170 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.238+0000 7f7a9f7fe640 1 -- 172.21.15.33:0/1486873263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90010430 con 0x7f7aa8106170 2024-04-23T18:27:40.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.238+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81a77f0 con 0x7f7aa8106170 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7a9f7fe640 1 -- 172.21.15.33:0/1486873263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f7a900046c0 con 0x7f7aa8106170 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 Processor -- start 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 -- start start 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb48f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 0x55cebcb4b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cebcaa36d0 con 0x55cebcb3e490 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cebca401a0 con 0x55cebcb3edb0 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7aafb76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cebca9f690 con 0x55cebcb49480 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.240+0000 7f7a9f7fe640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 0x7f7a940a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39682/0 (socket says 172.21.15.33:39682) 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/513396038 learned_addr learned my addr 172.21.15.33:0/513396038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa659c640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb48f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 0x7f7a940a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9f7fe640 1 -- 172.21.15.33:0/1486873263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7a900d0fb0 con 0x7f7aa8106170 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 0x55cebcb4b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/513396038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 msgr2=0x55cebcb4b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 0x55cebcb4b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/513396038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 msgr2=0x55cebcb48f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb48f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/513396038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebca4a5b0 con 0x55cebcb3e490 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 0x55cebcb4b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa659c640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0013de0 tx=0x7f7aa0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad644133ec5b3414 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 0x7f7a940a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0010c40 tx=0x7f7aa0015000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9cff9640 1 -- 172.21.15.33:0/513396038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000bb60 con 0x55cebcb3e490 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9cff9640 1 -- 172.21.15.33:0/513396038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0009be0 con 0x55cebcb3e490 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.241+0000 7f7a9cff9640 1 -- 172.21.15.33:0/513396038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0005640 con 0x55cebcb3e490 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 -- 172.21.15.33:0/513396038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 msgr2=0x55cebcb3e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0013de0 tx=0x7f7aa0010740 comp rx=0 tx=0).stop 2024-04-23T18:27:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 -- 172.21.15.33:0/513396038 shutdown_connections 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb49480 0x55cebcb4b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb48f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 --2- 172.21.15.33:0/513396038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb3e870 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 -- 172.21.15.33:0/513396038 >> 172.21.15.33:0/513396038 conn(0x55cebcb3a7b0 msgr2=0x55cebcb3ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 -- 172.21.15.33:0/513396038 shutdown_connections 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.242+0000 7f7aafb76740 1 -- 172.21.15.33:0/513396038 wait complete. 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 -- start start 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 Processor -- start 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 0x55cebcb46290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cebcb4c2b0 con 0x55cebcb3e490 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cebca3db20 con 0x55cebcb49480 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cebcb4c430 con 0x55cebcb3edb0 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aafb76740 10 client.0 _ll_register_callbacks cb 0x55cebca3d530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39696/0 (socket says 172.21.15.33:39696) 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 -- 172.21.15.33:0/4045374580 learned_addr learned my addr 172.21.15.33:0/4045374580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 0x55cebcb46290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 -- 172.21.15.33:0/4045374580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 msgr2=0x55cebcb46290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 0x55cebcb46290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 -- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 msgr2=0x55cebcb469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7aa659c640 1 -- 172.21.15.33:0/4045374580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebc7b5920 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.243+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 0x55cebcb46290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aa659c640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009230 tx=0x7f7a980316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659f311fd6290216 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98040070 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7a98004240 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cebca61500 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a98035040 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cebc881d30 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cebca61500 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a98004860 con 0x55cebcb3e490 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 10 client.15849 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:40.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 20 client.15849 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcb59610 0x55cebcb5ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 4 client.15849 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.244+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cebcab10f0 con 0x55cebcb59610 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.245+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcb59610 0x55cebcb5ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.245+0000 7f7a8dffb640 10 client.15849 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.245+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcb59610 0x55cebcb5ba00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d048ef6107973b2d server_cookie=7f8a078f7fc6690c in_seq=0 out_seq=0 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7a8dffb640 1 -- 172.21.15.33:0/4045374580 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cebcab10f0 con 0x55cebcb59610 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7a8dffb640 10 client.15849 handle_command_reply: tid=0 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 client.15849 shutdown 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 msgr2=0x55cebcb48f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009230 tx=0x7f7a980316d0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 shutdown_connections 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcb59610 0x55cebcb5ba00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3edb0 0x55cebcb46290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4045374580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3e490 0x55cebcb48f30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7a8dffb640 0 client.15849 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 >> 172.21.15.33:0/4045374580 conn(0x55cebcb3a7b0 msgr2=0x55cebcb405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 shutdown_connections 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 1 -- 172.21.15.33:0/4045374580 wait complete. 2024-04-23T18:27:40.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.246+0000 7f7aafb76740 20 client.15849 trim_cache size 0 max 16384 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 Processor -- start 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 -- start start 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81afe80 con 0x55cebcb3edb0 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b19c0 con 0x55cebcb49480 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7aafb76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b0550 con 0x55cebcb3e490 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39698/0 (socket says 172.21.15.33:39698) 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.263+0000 7f7a9ffff640 1 -- 172.21.15.33:0/4076087037 learned_addr learned my addr 172.21.15.33:0/4076087037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 -- 172.21.15.33:0/4076087037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 msgr2=0x55cebcb46250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aa659c640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb46250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 -- 172.21.15.33:0/4076087037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 msgr2=0x55cebcb612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 -- 172.21.15.33:0/4076087037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cebcb3e290 con 0x55cebcb3edb0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aa659c640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb46250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a90004a10 tx=0x7f7a9000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706bd0c92b02f0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a8ffff640 1 -- 172.21.15.33:0/4076087037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9009cd20 con 0x55cebcb3edb0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a8ffff640 1 -- 172.21.15.33:0/4076087037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a900a10d0 con 0x55cebcb3edb0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7a8ffff640 1 -- 172.21.15.33:0/4076087037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90099390 con 0x55cebcb3edb0 2024-04-23T18:27:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aafb76740 1 -- 172.21.15.33:0/4076087037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 msgr2=0x55cebcb48ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.264+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a90004a10 tx=0x7f7a9000ce60 comp rx=0 tx=0).stop 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- 172.21.15.33:0/4076087037 shutdown_connections 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb49480 0x55cebcb612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb3edb0 0x55cebcb48ad0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- 172.21.15.33:0/4076087037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb46250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- 172.21.15.33:0/4076087037 >> 172.21.15.33:0/4076087037 conn(0x55cebcb3a7b0 msgr2=0x55cebcb399d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- 172.21.15.33:0/4076087037 shutdown_connections 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- 172.21.15.33:0/4076087037 wait complete. 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- start start 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 Processor -- start 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb43620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb43b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b12f0 con 0x55cebcb49480 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b0c20 con 0x55cebcb3edb0 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81ade60 con 0x55cebcb3e490 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aafb76740 10 client.0 _ll_register_callbacks cb 0x7f7a98032d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aa6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39706/0 (socket says 172.21.15.33:39706) 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/3335693847 learned_addr learned my addr 172.21.15.33:0/3335693847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.265+0000 7f7aa659c640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb43620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a9ffff640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb43b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/3335693847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 msgr2=0x55cebcb43620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb43620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 msgr2=0x55cebcb43b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb43b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 -- 172.21.15.33:0/3335693847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cebcbbf080 con 0x55cebcb49480 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aa6d9d640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00092a0 tx=0x7f7aa0005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e323db0e0179372c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0034060 con 0x55cebcb49480 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7aa0005210 con 0x55cebcb49480 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a900a10d0 con 0x55cebcb49480 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7aa0005410 con 0x55cebcb49480 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aafb76740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-04-23T18:27:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cebcbc02a0 con 0x55cebcb49480 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.266+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a900a10d0 con 0x55cebcb49480 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa000f040 con 0x55cebcb49480 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 10 client.15855 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 20 client.15855 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcbc07f0 0x55cebcbc2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 4 client.15855 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "532a83b8", "release": true, "await": true}] 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "532a83b8", "release": true, "await": true}) -- 0x55cebcaaaf60 con 0x55cebcbc07f0 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aa659c640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcbc07f0 0x55cebcbc2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7a8dffb640 10 client.15855 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.267+0000 7f7aa659c640 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcbc07f0 0x55cebcbc2c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5113bd7989ab4b62 server_cookie=a2d30bfb9025dd68 in_seq=0 out_seq=0 2024-04-23T18:27:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7a8dffb640 1 -- 172.21.15.33:0/3335693847 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55cebcaaaf60 con 0x55cebcbc07f0 2024-04-23T18:27:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7a8dffb640 10 client.15855 handle_command_reply: tid=0 2024-04-23T18:27:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 client.15855 shutdown 2024-04-23T18:27:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 msgr2=0x55cebcbbeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00092a0 tx=0x7f7aa0005ae0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 shutdown_connections 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cebcbc07f0 0x55cebcbc2c10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cebcb49480 0x55cebcbbeb40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cebcb3edb0 0x55cebcb43b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7a8dffb640 0 client.15855 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 --2- 172.21.15.33:0/3335693847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cebcb3e490 0x55cebcb43620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 >> 172.21.15.33:0/3335693847 conn(0x55cebcb3a7b0 msgr2=0x55cebcb399d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 shutdown_connections 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.272+0000 7f7aafb76740 1 -- 172.21.15.33:0/3335693847 wait complete. 2024-04-23T18:27:40.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aafb76740 20 client.15855 trim_cache size 0 max 16384 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 28, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 8, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "532a83b8": { 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 8, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:27:40.270 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 81.7, 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:27:40.271 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 msgr2=0x7f7a940a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 0x7f7a940a6390 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0010c40 tx=0x7f7aa0015000 comp rx=0 tx=0).stop 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 msgr2=0x7f7aa81a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7a9000e9f0 tx=0x7f7a9000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 shutdown_connections 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a940a3ed0 0x7f7a940a6390 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7aa819a7e0 0x7f7aa819ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aa8106170 0x7f7aa81a0bb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 --2- 172.21.15.33:0/1486873263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa81010a0 0x7f7aa81a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 >> 172.21.15.33:0/1486873263 conn(0x7f7aa80771d0 msgr2=0x7f7aa80fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 shutdown_connections 2024-04-23T18:27:40.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:27:40.273+0000 7f7aace69640 1 -- 172.21.15.33:0/1486873263 wait complete. 2024-04-23T18:27:40.273 INFO:tasks.workunit.client.0.smithi033.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-23T18:27:40.273 INFO:tasks.workunit.client.0.smithi033.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-23T18:27:40.274 INFO:tasks.workunit.client.0.smithi033.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-23T18:27:40.277 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '532a83b8', seconds elapsed: 22.8 2024-04-23T18:27:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:40 smithi106 ceph-mon[32716]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:40 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3947261957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:40 smithi122 ceph-mon[32546]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:40 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3947261957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:40 smithi033 ceph-mon[29274]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-23T18:27:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:40 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3947261957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:27:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:42 smithi106 ceph-mon[32716]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-23T18:27:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:42 smithi122 ceph-mon[32546]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-23T18:27:43.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:42 smithi033 ceph-mon[29274]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 938 B/s wr, 13 op/s 2024-04-23T18:27:45.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:44 smithi106 ceph-mon[32716]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-23T18:27:45.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:44 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:27:45.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:44 smithi122 ceph-mon[32546]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-23T18:27:45.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:44 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:27:45.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:44 smithi033 ceph-mon[29274]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-23T18:27:45.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:44 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:27:47.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:46 smithi106 ceph-mon[32716]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:27:47.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:46 smithi122 ceph-mon[32546]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:27:47.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:46 smithi033 ceph-mon[29274]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-23T18:27:49.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:48 smithi106 ceph-mon[32716]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-04-23T18:27:49.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:48 smithi122 ceph-mon[32546]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-04-23T18:27:49.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:48 smithi033 ceph-mon[29274]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 12 op/s 2024-04-23T18:27:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 -- 172.21.15.33:0/166118558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd434107290 msgr2=0x7fd43410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 --2- 172.21.15.33:0/166118558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd434107290 0x7fd43410ee60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009ca0 tx=0x7fd42c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 -- 172.21.15.33:0/166118558 shutdown_connections 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 --2- 172.21.15.33:0/166118558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd434107290 0x7fd43410ee60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 --2- 172.21.15.33:0/166118558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434102b70 0x7fd434106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 --2- 172.21.15.33:0/166118558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4341021c0 0x7fd4341025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 -- 172.21.15.33:0/166118558 >> 172.21.15.33:0/166118558 conn(0x7fd4340fda20 msgr2=0x7fd4340ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 -- 172.21.15.33:0/166118558 shutdown_connections 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.155+0000 7fd438820640 1 -- 172.21.15.33:0/166118558 wait complete. 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.156+0000 7fd438820640 1 Processor -- start 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.156+0000 7fd438820640 1 -- start start 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd434107290 0x7fd434074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 0x7fd4341a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112200 con 0x7fd4341021c0 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112080 con 0x7fd434074cd0 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd434112380 con 0x7fd434107290 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40448/0 (socket says 172.21.15.33:40448) 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 -- 172.21.15.33:0/1540379560 learned_addr learned my addr 172.21.15.33:0/1540379560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd432d9d640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 0x7fd4341a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd431d9b640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd434107290 0x7fd434074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 -- 172.21.15.33:0/1540379560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd434107290 msgr2=0x7fd434074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd434107290 0x7fd434074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 -- 172.21.15.33:0/1540379560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 msgr2=0x7fd4341a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 0x7fd4341a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 -- 172.21.15.33:0/1540379560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4341a6eb0 con 0x7fd4341021c0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd432d9d640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 0x7fd4341a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd43259c640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d770 tx=0x7fd42400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd42b7fe640 1 -- 172.21.15.33:0/1540379560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424017070 con 0x7fd4341021c0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd42b7fe640 1 -- 172.21.15.33:0/1540379560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4240054d0 con 0x7fd4341021c0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.157+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341a71a0 con 0x7fd4341021c0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.158+0000 7fd42b7fe640 1 -- 172.21.15.33:0/1540379560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424004c30 con 0x7fd4341021c0 2024-04-23T18:27:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.158+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4341a76f0 con 0x7fd4341021c0 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.160+0000 7fd42b7fe640 1 -- 172.21.15.33:0/1540379560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fd424004e10 con 0x7fd4341021c0 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.160+0000 7fd43b60c740 1 Processor -- start 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 -- start start 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48d7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625c483c970 con 0x5625c48d7f40 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5625c47d94a0 con 0x5625c48e2800 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43b60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5625c4838930 con 0x5625c48d7620 2024-04-23T18:27:51.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd42b7fe640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 0x7fd4200a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd432d9d640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 0x7fd4200a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd431d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40454/0 (socket says 172.21.15.33:40454) 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 -- 172.21.15.33:0/208539679 learned_addr learned my addr 172.21.15.33:0/208539679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd432d9d640 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48d7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd42b7fe640 1 -- 172.21.15.33:0/1540379560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd4240d05d0 con 0x7fd4341021c0 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 -- 172.21.15.33:0/208539679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 msgr2=0x5625c48d7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48d7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 -- 172.21.15.33:0/208539679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 msgr2=0x5625c48e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.161+0000 7fd43259c640 1 -- 172.21.15.33:0/208539679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c454f210 con 0x5625c48d7f40 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd432d9d640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 0x7fd4200a64b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009430 tx=0x7fd42c009880 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd43259c640 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd42409c040 tx=0x7fd4240c7450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7c72b53eaed98c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd428ff9640 1 -- 172.21.15.33:0/208539679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424020030 con 0x5625c48d7f40 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd428ff9640 1 -- 172.21.15.33:0/208539679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd424066e50 con 0x5625c48d7f40 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd428ff9640 1 -- 172.21.15.33:0/208539679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424022e30 con 0x5625c48d7f40 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.162+0000 7fd43b60c740 1 -- 172.21.15.33:0/208539679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 msgr2=0x5625c48e22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd42409c040 tx=0x7fd4240c7450 comp rx=0 tx=0).stop 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- 172.21.15.33:0/208539679 shutdown_connections 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48e22c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- 172.21.15.33:0/208539679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48d7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- 172.21.15.33:0/208539679 >> 172.21.15.33:0/208539679 conn(0x5625c48d3b30 msgr2=0x5625c48d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- 172.21.15.33:0/208539679 shutdown_connections 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- 172.21.15.33:0/208539679 wait complete. 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- start start 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 Processor -- start 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7f40 0x5625c48df440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48e2800 0x5625c48df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625c48e53d0 con 0x5625c48d7620 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.163+0000 7fd43b60c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5625c47d70b0 con 0x5625c48d7f40 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43b60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5625c48e5550 con 0x5625c48e2800 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43b60c740 10 client.0 _ll_register_callbacks cb 0x5625c47c4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40466/0 (socket says 172.21.15.33:40466) 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 -- 172.21.15.33:0/3356983363 learned_addr learned my addr 172.21.15.33:0/3356983363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd432d9d640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48e2800 0x5625c48df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd431d9b640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7f40 0x5625c48df440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 -- 172.21.15.33:0/3356983363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48e2800 msgr2=0x5625c48df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48e2800 0x5625c48df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 -- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7f40 msgr2=0x5625c48df440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7f40 0x5625c48df440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 -- 172.21.15.33:0/3356983363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c461b620 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43259c640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd4240a5170 tx=0x7fd42409bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efa046c848a86f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424020040 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd42409c5a0 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd424066e50 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd4240a06f0 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43b60c740 10 client.15867 fetch_fsmap learned FSMap version 29 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.164+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c48f1300 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd424066e50 con 0x5625c48d7620 2024-04-23T18:27:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd4240d00f0 con 0x5625c48d7620 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 10 client.15867 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 20 client.15867 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c48f2650 0x5625c48f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 4 client.15867 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c484a390 con 0x5625c48f2650 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.165+0000 7fd431d9b640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c48f2650 0x5625c48f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.166+0000 7fd431d9b640 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c48f2650 0x5625c48f4a40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=269675ea25d57e85 server_cookie=efa144274a81b1db in_seq=0 out_seq=0 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.166+0000 7fd419ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.166+0000 7fd419ffb640 1 -- 172.21.15.33:0/3356983363 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625c484a390 con 0x5625c48f2650 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.166+0000 7fd419ffb640 10 client.15867 handle_command_reply: tid=0 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.166+0000 7fd43b60c740 1 client.15867 shutdown 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 msgr2=0x5625c48e2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd4240a5170 tx=0x7fd42409bd80 comp rx=0 tx=0).stop 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 shutdown_connections 2024-04-23T18:27:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c48f2650 0x5625c48f4a40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48e2800 0x5625c48df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7f40 0x5625c48df440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 --2- 172.21.15.33:0/3356983363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7620 0x5625c48e2150 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 >> 172.21.15.33:0/3356983363 conn(0x5625c48d3b30 msgr2=0x5625c48d9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd419ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 shutdown_connections 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 1 -- 172.21.15.33:0/3356983363 wait complete. 2024-04-23T18:27:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.167+0000 7fd43b60c740 20 client.15867 trim_cache size 0 max 16384 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 Processor -- start 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 -- start start 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48fa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341afb80 con 0x5625c48d7f40 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341b16f0 con 0x5625c48e2800 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43b60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341b0250 con 0x5625c48d7620 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40474/0 (socket says 172.21.15.33:40474) 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 -- 172.21.15.33:0/721909315 learned_addr learned my addr 172.21.15.33:0/721909315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd43259c640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd432d9d640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48fa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 -- 172.21.15.33:0/721909315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 msgr2=0x5625c48ee290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48ee290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 -- 172.21.15.33:0/721909315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 msgr2=0x5625c48fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48fa1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 -- 172.21.15.33:0/721909315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c48dfd10 con 0x5625c48d7f40 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd432d9d640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.182+0000 7fd431d9b640 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd41c004700 tx=0x7fd41c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad765937b3f3aea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd41bfff640 1 -- 172.21.15.33:0/721909315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c003d20 con 0x5625c48d7f40 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd41bfff640 1 -- 172.21.15.33:0/721909315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0054d0 con 0x5625c48d7f40 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd41bfff640 1 -- 172.21.15.33:0/721909315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c01c070 con 0x5625c48d7f40 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 -- 172.21.15.33:0/721909315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 msgr2=0x5625c48d2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd41c004700 tx=0x7fd41c00cbe0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 -- 172.21.15.33:0/721909315 shutdown_connections 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48e2800 0x5625c48fa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48d7f40 0x5625c48d2b30 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 --2- 172.21.15.33:0/721909315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7620 0x5625c48ee290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 -- 172.21.15.33:0/721909315 >> 172.21.15.33:0/721909315 conn(0x5625c48d3b30 msgr2=0x5625c48d2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 -- 172.21.15.33:0/721909315 shutdown_connections 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.183+0000 7fd43b60c740 1 -- 172.21.15.33:0/721909315 wait complete. 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 -- start start 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 Processor -- start 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7620 0x5625c48e7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7f40 0x5625c48e7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341b0ff0 con 0x5625c48e2800 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341b0920 con 0x5625c48d7620 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341adb60 con 0x5625c48d7f40 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43b60c740 10 client.0 _ll_register_callbacks cb 0x7fd424005700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40488/0 (socket says 172.21.15.33:40488) 2024-04-23T18:27:51.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 -- 172.21.15.33:0/2195885367 learned_addr learned my addr 172.21.15.33:0/2195885367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd43259c640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7620 0x5625c48e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd431d9b640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7f40 0x5625c48e7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 -- 172.21.15.33:0/2195885367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7f40 msgr2=0x5625c48e7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7f40 0x5625c48e7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 -- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7620 msgr2=0x5625c48e7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7620 0x5625c48e7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.184+0000 7fd432d9d640 1 -- 172.21.15.33:0/2195885367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c4956f80 con 0x5625c48e2800 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd432d9d640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009ca0 tx=0x7fd42c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d62e84dfb786686 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c003d00 con 0x5625c48e2800 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fd42c004040 con 0x5625c48e2800 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41c0054d0 con 0x5625c48e2800 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd42c05f050 con 0x5625c48e2800 2024-04-23T18:27:51.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625c49581b0 con 0x5625c48e2800 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd41c0054d0 con 0x5625c48e2800 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fd42c02f9e0 con 0x5625c48e2800 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 20 client.15873 populate_metadata read hostname 'smithi033' 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c49589e0 0x5625c495ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 4 client.15873 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.185+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625c4844200 con 0x5625c49589e0 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.186+0000 7fd43259c640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c49589e0 0x5625c495ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.186+0000 7fd43259c640 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c49589e0 0x5625c495ae00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2009f12459736a0 server_cookie=4abcd56c3bcd487a in_seq=0 out_seq=0 2024-04-23T18:27:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.186+0000 7fd419ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd419ffb640 1 -- 172.21.15.33:0/2195885367 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625c4844200 con 0x5625c49589e0 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd419ffb640 10 client.15873 handle_command_reply: tid=0 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 client.15873 shutdown 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 msgr2=0x5625c4956a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009ca0 tx=0x7fd42c048000 comp rx=0 tx=0).stop 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 shutdown_connections 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625c49589e0 0x5625c495ae00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c48e2800 0x5625c4956a40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625c48d7f40 0x5625c48e7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 --2- 172.21.15.33:0/2195885367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5625c48d7620 0x5625c48e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd419ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 >> 172.21.15.33:0/2195885367 conn(0x5625c48d3b30 msgr2=0x5625c48d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 shutdown_connections 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 1 -- 172.21.15.33:0/2195885367 wait complete. 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd43b60c740 20 client.15873 trim_cache size 0 max 16384 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:27:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 msgr2=0x7fd4200a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 0x7fd4200a64b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009430 tx=0x7fd42c009880 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 msgr2=0x7fd434074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd42400d770 tx=0x7fd42400dc30 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.194+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 shutdown_connections 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4200a3ff0 0x7fd4200a64b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd434074cd0 0x7fd4341a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd434107290 0x7fd434074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 --2- 172.21.15.33:0/1540379560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4341021c0 0x7fd434074250 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 >> 172.21.15.33:0/1540379560 conn(0x7fd4340fda20 msgr2=0x7fd4340fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:27:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 shutdown_connections 2024-04-23T18:27:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:27:51.195+0000 7fd438820640 1 -- 172.21.15.33:0/1540379560 wait complete. 2024-04-23T18:27:51.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:50 smithi122 ceph-mon[32546]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:27:51.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:50 smithi033 ceph-mon[29274]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:27:51.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:50 smithi106 ceph-mon[32716]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:27:53.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:52 smithi122 ceph-mon[32546]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T18:27:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:52 smithi033 ceph-mon[29274]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T18:27:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:52 smithi106 ceph-mon[32716]: pgmap v44: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 18 op/s 2024-04-23T18:27:55.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:54 smithi122 ceph-mon[32546]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-23T18:27:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:54 smithi033 ceph-mon[29274]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-23T18:27:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:54 smithi106 ceph-mon[32716]: pgmap v45: 97 pgs: 97 active+clean; 1.5 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-04-23T18:27:57.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:56 smithi122 ceph-mon[32546]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-23T18:27:57.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:56 smithi033 ceph-mon[29274]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-23T18:27:57.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:56 smithi106 ceph-mon[32716]: pgmap v46: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 21 op/s 2024-04-23T18:27:59.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:27:58 smithi122 ceph-mon[32546]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:27:59.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:27:58 smithi033 ceph-mon[29274]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:27:59.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:27:58 smithi106 ceph-mon[32716]: pgmap v47: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:28:01.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:00 smithi122 ceph-mon[32546]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:28:01.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:00 smithi033 ceph-mon[29274]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:28:01.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:00 smithi106 ceph-mon[32716]: pgmap v48: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-23T18:28:03.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:02 smithi122 ceph-mon[32546]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-23T18:28:03.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:02 smithi122 ceph-mon[32546]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:28:03.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:02 smithi033 ceph-mon[29274]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-23T18:28:03.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:02 smithi033 ceph-mon[29274]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:28:03.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:02 smithi106 ceph-mon[32716]: pgmap v49: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 KiB/s wr, 23 op/s 2024-04-23T18:28:03.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:02 smithi106 ceph-mon[32716]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-23T18:28:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 -- 172.21.15.33:0/3394429948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c102b70 msgr2=0x7ffa7c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3394429948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c102b70 0x7ffa7c106bb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009900 tx=0x7ffa6402a410 comp rx=0 tx=0).stop 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 -- 172.21.15.33:0/3394429948 shutdown_connections 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3394429948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c107290 0x7ffa7c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3394429948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c102b70 0x7ffa7c106bb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3394429948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 -- 172.21.15.33:0/3394429948 >> 172.21.15.33:0/3394429948 conn(0x7ffa7c0fda20 msgr2=0x7ffa7c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 -- 172.21.15.33:0/3394429948 shutdown_connections 2024-04-23T18:28:04.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.294+0000 7ffa80df7640 1 -- 172.21.15.33:0/3394429948 wait complete. 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.295+0000 7ffa80df7640 1 Processor -- start 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.295+0000 7ffa80df7640 1 -- start start 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c107290 0x7ffa7c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c112220 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1120a0 con 0x7ffa7c1021c0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa80df7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1123a0 con 0x7ffa7c107290 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:57928/0 (socket says 172.21.15.33:57928) 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7a59c640 1 -- 172.21.15.33:0/3182930062 learned_addr learned my addr 172.21.15.33:0/3182930062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52712/0 (socket says 172.21.15.33:52712) 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c107290 0x7ffa7c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 -- 172.21.15.33:0/3182930062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c107290 msgr2=0x7ffa7c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c107290 0x7ffa7c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 -- 172.21.15.33:0/3182930062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 msgr2=0x7ffa7c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 -- 172.21.15.33:0/3182930062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c19da90 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa7ad9d640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009fd0 tx=0x7ffa7400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.296+0000 7ffa737fe640 1 -- 172.21.15.33:0/3182930062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7400ce20 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.297+0000 7ffa737fe640 1 -- 172.21.15.33:0/3182930062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa74005020 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.297+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c19dd80 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.297+0000 7ffa737fe640 1 -- 172.21.15.33:0/3182930062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74010690 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.297+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1a54f0 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.299+0000 7ffa737fe640 1 -- 172.21.15.33:0/3182930062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa740054d0 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.299+0000 7ffa83be7740 1 Processor -- start 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 -- start start 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38125a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb38132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb3808a970 con 0x55eb38125f40 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb380274a0 con 0x55eb38125620 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa83be7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb38086930 con 0x55eb38130800 2024-04-23T18:28:04.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52724/0 (socket says 172.21.15.33:52724) 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa7a59c640 1 -- 172.21.15.33:0/1462957067 learned_addr learned my addr 172.21.15.33:0/1462957067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa737fe640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 0x7ffa680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38125a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb38132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.300+0000 7ffa7ad9d640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 0x7ffa680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 -- 172.21.15.33:0/1462957067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 msgr2=0x55eb38132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb38132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 -- 172.21.15.33:0/1462957067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 msgr2=0x55eb38125a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38125a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 -- 172.21.15.33:0/1462957067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb37d9d210 con 0x55eb38125f40 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb38132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa737fe640 1 -- 172.21.15.33:0/3182930062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7ffa740d0e70 con 0x7ffa7c19cfa0 2024-04-23T18:28:04.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e10 tx=0x7ffa6c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bec84cb7a6c781 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa7ad9d640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 0x7ffa680a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa74099280 tx=0x7ffa740c5f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa70ff9640 1 -- 172.21.15.33:0/1462957067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c01e070 con 0x55eb38125f40 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa70ff9640 1 -- 172.21.15.33:0/1462957067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6c019ae0 con 0x55eb38125f40 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.301+0000 7ffa70ff9640 1 -- 172.21.15.33:0/1462957067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c00fb10 con 0x55eb38125f40 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- 172.21.15.33:0/1462957067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 msgr2=0x55eb381302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009e10 tx=0x7ffa6c011810 comp rx=0 tx=0).stop 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- 172.21.15.33:0/1462957067 shutdown_connections 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb38132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb381302c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 --2- 172.21.15.33:0/1462957067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38125a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- 172.21.15.33:0/1462957067 >> 172.21.15.33:0/1462957067 conn(0x55eb38121b30 msgr2=0x55eb38121f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- 172.21.15.33:0/1462957067 shutdown_connections 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- 172.21.15.33:0/1462957067 wait complete. 2024-04-23T18:28:04.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 -- start start 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.302+0000 7ffa83be7740 1 Processor -- start 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb381300f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb3812d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb38133430 con 0x55eb38125f40 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb380250b0 con 0x55eb38125620 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb381335b0 con 0x55eb38130800 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa83be7740 10 client.0 _ll_register_callbacks cb 0x55eb38012d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52728/0 (socket says 172.21.15.33:52728) 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 -- 172.21.15.33:0/2254033246 learned_addr learned my addr 172.21.15.33:0/2254033246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb381300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb3812d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 -- 172.21.15.33:0/2254033246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 msgr2=0x55eb3812d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb3812d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 -- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 msgr2=0x55eb381300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb381300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 -- 172.21.15.33:0/2254033246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb37e69620 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb3812d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00f730 tx=0x7ffa6c009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa298928913531f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.303+0000 7ffa61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c01e050 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7ffa6c0198c0 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c019ae0 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa6c033050 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb3813f300 con 0x55eb38125f40 2024-04-23T18:28:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6c019ae0 con 0x55eb38125f40 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ffa6c0046c0 con 0x55eb38125f40 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 20 client.15885 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38140700 0x55eb38142af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 4 client.15885 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb38098390 con 0x55eb38140700 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.304+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38140700 0x55eb38142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.305+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38140700 0x55eb38142af0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7443d9b9d408dd45 server_cookie=8ae8012ddfad3b53 in_seq=0 out_seq=0 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.305+0000 7ffa61ffb640 10 client.15885 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa61ffb640 1 -- 172.21.15.33:0/2254033246 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb38098390 con 0x55eb38140700 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa61ffb640 10 client.15885 handle_command_reply: tid=0 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 client.15885 shutdown 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 msgr2=0x55eb3812d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00f730 tx=0x7ffa6c009b60 comp rx=0 tx=0).stop 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 shutdown_connections 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38140700 0x55eb38142af0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38130800 0x55eb3812d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812d3e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa61ffb640 0 client.15885 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 --2- 172.21.15.33:0/2254033246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb381300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 >> 172.21.15.33:0/2254033246 conn(0x55eb38121b30 msgr2=0x55eb381278b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 shutdown_connections 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 1 -- 172.21.15.33:0/2254033246 wait complete. 2024-04-23T18:28:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.306+0000 7ffa83be7740 20 client.15885 trim_cache size 0 max 16384 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 Processor -- start 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 -- start start 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38121b30 0x55eb3813c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38125f40 0x55eb3812fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1ada50 con 0x55eb38125620 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1af5c0 con 0x55eb38121b30 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa83be7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1ae120 con 0x55eb38125f40 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa7a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52730/0 (socket says 172.21.15.33:52730) 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.321+0000 7ffa7a59c640 1 -- 172.21.15.33:0/3373559257 learned_addr learned my addr 172.21.15.33:0/3373559257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38125f40 0x55eb3812fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38121b30 0x55eb3813c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 -- 172.21.15.33:0/3373559257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38125f40 msgr2=0x55eb3812fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38125f40 0x55eb3812fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 -- 172.21.15.33:0/3373559257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38121b30 msgr2=0x55eb3813c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38121b30 0x55eb3813c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 -- 172.21.15.33:0/3373559257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb38132f70 con 0x55eb38125620 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009d60 tx=0x7ffa6c011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff02439f5f460251 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa63fff640 1 -- 172.21.15.33:0/3373559257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c015070 con 0x55eb38125620 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa63fff640 1 -- 172.21.15.33:0/3373559257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6c02a070 con 0x55eb38125620 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.322+0000 7ffa63fff640 1 -- 172.21.15.33:0/3373559257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c01e040 con 0x55eb38125620 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- 172.21.15.33:0/3373559257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 msgr2=0x55eb38128850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c009d60 tx=0x7ffa6c011e40 comp rx=0 tx=0).stop 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- 172.21.15.33:0/3373559257 shutdown_connections 2024-04-23T18:28:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38125f40 0x55eb3812fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125620 0x55eb38128850 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- 172.21.15.33:0/3373559257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38121b30 0x55eb3813c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- 172.21.15.33:0/3373559257 >> 172.21.15.33:0/3373559257 conn(0x55eb3812da80 msgr2=0x55eb38128140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- 172.21.15.33:0/3373559257 shutdown_connections 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- 172.21.15.33:0/3373559257 wait complete. 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- start start 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 Processor -- start 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38121b30 0x55eb38138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38129cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1aeec0 con 0x55eb38125f40 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1ae7f0 con 0x55eb38125620 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c1aba30 con 0x55eb38121b30 2024-04-23T18:28:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa83be7740 10 client.0 _ll_register_callbacks cb 0x7ffa6c009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.323+0000 7ffa79d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52732/0 (socket says 172.21.15.33:52732) 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 -- 172.21.15.33:0/134659979 learned_addr learned my addr 172.21.15.33:0/134659979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38121b30 0x55eb38138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38129cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 -- 172.21.15.33:0/134659979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38121b30 msgr2=0x55eb38138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38121b30 0x55eb38138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 -- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 msgr2=0x55eb38129cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38129cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 -- 172.21.15.33:0/134659979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3812a970 con 0x55eb38125f40 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38129cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa79d9b640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffa6402a9d0 tx=0x7ffa64040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4cb1cba1273a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64035040 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7ffa6404a840 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa6c02a070 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa640035f0 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa83be7740 10 client.15891 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.324+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55eb381a6a60 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa6c02a070 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7ffa6405d050 con 0x55eb38125f40 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 20 client.15891 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38130800 0x55eb381a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 4 client.15891 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb38092200 con 0x55eb38130800 2024-04-23T18:28:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.325+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38130800 0x55eb381a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.326+0000 7ffa61ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.326+0000 7ffa7a59c640 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38130800 0x55eb381a91a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a00d32f104fcd90 server_cookie=99eadee12df43b1c in_seq=0 out_seq=0 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa61ffb640 1 -- 172.21.15.33:0/134659979 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb38092200 con 0x55eb38130800 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa61ffb640 10 client.15891 handle_command_reply: tid=0 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 client.15891 shutdown 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 msgr2=0x55eb3812a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ffa6402a9d0 tx=0x7ffa64040040 comp rx=0 tx=0).stop 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 shutdown_connections 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb38130800 0x55eb381a91a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb38125f40 0x55eb3812a230 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa61ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb38125620 0x55eb38129cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 --2- 172.21.15.33:0/134659979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb38121b30 0x55eb38138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.333+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 >> 172.21.15.33:0/134659979 conn(0x55eb3812da80 msgr2=0x55eb38127830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 shutdown_connections 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa83be7740 1 -- 172.21.15.33:0/134659979 wait complete. 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa83be7740 20 client.15891 trim_cache size 0 max 16384 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:28:04.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 msgr2=0x7ffa680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 0x7ffa680a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa74099280 tx=0x7ffa740c5f90 comp rx=0 tx=0).stop 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 msgr2=0x7ffa7c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ffa74009fd0 tx=0x7ffa7400ef10 comp rx=0 tx=0).stop 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 shutdown_connections 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa680a3dc0 0x7ffa680a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa7c19cfa0 0x7ffa7c19d450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa7c107290 0x7ffa7c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 --2- 172.21.15.33:0/3182930062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa7c1021c0 0x7ffa7c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 >> 172.21.15.33:0/3182930062 conn(0x7ffa7c0fda20 msgr2=0x7ffa7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 shutdown_connections 2024-04-23T18:28:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:04.334+0000 7ffa80df7640 1 -- 172.21.15.33:0/3182930062 wait complete. 2024-04-23T18:28:05.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:04 smithi122 ceph-mon[32546]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:28:05.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:04 smithi033 ceph-mon[29274]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:28:05.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:04 smithi106 ceph-mon[32716]: pgmap v50: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:28:07.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:06 smithi106 ceph-mon[32716]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 40 op/s 2024-04-23T18:28:07.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:06 smithi122 ceph-mon[32546]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 40 op/s 2024-04-23T18:28:07.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:06 smithi033 ceph-mon[29274]: pgmap v51: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 42 KiB/s wr, 40 op/s 2024-04-23T18:28:09.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:08 smithi122 ceph-mon[32546]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 32 op/s 2024-04-23T18:28:09.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:08 smithi033 ceph-mon[29274]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 32 op/s 2024-04-23T18:28:09.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:08 smithi106 ceph-mon[32716]: pgmap v52: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 32 op/s 2024-04-23T18:28:11.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:10 smithi122 ceph-mon[32546]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:28:11.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:10 smithi033 ceph-mon[29274]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:28:11.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:10 smithi106 ceph-mon[32716]: pgmap v53: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 35 KiB/s wr, 31 op/s 2024-04-23T18:28:13.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:12 smithi122 ceph-mon[32546]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 KiB/s wr, 38 op/s 2024-04-23T18:28:13.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:12 smithi033 ceph-mon[29274]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 KiB/s wr, 38 op/s 2024-04-23T18:28:13.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:12 smithi106 ceph-mon[32716]: pgmap v54: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 KiB/s wr, 38 op/s 2024-04-23T18:28:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:14 smithi122 ceph-mon[32546]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:28:15.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:14 smithi033 ceph-mon[29274]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:28:15.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:14 smithi106 ceph-mon[32716]: pgmap v55: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:28:17.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:16 smithi122 ceph-mon[32546]: pgmap v56: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 36 op/s 2024-04-23T18:28:17.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:16 smithi033 ceph-mon[29274]: pgmap v56: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 36 op/s 2024-04-23T18:28:17.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:16 smithi106 ceph-mon[32716]: pgmap v56: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 36 op/s 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 -- 172.21.15.33:0/4181819768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 msgr2=0x7fb72010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 --2- 172.21.15.33:0/4181819768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb72010bfe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb710009900 tx=0x7fb71002a510 comp rx=0 tx=0).stop 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 -- 172.21.15.33:0/4181819768 shutdown_connections 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 --2- 172.21.15.33:0/4181819768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 0x7fb72010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 --2- 172.21.15.33:0/4181819768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb72010bfe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 --2- 172.21.15.33:0/4181819768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 0x7fb7201035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 -- 172.21.15.33:0/4181819768 >> 172.21.15.33:0/4181819768 conn(0x7fb720077fd0 msgr2=0x7fb7200783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 -- 172.21.15.33:0/4181819768 shutdown_connections 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.434+0000 7fb727a7c640 1 -- 172.21.15.33:0/4181819768 wait complete. 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 Processor -- start 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 -- start start 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 0x7fb720100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 0x7fb7201018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201121b0 con 0x7fb720103b00 2024-04-23T18:28:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb720112030 con 0x7fb72010c520 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb727a7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb720112330 con 0x7fb7201031e0 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb725017640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb725017640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54516/0 (socket says 172.21.15.33:54516) 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb725017640 1 -- 172.21.15.33:0/1824374455 learned_addr learned my addr 172.21.15.33:0/1824374455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb725818640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 0x7fb720100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.435+0000 7fb726019640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 0x7fb7201018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 -- 172.21.15.33:0/1824374455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 msgr2=0x7fb720100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 0x7fb720100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 -- 172.21.15.33:0/1824374455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 msgr2=0x7fb7201018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 0x7fb7201018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 -- 172.21.15.33:0/1824374455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7201a6bb0 con 0x7fb720103b00 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb725017640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb710005940 tx=0x7fb710004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb716ffd640 1 -- 172.21.15.33:0/1824374455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71003f030 con 0x7fb720103b00 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb716ffd640 1 -- 172.21.15.33:0/1824374455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb710003aa0 con 0x7fb720103b00 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7201a6e40 con 0x7fb720103b00 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb716ffd640 1 -- 172.21.15.33:0/1824374455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710035070 con 0x7fb720103b00 2024-04-23T18:28:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.436+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7201a72b0 con 0x7fb720103b00 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.438+0000 7fb716ffd640 1 -- 172.21.15.33:0/1824374455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb710042030 con 0x7fb720103b00 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 Processor -- start 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 -- start start 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d74622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d7464bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a1d73bc970 con 0x55a1d7457620 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1d73594a0 con 0x55a1d7457f40 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb72a797740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a1d73b8930 con 0x55a1d7462800 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.439+0000 7fb726019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb716ffd640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 0x7fb70c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54530/0 (socket says 172.21.15.33:54530) 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 -- 172.21.15.33:0/1062463840 learned_addr learned my addr 172.21.15.33:0/1062463840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb725017640 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d7464bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb725818640 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d74622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 0x7fb70c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 -- 172.21.15.33:0/1062463840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 msgr2=0x55a1d7464bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d7464bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 -- 172.21.15.33:0/1062463840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 msgr2=0x55a1d74622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d74622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 -- 172.21.15.33:0/1062463840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d70cf210 con 0x55a1d7457620 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb716ffd640 1 -- 172.21.15.33:0/1824374455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb7100f0d90 con 0x7fb720103b00 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb718005c40 tx=0x7fb718013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5938e02d652977c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb70bfff640 1 -- 172.21.15.33:0/1062463840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718022020 con 0x55a1d7457620 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb726019640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 0x7fb70c0a6330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb718023000 tx=0x7fb718021000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb70bfff640 1 -- 172.21.15.33:0/1062463840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7180134e0 con 0x55a1d7457620 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.440+0000 7fb70bfff640 1 -- 172.21.15.33:0/1062463840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180109a0 con 0x55a1d7457620 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 -- 172.21.15.33:0/1062463840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 msgr2=0x55a1d7457a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb718005c40 tx=0x7fb718013f50 comp rx=0 tx=0).stop 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 -- 172.21.15.33:0/1062463840 shutdown_connections 2024-04-23T18:28:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d7464bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d74622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 --2- 172.21.15.33:0/1062463840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d7457a00 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 -- 172.21.15.33:0/1062463840 >> 172.21.15.33:0/1062463840 conn(0x55a1d7453b30 msgr2=0x55a1d7453f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 -- 172.21.15.33:0/1062463840 shutdown_connections 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.441+0000 7fb72a797740 1 -- 172.21.15.33:0/1062463840 wait complete. 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 -- start start 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 Processor -- start 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d745f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d745f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a1d7465430 con 0x55a1d7457620 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1d73570b0 con 0x55a1d7457f40 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a1d74655b0 con 0x55a1d7462800 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb72a797740 10 client.0 _ll_register_callbacks cb 0x55a1d7344d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb725818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb725818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54540/0 (socket says 172.21.15.33:54540) 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb725818640 1 -- 172.21.15.33:0/2884767038 learned_addr learned my addr 172.21.15.33:0/2884767038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.442+0000 7fb725017640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d745f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb726019640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d745f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 -- 172.21.15.33:0/2884767038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 msgr2=0x55a1d745f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d745f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 -- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 msgr2=0x55a1d745f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d745f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 -- 172.21.15.33:0/2884767038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d719b620 con 0x55a1d7457620 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725017640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d745f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb725818640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b990 tx=0x7fb71c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2548edd718ca6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb7097fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c011440 con 0x55a1d7457620 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1d73ca390 con 0x55a1d7457620 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb71c0115e0 con 0x55a1d7457620 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0117e0 con 0x55a1d7457620 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb72a797740 10 client.15903 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:17.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.443+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1d7471360 con 0x55a1d7457620 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a1d73ca390 con 0x55a1d7457620 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb71c01b940 con 0x55a1d7457620 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 10 client.15903 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 20 client.15903 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d7472870 0x55a1d7474cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 4 client.15903 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1d73c4200 con 0x55a1d7472870 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.444+0000 7fb725017640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d7472870 0x55a1d7474cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.445+0000 7fb725017640 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d7472870 0x55a1d7474cc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f10d0d5d5ba07eec server_cookie=6fd8d0e947959af8 in_seq=0 out_seq=0 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.445+0000 7fb7097fa640 10 client.15903 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.445+0000 7fb7097fa640 1 -- 172.21.15.33:0/2884767038 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1d73c4200 con 0x55a1d7472870 2024-04-23T18:28:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.445+0000 7fb7097fa640 10 client.15903 handle_command_reply: tid=0 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.445+0000 7fb72a797740 1 client.15903 shutdown 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 msgr2=0x55a1d74620f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00b990 tx=0x7fb71c00be50 comp rx=0 tx=0).stop 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 shutdown_connections 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d7472870 0x55a1d7474cc0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7462800 0x55a1d745f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457f40 0x55a1d745f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb7097fa640 0 client.15903 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 --2- 172.21.15.33:0/2884767038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457620 0x55a1d74620f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 >> 172.21.15.33:0/2884767038 conn(0x55a1d7453b30 msgr2=0x55a1d74598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 shutdown_connections 2024-04-23T18:28:17.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 1 -- 172.21.15.33:0/2884767038 wait complete. 2024-04-23T18:28:17.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.446+0000 7fb72a797740 20 client.15903 trim_cache size 0 max 16384 2024-04-23T18:28:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.461+0000 7fb72a797740 1 Processor -- start 2024-04-23T18:28:17.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.461+0000 7fb72a797740 1 -- start start 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 0x55a1d746e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457f40 0x55a1d745a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201afc80 con 0x55a1d7462800 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b17f0 con 0x55a1d7457620 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb72a797740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b0350 con 0x55a1d7457f40 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54550/0 (socket says 172.21.15.33:54550) 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 -- 172.21.15.33:0/4163510582 learned_addr learned my addr 172.21.15.33:0/4163510582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb725818640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 0x55a1d746e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb725017640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457f40 0x55a1d745a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 -- 172.21.15.33:0/4163510582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457f40 msgr2=0x55a1d745a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457f40 0x55a1d745a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 -- 172.21.15.33:0/4163510582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 msgr2=0x55a1d746e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 0x55a1d746e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 -- 172.21.15.33:0/4163510582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1d745fc80 con 0x55a1d7462800 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb725818640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 0x55a1d746e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.462+0000 7fb726019640 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb718005c40 tx=0x7fb718020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fdd04a04f7d58b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb70b7fe640 1 -- 172.21.15.33:0/4163510582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71800fde0 con 0x55a1d7462800 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb70b7fe640 1 -- 172.21.15.33:0/4163510582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb718012cd0 con 0x55a1d7462800 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb70b7fe640 1 -- 172.21.15.33:0/4163510582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180109a0 con 0x55a1d7462800 2024-04-23T18:28:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 -- 172.21.15.33:0/4163510582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 msgr2=0x55a1d7461c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb718005c40 tx=0x7fb718020040 comp rx=0 tx=0).stop 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 -- 172.21.15.33:0/4163510582 shutdown_connections 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7462800 0x55a1d7461c20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457f40 0x55a1d745a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 --2- 172.21.15.33:0/4163510582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7457620 0x55a1d746e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 -- 172.21.15.33:0/4163510582 >> 172.21.15.33:0/4163510582 conn(0x55a1d7453b30 msgr2=0x55a1d7459860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 -- 172.21.15.33:0/4163510582 shutdown_connections 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.463+0000 7fb72a797740 1 -- 172.21.15.33:0/4163510582 wait complete. 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 -- start start 2024-04-23T18:28:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 Processor -- start 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 0x55a1d746abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 0x55a1d745c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b10f0 con 0x55a1d7457f40 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201b0a20 con 0x55a1d7462800 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201adc60 con 0x55a1d7457620 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb72a797740 10 client.0 _ll_register_callbacks cb 0x7fb71c0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb725017640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb725017640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54562/0 (socket says 172.21.15.33:54562) 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb725017640 1 -- 172.21.15.33:0/3714497608 learned_addr learned my addr 172.21.15.33:0/3714497608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:17.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.464+0000 7fb725818640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 0x55a1d746abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb726019640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 0x55a1d745c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 -- 172.21.15.33:0/3714497608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 msgr2=0x55a1d746abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 0x55a1d746abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 -- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 msgr2=0x55a1d745c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 0x55a1d745c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 -- 172.21.15.33:0/3714497608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1d745cb40 con 0x55a1d7457f40 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb726019640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 0x55a1d745c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725818640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 0x55a1d746abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb725017640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b9760 tx=0x7fb7100f31e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a6477dd8a99010 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:17.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710049040 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb7100b9400 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb718012cd0 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7100c5950 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1d74d8eb0 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb718012cd0 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb7100ee7a0 con 0x55a1d7457f40 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:17.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 20 client.15909 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d74d95c0 0x55a1d74db9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 4 client.15909 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.465+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1d73405f0 con 0x55a1d74d95c0 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.466+0000 7fb725818640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d74d95c0 0x55a1d74db9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.466+0000 7fb7097fa640 10 client.15909 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:17.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.466+0000 7fb725818640 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d74d95c0 0x55a1d74db9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676bdb3b965e8407 server_cookie=a85ec3bc2990823d in_seq=0 out_seq=0 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.473+0000 7fb7097fa640 1 -- 172.21.15.33:0/3714497608 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1d73405f0 con 0x55a1d74d95c0 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.473+0000 7fb7097fa640 10 client.15909 handle_command_reply: tid=0 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.473+0000 7fb72a797740 1 client.15909 shutdown 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.473+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 msgr2=0x55a1d745bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.473+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b9760 tx=0x7fb7100f31e0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 shutdown_connections 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a1d74d95c0 0x55a1d74db9e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1d7462800 0x55a1d745c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1d7457f40 0x55a1d745bdd0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb7097fa640 0 client.15909 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:17.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 --2- 172.21.15.33:0/3714497608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a1d7457620 0x55a1d746abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 >> 172.21.15.33:0/3714497608 conn(0x55a1d7453b30 msgr2=0x55a1d7458fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 shutdown_connections 2024-04-23T18:28:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 1 -- 172.21.15.33:0/3714497608 wait complete. 2024-04-23T18:28:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb72a797740 20 client.15909 trim_cache size 0 max 16384 2024-04-23T18:28:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:28:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:28:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:28:17.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 msgr2=0x7fb70c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 0x7fb70c0a6330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb718023000 tx=0x7fb718021000 comp rx=0 tx=0).stop 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 msgr2=0x7fb720101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb710005940 tx=0x7fb710004270 comp rx=0 tx=0).stop 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 shutdown_connections 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb70c0a3e70 0x7fb70c0a6330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb72010c520 0x7fb7201018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb720103b00 0x7fb720101370 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 --2- 172.21.15.33:0/1824374455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7201031e0 0x7fb720100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 >> 172.21.15.33:0/1824374455 conn(0x7fb720077fd0 msgr2=0x7fb7200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 shutdown_connections 2024-04-23T18:28:17.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:17.474+0000 7fb727a7c640 1 -- 172.21.15.33:0/1824374455 wait complete. 2024-04-23T18:28:19.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:18 smithi122 ceph-mon[32546]: pgmap v57: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:19.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:18 smithi033 ceph-mon[29274]: pgmap v57: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:18 smithi106 ceph-mon[32716]: pgmap v57: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:21.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:20 smithi122 ceph-mon[32546]: pgmap v58: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:21.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:20 smithi033 ceph-mon[29274]: pgmap v58: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:20 smithi106 ceph-mon[32716]: pgmap v58: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.2 KiB/s wr, 12 op/s 2024-04-23T18:28:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:22 smithi122 ceph-mon[32546]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T18:28:23.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:22 smithi033 ceph-mon[29274]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T18:28:23.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:22 smithi106 ceph-mon[32716]: pgmap v59: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 20 op/s 2024-04-23T18:28:25.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:24 smithi122 ceph-mon[32546]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-23T18:28:25.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:24 smithi033 ceph-mon[29274]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-23T18:28:25.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:24 smithi106 ceph-mon[32716]: pgmap v60: 97 pgs: 97 active+clean; 1.7 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 14 op/s 2024-04-23T18:28:27.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:26 smithi122 ceph-mon[32546]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-04-23T18:28:27.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:26 smithi033 ceph-mon[29274]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-04-23T18:28:27.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:26 smithi106 ceph-mon[32716]: pgmap v61: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 20 op/s 2024-04-23T18:28:29.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:28 smithi122 ceph-mon[32546]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:29.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:28 smithi033 ceph-mon[29274]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:29.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:28 smithi106 ceph-mon[32716]: pgmap v62: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 -- 172.21.15.33:0/1987664631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c103200 msgr2=0x7f7f1c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 --2- 172.21.15.33:0/1987664631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c103200 0x7f7f1c1035e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009900 tx=0x7f7f1002a480 comp rx=0 tx=0).stop 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 -- 172.21.15.33:0/1987664631 shutdown_connections 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 --2- 172.21.15.33:0/1987664631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 --2- 172.21.15.33:0/1987664631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 0x7f7f1c10c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 --2- 172.21.15.33:0/1987664631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c103200 0x7f7f1c1035e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 -- 172.21.15.33:0/1987664631 >> 172.21.15.33:0/1987664631 conn(0x7f7f1c0771d0 msgr2=0x7f7f1c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.574+0000 7f7f22394640 1 -- 172.21.15.33:0/1987664631 shutdown_connections 2024-04-23T18:28:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.575+0000 7f7f22394640 1 -- 172.21.15.33:0/1987664631 wait complete. 2024-04-23T18:28:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.575+0000 7f7f22394640 1 Processor -- start 2024-04-23T18:28:30.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.575+0000 7f7f22394640 1 -- start start 2024-04-23T18:28:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c103200 0x7f7f1c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 0x7f7f1c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c112270 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1120f0 con 0x7f7f1c103b20 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f22394640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1123f0 con 0x7f7f1c103200 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56250/0 (socket says 172.21.15.33:56250) 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 -- 172.21.15.33:0/1359764258 learned_addr learned my addr 172.21.15.33:0/1359764258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c103200 0x7f7f1c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 0x7f7f1c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 -- 172.21.15.33:0/1359764258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c103200 msgr2=0x7f7f1c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c103200 0x7f7f1c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 -- 172.21.15.33:0/1359764258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 msgr2=0x7f7f1c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 0x7f7f1c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 -- 172.21.15.33:0/1359764258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c19d710 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.576+0000 7f7f20931640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9d0 tx=0x7f7f1400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.577+0000 7f7f197fa640 1 -- 172.21.15.33:0/1359764258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400cd80 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.577+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19da00 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.577+0000 7f7f197fa640 1 -- 172.21.15.33:0/1359764258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f14005020 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.577+0000 7f7f197fa640 1 -- 172.21.15.33:0/1359764258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f140116d0 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.577+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a5290 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f197fa640 1 -- 172.21.15.33:0/1359764258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7f140051d0 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 Processor -- start 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 -- start start 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x5587376f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558737704c10 0x558737707000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55873765a970 con 0x5587376f5e40 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5587375f74a0 con 0x5587376f5520 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f197fa640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 0x7f7f040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.579+0000 7f7f25188740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558737656930 con 0x558737704c10 2024-04-23T18:28:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f20931640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 0x7f7f040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56254/0 (socket says 172.21.15.33:56254) 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 -- 172.21.15.33:0/2722811601 learned_addr learned my addr 172.21.15.33:0/2722811601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x5587376f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558737704c10 0x558737707000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f197fa640 1 -- 172.21.15.33:0/1359764258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7f140d0c50 con 0x7f7f1c10c5d0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 -- 172.21.15.33:0/2722811601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558737704c10 msgr2=0x558737707000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558737704c10 0x558737707000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 -- 172.21.15.33:0/2722811601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 msgr2=0x5587376f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x5587376f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 -- 172.21.15.33:0/2722811601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873736d210 con 0x5587376f5e40 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f20931640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 0x7f7f040a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f14099620 tx=0x7f7f1401d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f10004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402fa96ef17bcde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f027fc640 1 -- 172.21.15.33:0/2722811601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10035040 con 0x5587376f5e40 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.580+0000 7f7f027fc640 1 -- 172.21.15.33:0/2722811601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f100054f0 con 0x5587376f5e40 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f027fc640 1 -- 172.21.15.33:0/2722811601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f100035f0 con 0x5587376f5e40 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 -- 172.21.15.33:0/2722811601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 msgr2=0x5587377046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7f10000c00 tx=0x7f7f10004170 comp rx=0 tx=0).stop 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 -- 172.21.15.33:0/2722811601 shutdown_connections 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558737704c10 0x558737707000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587377046d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 --2- 172.21.15.33:0/2722811601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x5587376f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 -- 172.21.15.33:0/2722811601 >> 172.21.15.33:0/2722811601 conn(0x5587376f1b30 msgr2=0x5587376f1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 -- 172.21.15.33:0/2722811601 shutdown_connections 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.581+0000 7f7f25188740 1 -- 172.21.15.33:0/2722811601 wait complete. 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 -- start start 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 Processor -- start 2024-04-23T18:28:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x558737701770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x558737701cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5587377077e0 con 0x5587376f5520 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5587375f50b0 con 0x558737704c10 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558737707960 con 0x5587376f5e40 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f25188740 10 client.0 _ll_register_callbacks cb 0x5587375e2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56258/0 (socket says 172.21.15.33:56258) 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/1060698094 learned_addr learned my addr 172.21.15.33:0/1060698094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x558737701770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.582+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x558737701cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/1060698094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 msgr2=0x558737701770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x558737701770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 msgr2=0x558737701cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x558737701cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/1060698094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558737439620 con 0x5587376f5520 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005890 tx=0x7f7f0c009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c0268d7548f50b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7efb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c019040 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7f0c0047a0 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f100054f0 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f0c0049a0 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f25188740 10 client.15921 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5587377136c0 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.583+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f100054f0 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f7f0c01f520 con 0x5587376f5520 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 10 client.15921 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 10 client.15921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 20 client.15921 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558737714b80 0x558737716f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 4 client.15921 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558737668390 con 0x558737714b80 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.584+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558737714b80 0x558737716f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.585+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558737714b80 0x558737716f70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=903709e159f99f28 server_cookie=d1519b62c41dfad1 in_seq=0 out_seq=0 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.585+0000 7f7efb7fe640 10 client.15921 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.585+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1060698094 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558737668390 con 0x558737714b80 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.585+0000 7f7efb7fe640 10 client.15921 handle_command_reply: tid=0 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.585+0000 7f7f25188740 1 client.15921 shutdown 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 msgr2=0x558737704480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c005890 tx=0x7f7f0c009b40 comp rx=0 tx=0).stop 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 shutdown_connections 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558737714b80 0x558737716f70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x558737701cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x558737701770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7efb7fe640 0 client.15921 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 --2- 172.21.15.33:0/1060698094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5520 0x558737704480 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 >> 172.21.15.33:0/1060698094 conn(0x5587376f1b30 msgr2=0x5587376f79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 shutdown_connections 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 1 -- 172.21.15.33:0/1060698094 wait complete. 2024-04-23T18:28:30.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.586+0000 7f7f25188740 20 client.15921 trim_cache size 0 max 16384 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 Processor -- start 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 -- start start 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x558737710710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x5587376fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1ab9a0 con 0x558737704c10 2024-04-23T18:28:30.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1ad510 con 0x5587376f5520 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f25188740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1ac070 con 0x5587376f5e40 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56264/0 (socket says 172.21.15.33:56264) 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/3199864735 learned_addr learned my addr 172.21.15.33:0/3199864735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x558737710710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x5587376fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/3199864735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 msgr2=0x5587376fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x5587376fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/3199864735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 msgr2=0x558737710710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x558737710710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.601+0000 7f7f1b7fe640 1 -- 172.21.15.33:0/3199864735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587376f1960 con 0x558737704c10 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c010a40 tx=0x7f7f0c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a53276429343a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f01ffb640 1 -- 172.21.15.33:0/3199864735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c02f040 con 0x558737704c10 2024-04-23T18:28:30.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f01ffb640 1 -- 172.21.15.33:0/3199864735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c014070 con 0x558737704c10 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f01ffb640 1 -- 172.21.15.33:0/3199864735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c01f400 con 0x558737704c10 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 -- 172.21.15.33:0/3199864735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 msgr2=0x558737703fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c010a40 tx=0x7f7f0c005f90 comp rx=0 tx=0).stop 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 -- 172.21.15.33:0/3199864735 shutdown_connections 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558737704c10 0x558737703fb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5e40 0x5587376fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 --2- 172.21.15.33:0/3199864735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5587376f5520 0x558737710710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 -- 172.21.15.33:0/3199864735 >> 172.21.15.33:0/3199864735 conn(0x5587376f1b30 msgr2=0x5587376fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 -- 172.21.15.33:0/3199864735 shutdown_connections 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.602+0000 7f7f25188740 1 -- 172.21.15.33:0/3199864735 wait complete. 2024-04-23T18:28:30.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 -- start start 2024-04-23T18:28:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 Processor -- start 2024-04-23T18:28:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5520 0x55873770ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x5587376fe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1ace10 con 0x5587376f5e40 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1ac740 con 0x558737704c10 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c1a9980 con 0x5587376f5520 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f25188740 10 client.0 _ll_register_callbacks cb 0x7f7f0c010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56278/0 (socket says 172.21.15.33:56278) 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.603+0000 7f7f1bfff640 1 -- 172.21.15.33:0/1152270334 learned_addr learned my addr 172.21.15.33:0/1152270334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5520 0x55873770ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x5587376fe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 -- 172.21.15.33:0/1152270334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5520 msgr2=0x55873770ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5520 0x55873770ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 -- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 msgr2=0x5587376fe540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x5587376fe540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 -- 172.21.15.33:0/1152270334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587376feb00 con 0x5587376f5e40 2024-04-23T18:28:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f1bfff640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009230 tx=0x7f7f10004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05e802056e46cff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7efb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10035040 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7f10053160 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0c014070 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7f10053360 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f25188740 10 client.15927 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.604+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55873777aed0 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c014070 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f7f10003bd0 con 0x5587376f5e40 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 20 client.15927 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55873777b2c0 0x55873777d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 4 client.15927 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558737702040 con 0x55873777b2c0 2024-04-23T18:28:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.605+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55873777b2c0 0x55873777d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.606+0000 7f7f1b7fe640 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55873777b2c0 0x55873777d6b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eea17fa19f23a67 server_cookie=78182d08960a2163 in_seq=0 out_seq=0 2024-04-23T18:28:30.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.606+0000 7f7efb7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7efb7fe640 1 -- 172.21.15.33:0/1152270334 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558737702040 con 0x55873777b2c0 2024-04-23T18:28:30.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7efb7fe640 10 client.15927 handle_command_reply: tid=0 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 client.15927 shutdown 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 msgr2=0x5587376fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009230 tx=0x7f7f10004b50 comp rx=0 tx=0).stop 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 shutdown_connections 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55873777b2c0 0x55873777d6b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558737704c10 0x5587376fe540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5587376f5e40 0x5587376fe000 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 --2- 172.21.15.33:0/1152270334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587376f5520 0x55873770ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7efb7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.613+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 >> 172.21.15.33:0/1152270334 conn(0x5587376f1b30 msgr2=0x5587376fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 shutdown_connections 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f25188740 1 -- 172.21.15.33:0/1152270334 wait complete. 2024-04-23T18:28:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f25188740 20 client.15927 trim_cache size 0 max 16384 2024-04-23T18:28:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:28:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:28:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:28:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 msgr2=0x7f7f040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 0x7f7f040a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f14099620 tx=0x7f7f1401d000 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 msgr2=0x7f7f1c19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400e9d0 tx=0x7f7f1400ee90 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 shutdown_connections 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7f040a3dc0 0x7f7f040a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7f1c10c5d0 0x7f7f1c19cfe0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c103b20 0x7f7f1c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 --2- 172.21.15.33:0/1359764258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f1c103200 0x7f7f1c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 >> 172.21.15.33:0/1359764258 conn(0x7f7f1c0771d0 msgr2=0x7f7f1c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 shutdown_connections 2024-04-23T18:28:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:30.614+0000 7f7f22394640 1 -- 172.21.15.33:0/1359764258 wait complete. 2024-04-23T18:28:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:30 smithi122 ceph-mon[32546]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:31.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:30 smithi033 ceph-mon[29274]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:31.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:30 smithi106 ceph-mon[32716]: pgmap v63: 97 pgs: 97 active+clean; 1.8 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:33.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:32 smithi122 ceph-mon[32546]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:33.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:32 smithi033 ceph-mon[29274]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:33.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:32 smithi106 ceph-mon[32716]: pgmap v64: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:34 smithi122 ceph-mon[32546]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-04-23T18:28:35.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:34 smithi033 ceph-mon[29274]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-04-23T18:28:35.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:34 smithi106 ceph-mon[32716]: pgmap v65: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 13 op/s 2024-04-23T18:28:36.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:35 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:36.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:35 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:36.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:35 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:37.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:36 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:37.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:37.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:36 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: pgmap v66: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:28:37.349 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:36 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:28:39.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:38 smithi033 ceph-mon[29274]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:39.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:38 smithi106 ceph-mon[32716]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:39.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:38 smithi122 ceph-mon[32546]: pgmap v67: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:40 smithi033 ceph-mon[29274]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:41.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:40 smithi106 ceph-mon[32716]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:41.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:40 smithi122 ceph-mon[32546]: pgmap v68: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:43.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:42 smithi033 ceph-mon[29274]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:43.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:42 smithi106 ceph-mon[32716]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:43.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:42 smithi122 ceph-mon[32546]: pgmap v69: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:43.699 INFO:tasks.workunit.client.0.smithi033.stdout:1/600: open fd = 3 2024-04-23T18:28:43.699 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:2/600: open fd = 3 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:3/600: open fd = 3 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:4/600: open fd = 3 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:5/600: open fd = 3 2024-04-23T18:28:43.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:6/600: open fd = 3 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:7/600: open fd = 3 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:8/600: open fd = 3 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:9/600: open fd = 3 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.701 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:10/600: open fd = 3 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:11/600: open fd = 3 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:12/600: open fd = 3 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:13/600: open fd = 3 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:14/600: open fd = 3 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:15/600: open fd = 3 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.703 INFO:tasks.workunit.client.0.smithi033.stdout:16/600: open fd = 3 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:17/600: open fd = 3 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:18/600: open fd = 3 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.704 INFO:tasks.workunit.client.0.smithi033.stdout:19/600: open fd = 3 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:20/600: open fd = 3 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:21/600: open fd = 3 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:22/600: open fd = 3 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.705 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:23/600: open fd = 3 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:24/600: open fd = 3 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:25/600: open fd = 3 2024-04-23T18:28:43.706 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:26/600: open fd = 3 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:27/600: open fd = 3 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:28/600: open fd = 3 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.707 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:29/600: open fd = 3 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:30/600: open fd = 3 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:31/600: open fd = 3 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:32/600: open fd = 3 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.708 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:33/600: open fd = 3 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:34/600: open fd = 3 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:35/600: open fd = 3 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:36/600: open fd = 3 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.709 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:37/600: open fd = 3 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:38/600: open fd = 3 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:39/600: open fd = 3 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.710 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:40/600: open fd = 3 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:41/600: open fd = 3 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:42/600: open fd = 3 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:43/600: open fd = 3 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.711 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:44/600: open fd = 3 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:45/600: open fd = 3 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:46/600: open fd = 3 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:47/600: open fd = 3 2024-04-23T18:28:43.712 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:48/600: open fd = 3 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:49/600: open fd = 3 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.713 INFO:tasks.workunit.client.0.smithi033.stdout:50/600: open fd = 3 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:51/600: open fd = 3 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.714 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:52/600: open fd = 3 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:53/600: open fd = 3 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.715 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:54/600: open fd = 3 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:55/600: open fd = 3 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:56/600: open fd = 3 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:57/600: open fd = 3 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:58/600: open fd = 3 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:59/600: open fd = 3 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:60/600: open fd = 3 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:61/600: open fd = 3 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:62/600: open fd = 3 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:63/600: open fd = 3 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:28:43.719 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:28:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 -- 172.21.15.33:0/3053154728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938107290 msgr2=0x7f793810ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 --2- 172.21.15.33:0/3053154728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938107290 0x7f793810ee60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ca0 tx=0x7f793002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 -- 172.21.15.33:0/3053154728 shutdown_connections 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 --2- 172.21.15.33:0/3053154728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7938107290 0x7f793810ee60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 --2- 172.21.15.33:0/3053154728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938102b70 0x7f7938106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 --2- 172.21.15.33:0/3053154728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f79381025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 -- 172.21.15.33:0/3053154728 >> 172.21.15.33:0/3053154728 conn(0x7f79380fda20 msgr2=0x7f79380ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 -- 172.21.15.33:0/3053154728 shutdown_connections 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.713+0000 7f793dd20640 1 -- 172.21.15.33:0/3053154728 wait complete. 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 Processor -- start 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 -- start start 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f7938075e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938107290 0x7f7938074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.714+0000 7f793dd20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112200 con 0x7f79380747e0 2024-04-23T18:28:43.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f793dd20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112080 con 0x7f7938107290 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f793dd20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112380 con 0x7f79381021c0 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57384/0 (socket says 172.21.15.33:57384) 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 -- 172.21.15.33:0/1497642205 learned_addr learned my addr 172.21.15.33:0/1497642205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7936ffd640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938107290 0x7f7938074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f79377fe640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f7938075e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 -- 172.21.15.33:0/1497642205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 msgr2=0x7f7938075e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f7938075e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 -- 172.21.15.33:0/1497642205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938107290 msgr2=0x7f7938074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938107290 0x7f7938074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 -- 172.21.15.33:0/1497642205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79381ab320 con 0x7f79380747e0 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f79377fe640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f7938075e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7937fff640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7930005cc0 tx=0x7f7930031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f7934ff9640 1 -- 172.21.15.33:0/1497642205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930003b70 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.715+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79381ab5b0 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.716+0000 7f7934ff9640 1 -- 172.21.15.33:0/1497642205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79300318f0 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.716+0000 7f7934ff9640 1 -- 172.21.15.33:0/1497642205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793002f950 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.716+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381abb50 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7934ff9640 1 -- 172.21.15.33:0/1497642205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f793002fb10 con 0x7f79380747e0 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 Processor -- start 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 -- start start 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28f40 0x564f6fc332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc35bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f6fb8d970 con 0x564f6fc28620 2024-04-23T18:28:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564f6fb2a4a0 con 0x564f6fc33800 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7940a3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564f6fb89930 con 0x564f6fc28f40 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57396/0 (socket says 172.21.15.33:57396) 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.718+0000 7f7936ffd640 1 -- 172.21.15.33:0/3349444848 learned_addr learned my addr 172.21.15.33:0/3349444848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f79377fe640 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28f40 0x564f6fc332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7934ff9640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 0x7f79200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 -- 172.21.15.33:0/3349444848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28f40 msgr2=0x564f6fc332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28f40 0x564f6fc332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 -- 172.21.15.33:0/3349444848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 msgr2=0x564f6fc35bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc35bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 -- 172.21.15.33:0/3349444848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6f8a0210 con 0x564f6fc28620 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7937fff640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 0x7f79200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7936ffd640 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f792c009ca0 tx=0x7f792c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739859285fab3e06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f7934ff9640 1 -- 172.21.15.33:0/1497642205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f79300f1c70 con 0x7f79380747e0 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f791dffb640 1 -- 172.21.15.33:0/3349444848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c009e50 con 0x564f6fc28620 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f791dffb640 1 -- 172.21.15.33:0/3349444848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c005020 con 0x564f6fc28620 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.719+0000 7f791dffb640 1 -- 172.21.15.33:0/3349444848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c004b20 con 0x564f6fc28620 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- 172.21.15.33:0/3349444848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 msgr2=0x564f6fc28a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7937fff640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 0x7f79200a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79300bda30 tx=0x7f79300b9990 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f792c009ca0 tx=0x7f792c011810 comp rx=0 tx=0).stop 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- 172.21.15.33:0/3349444848 shutdown_connections 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc35bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28f40 0x564f6fc332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 --2- 172.21.15.33:0/3349444848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28620 0x564f6fc28a00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- 172.21.15.33:0/3349444848 >> 172.21.15.33:0/3349444848 conn(0x564f6fc24b30 msgr2=0x564f6fc24f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- 172.21.15.33:0/3349444848 shutdown_connections 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- 172.21.15.33:0/3349444848 wait complete. 2024-04-23T18:28:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.720+0000 7f7940a3a740 1 -- start start 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 Processor -- start 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc33160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc309b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564f6fc363d0 con 0x564f6fc28f40 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564f6fb280b0 con 0x564f6fc33800 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564f6fc36550 con 0x564f6fc28620 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7940a3a740 10 client.0 _ll_register_callbacks cb 0x564f6fb15d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57412/0 (socket says 172.21.15.33:57412) 2024-04-23T18:28:43.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 -- 172.21.15.33:0/458389541 learned_addr learned my addr 172.21.15.33:0/458389541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7936ffd640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7936ffd640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc33160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 -- 172.21.15.33:0/458389541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 msgr2=0x564f6fc33160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc33160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 -- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 msgr2=0x564f6fc309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc309b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 -- 172.21.15.33:0/458389541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6f96c620 con 0x564f6fc28f40 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7936ffd640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f79377fe640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7928013790 tx=0x7f7928013b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d838591d577921 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.721+0000 7f7916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928020060 con 0x564f6fc28f40 2024-04-23T18:28:43.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f7928011dd0 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f792c005020 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f792800f450 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 10 client.15939 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f6fc42360 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c005020 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f792801a030 con 0x564f6fc28f40 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 20 client.15939 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fc43890 0x564f6fc45c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 4 client.15939 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.722+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f6fb9b390 con 0x564f6fc43890 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.723+0000 7f7936ffd640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fc43890 0x564f6fc45c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.723+0000 7f7936ffd640 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fc43890 0x564f6fc45c80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d219c377ea9eada server_cookie=f85d08d0e66b504f in_seq=0 out_seq=0 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.723+0000 7f7916ffd640 10 client.15939 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7916ffd640 1 -- 172.21.15.33:0/458389541 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f6fb9b390 con 0x564f6fc43890 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7916ffd640 10 client.15939 handle_command_reply: tid=0 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 client.15939 shutdown 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 msgr2=0x564f6fc30450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7928013790 tx=0x7f7928013b90 comp rx=0 tx=0).stop 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 shutdown_connections 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fc43890 0x564f6fc45c80 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc33800 0x564f6fc309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc28f40 0x564f6fc30450 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 --2- 172.21.15.33:0/458389541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc33160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 >> 172.21.15.33:0/458389541 conn(0x564f6fc24b30 msgr2=0x564f6fc2a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7916ffd640 0 client.15939 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 shutdown_connections 2024-04-23T18:28:43.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 1 -- 172.21.15.33:0/458389541 wait complete. 2024-04-23T18:28:43.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.724+0000 7f7940a3a740 20 client.15939 trim_cache size 0 max 16384 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 Processor -- start 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 -- start start 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc3f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc2b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b4070 con 0x564f6fc33800 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b5be0 con 0x564f6fc28f40 2024-04-23T18:28:43.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.739+0000 7f7940a3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b4740 con 0x564f6fc28620 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57420/0 (socket says 172.21.15.33:57420) 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 -- 172.21.15.33:0/2758456626 learned_addr learned my addr 172.21.15.33:0/2758456626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f79377fe640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc2b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 -- 172.21.15.33:0/2758456626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 msgr2=0x564f6fc3f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc3f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 -- 172.21.15.33:0/2758456626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 msgr2=0x564f6fc2b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc2b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 -- 172.21.15.33:0/2758456626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f6fc24660 con 0x564f6fc33800 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f79377fe640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc2b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f7936ffd640 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f792c009820 tx=0x7f792c004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d95350f6749ab92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f791d7fa640 1 -- 172.21.15.33:0/2758456626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c0055c0 con 0x564f6fc33800 2024-04-23T18:28:43.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f791d7fa640 1 -- 172.21.15.33:0/2758456626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c032020 con 0x564f6fc33800 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.740+0000 7f791d7fa640 1 -- 172.21.15.33:0/2758456626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c02dd40 con 0x564f6fc33800 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- 172.21.15.33:0/2758456626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 msgr2=0x564f6fc32c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f792c009820 tx=0x7f792c004580 comp rx=0 tx=0).stop 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- 172.21.15.33:0/2758456626 shutdown_connections 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fc32c90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc2b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 --2- 172.21.15.33:0/2758456626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc3f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- 172.21.15.33:0/2758456626 >> 172.21.15.33:0/2758456626 conn(0x564f6fc24b30 msgr2=0x564f6fc2b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- 172.21.15.33:0/2758456626 shutdown_connections 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- 172.21.15.33:0/2758456626 wait complete. 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 -- start start 2024-04-23T18:28:43.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.741+0000 7f7940a3a740 1 Processor -- start 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc37920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc37e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b54e0 con 0x564f6fc33800 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b4e10 con 0x564f6fc28f40 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79381b2050 con 0x564f6fc28620 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7940a3a740 10 client.0 _ll_register_callbacks cb 0x7f7928009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f79377fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc37e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57422/0 (socket says 172.21.15.33:57422) 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 -- 172.21.15.33:0/1299037969 learned_addr learned my addr 172.21.15.33:0/1299037969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc37920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 -- 172.21.15.33:0/1299037969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 msgr2=0x564f6fc37920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc37920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 -- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 msgr2=0x564f6fc37e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc37e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 -- 172.21.15.33:0/1299037969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f6fca93c0 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7936ffd640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f792c0053d0 tx=0x7f792c030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f9a5865be125b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.742+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c02d2c0 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f792c033040 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f792c032020 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f792c049050 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f6fcaa640 con 0x564f6fc33800 2024-04-23T18:28:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c032020 con 0x564f6fc33800 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f792c004e40 con 0x564f6fc33800 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 20 client.15945 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fcaaab0 0x564f6fcaced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 4 client.15945 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f6fb95200 con 0x564f6fcaaab0 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.743+0000 7f79377fe640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fcaaab0 0x564f6fcaced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.744+0000 7f7916ffd640 10 client.15945 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.744+0000 7f79377fe640 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fcaaab0 0x564f6fcaced0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225aabfe543957f5 server_cookie=45cc46d7354dcd57 in_seq=0 out_seq=0 2024-04-23T18:28:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7916ffd640 1 -- 172.21.15.33:0/1299037969 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f6fb95200 con 0x564f6fcaaab0 2024-04-23T18:28:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7916ffd640 10 client.15945 handle_command_reply: tid=0 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7940a3a740 1 client.15945 shutdown 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 msgr2=0x564f6fca8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f792c0053d0 tx=0x7f792c030040 comp rx=0 tx=0).stop 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 shutdown_connections 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.751+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564f6fcaaab0 0x564f6fcaced0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564f6fc33800 0x564f6fca8e50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564f6fc28f40 0x564f6fc37e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7916ffd640 0 client.15945 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 --2- 172.21.15.33:0/1299037969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f6fc28620 0x564f6fc37920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 >> 172.21.15.33:0/1299037969 conn(0x564f6fc24b30 msgr2=0x564f6fc23cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 shutdown_connections 2024-04-23T18:28:43.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 1 -- 172.21.15.33:0/1299037969 wait complete. 2024-04-23T18:28:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f7940a3a740 20 client.15945 trim_cache size 0 max 16384 2024-04-23T18:28:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:28:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:28:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:28:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 msgr2=0x7f79200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 0x7f79200a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79300bda30 tx=0x7f79300b9990 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 msgr2=0x7f7938074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7930005cc0 tx=0x7f7930031700 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 shutdown_connections 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f79200a3dc0 0x7f79200a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79380747e0 0x7f7938074c70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7938107290 0x7f7938074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 --2- 172.21.15.33:0/1497642205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79381021c0 0x7f7938075e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 >> 172.21.15.33:0/1497642205 conn(0x7f79380fda20 msgr2=0x7f79380fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.752+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 shutdown_connections 2024-04-23T18:28:43.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:43.753+0000 7f793dd20640 1 -- 172.21.15.33:0/1497642205 wait complete. 2024-04-23T18:28:45.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:44 smithi033 ceph-mon[29274]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:45.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:44 smithi106 ceph-mon[32716]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:45.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:44 smithi122 ceph-mon[32546]: pgmap v70: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:47.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:46 smithi033 ceph-mon[29274]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-23T18:28:47.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:46 smithi106 ceph-mon[32716]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-23T18:28:47.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:46 smithi122 ceph-mon[32546]: pgmap v71: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 18 KiB/s wr, 22 op/s 2024-04-23T18:28:49.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:48 smithi033 ceph-mon[29274]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:49.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:48 smithi106 ceph-mon[32716]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:49.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:48 smithi122 ceph-mon[32546]: pgmap v72: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:51.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:50 smithi033 ceph-mon[29274]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:51.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:50 smithi106 ceph-mon[32716]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:51.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:50 smithi122 ceph-mon[32546]: pgmap v73: 97 pgs: 97 active+clean; 2.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:52 smithi033 ceph-mon[29274]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:52 smithi106 ceph-mon[32716]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:52 smithi122 ceph-mon[32546]: pgmap v74: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:28:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:54 smithi033 ceph-mon[29274]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:54 smithi106 ceph-mon[32716]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:55.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:54 smithi122 ceph-mon[32546]: pgmap v75: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 -- 172.21.15.33:0/3997345859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f8106170 msgr2=0x7f24f810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 --2- 172.21.15.33:0/3997345859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f8106170 0x7f24f810dd40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f24f0009900 tx=0x7f24f002a410 comp rx=0 tx=0).stop 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 -- 172.21.15.33:0/3997345859 shutdown_connections 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 --2- 172.21.15.33:0/3997345859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f8106170 0x7f24f810dd40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 --2- 172.21.15.33:0/3997345859 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8101aa0 0x7f24f8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 --2- 172.21.15.33:0/3997345859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f81010f0 0x7f24f81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 -- 172.21.15.33:0/3997345859 >> 172.21.15.33:0/3997345859 conn(0x7f24f8077fd0 msgr2=0x7f24f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 -- 172.21.15.33:0/3997345859 shutdown_connections 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.854+0000 7f24fee05640 1 -- 172.21.15.33:0/3997345859 wait complete. 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 Processor -- start 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 -- start start 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 0x7f24f8197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f819be40 0x7f24f819c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81110e0 con 0x7f24f81010f0 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8110f60 con 0x7f24f8106170 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fee05640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24f8111260 con 0x7f24f819be40 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34222/0 (socket says 172.21.15.33:34222) 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 -- 172.21.15.33:0/1342159474 learned_addr learned my addr 172.21.15.33:0/1342159474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:56.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f819be40 0x7f24f819c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 0x7f24f8197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 -- 172.21.15.33:0/1342159474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f819be40 msgr2=0x7f24f819c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f819be40 0x7f24f819c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 -- 172.21.15.33:0/1342159474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 msgr2=0x7f24f8197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 0x7f24f8197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24fcba1640 1 -- 172.21.15.33:0/1342159474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f819c930 con 0x7f24f81010f0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.855+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 0x7f24f8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d770 tx=0x7f24e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24f5ffb640 1 -- 172.21.15.33:0/1342159474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0017070 con 0x7f24f81010f0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24f5ffb640 1 -- 172.21.15.33:0/1342159474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e00054d0 con 0x7f24f81010f0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f819cc20 con 0x7f24f81010f0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24f5ffb640 1 -- 172.21.15.33:0/1342159474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e0004c30 con 0x7f24f81010f0 2024-04-23T18:28:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.856+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f81a4410 con 0x7f24f81010f0 2024-04-23T18:28:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.858+0000 7f24f5ffb640 1 -- 172.21.15.33:0/1342159474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f24e0004e10 con 0x7f24f81010f0 2024-04-23T18:28:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 Processor -- start 2024-04-23T18:28:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24f5ffb640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 -- start start 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a67520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24f5ffb640 1 -- 172.21.15.33:0/1342159474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f24e00d0240 con 0x7f24f81010f0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c8d39cc970 con 0x55c8d3a67af0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c8d39694a0 con 0x55c8d3a76950 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f2501bf6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c8d39c8930 con 0x55c8d3a68cd0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34238/0 (socket says 172.21.15.33:34238) 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.859+0000 7f24f7fff640 1 -- 172.21.15.33:0/1173245536 learned_addr learned my addr 172.21.15.33:0/1173245536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a67520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 0x7f24e40a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f0006010 tx=0x7f24f0032b80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 -- 172.21.15.33:0/1173245536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 msgr2=0x55c8d3a67520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a67520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 -- 172.21.15.33:0/1173245536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 msgr2=0x55c8d3a78d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a78d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 -- 172.21.15.33:0/1173245536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d36df210 con 0x55c8d3a67af0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a67520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009f50 tx=0x7f24e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac4f99788ab1697 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24eeffd640 1 -- 172.21.15.33:0/1173245536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8016070 con 0x55c8d3a67af0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24eeffd640 1 -- 172.21.15.33:0/1173245536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e801bb10 con 0x55c8d3a67af0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.860+0000 7f24eeffd640 1 -- 172.21.15.33:0/1173245536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8012cf0 con 0x55c8d3a67af0 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- 172.21.15.33:0/1173245536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 msgr2=0x55c8d3a67f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009f50 tx=0x7f24e8010800 comp rx=0 tx=0).stop 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- 172.21.15.33:0/1173245536 shutdown_connections 2024-04-23T18:28:56.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a78d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a67f70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1173245536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a67520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- 172.21.15.33:0/1173245536 >> 172.21.15.33:0/1173245536 conn(0x55c8d3a63b30 msgr2=0x55c8d3a63f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- 172.21.15.33:0/1173245536 shutdown_connections 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- 172.21.15.33:0/1173245536 wait complete. 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 -- start start 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.861+0000 7f2501bf6740 1 Processor -- start 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a67af0 0x55c8d3a76200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3a797e0 con 0x55c8d3a68cd0 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c8d39670b0 con 0x55c8d3a67af0 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c8d3a79960 con 0x55c8d3a76950 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f2501bf6740 10 client.0 _ll_register_callbacks cb 0x55c8d3954d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34254/0 (socket says 172.21.15.33:34254) 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 -- 172.21.15.33:0/1571858854 learned_addr learned my addr 172.21.15.33:0/1571858854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a67af0 0x55c8d3a76200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 -- 172.21.15.33:0/1571858854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 msgr2=0x55c8d3a73b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a73b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 -- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a67af0 msgr2=0x55c8d3a76200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a67af0 0x55c8d3a76200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24f7fff640 1 -- 172.21.15.33:0/1571858854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d37ab620 con 0x55c8d3a68cd0 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.862+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24f7fff640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009f20 tx=0x7f24e8012680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661af13e4d3e26e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:56.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8016030 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f24e8012d40 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e801bb10 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24e8004a40 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 10 client.15957 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8d3a857d0 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e801bb10 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f24e800f040 con 0x55c8d3a68cd0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 20 client.15957 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3a86c30 0x55c8d3a89020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 4 client.15957 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.863+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8d39da390 con 0x55c8d3a86c30 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.864+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3a86c30 0x55c8d3a89020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.864+0000 7f24fcba1640 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3a86c30 0x55c8d3a89020 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45d9d104a724298 server_cookie=639e375f42134ce3 in_seq=0 out_seq=0 2024-04-23T18:28:56.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.864+0000 7f24d7fff640 10 client.15957 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f24d7fff640 1 -- 172.21.15.33:0/1571858854 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8d39da390 con 0x55c8d3a86c30 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f24d7fff640 10 client.15957 handle_command_reply: tid=0 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 client.15957 shutdown 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 msgr2=0x55c8d3a73560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009f20 tx=0x7f24e8012680 comp rx=0 tx=0).stop 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 shutdown_connections 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3a86c30 0x55c8d3a89020 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a73b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a73560 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f24d7fff640 0 client.15957 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 --2- 172.21.15.33:0/1571858854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a67af0 0x55c8d3a76200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 >> 172.21.15.33:0/1571858854 conn(0x55c8d3a63b30 msgr2=0x55c8d3a699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 shutdown_connections 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 1 -- 172.21.15.33:0/1571858854 wait complete. 2024-04-23T18:28:56.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.865+0000 7f2501bf6740 20 client.15957 trim_cache size 0 max 16384 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.880+0000 7f2501bf6740 1 Processor -- start 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.880+0000 7f2501bf6740 1 -- start start 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a67af0 0x55c8d3a826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ab8a0 con 0x55c8d3a68cd0 2024-04-23T18:28:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ad410 con 0x55c8d3a76950 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f2501bf6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81abf70 con 0x55c8d3a67af0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34268/0 (socket says 172.21.15.33:34268) 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 -- 172.21.15.33:0/549604799 learned_addr learned my addr 172.21.15.33:0/549604799 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24fcba1640 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a67af0 0x55c8d3a826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 -- 172.21.15.33:0/549604799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a67af0 msgr2=0x55c8d3a826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a67af0 0x55c8d3a826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 -- 172.21.15.33:0/549604799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 msgr2=0x55c8d3a70ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a70ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 -- 172.21.15.33:0/549604799 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8d3a71ab0 con 0x55c8d3a68cd0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24f7fff640 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f24e8010d00 tx=0x7f24e8009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4039f0c0b6f3619f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24ee7fc640 1 -- 172.21.15.33:0/549604799 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8016070 con 0x55c8d3a68cd0 2024-04-23T18:28:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24ee7fc640 1 -- 172.21.15.33:0/549604799 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e801b460 con 0x55c8d3a68cd0 2024-04-23T18:28:56.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.881+0000 7f24ee7fc640 1 -- 172.21.15.33:0/549604799 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8003d90 con 0x55c8d3a68cd0 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- 172.21.15.33:0/549604799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 msgr2=0x55c8d3a63430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f24e8010d00 tx=0x7f24e8009c50 comp rx=0 tx=0).stop 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- 172.21.15.33:0/549604799 shutdown_connections 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a76950 0x55c8d3a70ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a63430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 --2- 172.21.15.33:0/549604799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a67af0 0x55c8d3a826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- 172.21.15.33:0/549604799 >> 172.21.15.33:0/549604799 conn(0x55c8d3a63b30 msgr2=0x55c8d3a75620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- 172.21.15.33:0/549604799 shutdown_connections 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- 172.21.15.33:0/549604799 wait complete. 2024-04-23T18:28:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 -- start start 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.882+0000 7f2501bf6740 1 Processor -- start 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a7f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a72e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81acd10 con 0x55c8d3a67af0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81ac640 con 0x55c8d3a68cd0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24f81a9880 con 0x55c8d3a76950 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f2501bf6740 10 client.0 _ll_register_callbacks cb 0x7f24e8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34272/0 (socket says 172.21.15.33:34272) 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 -- 172.21.15.33:0/3639339569 learned_addr learned my addr 172.21.15.33:0/3639339569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fd3a2640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a72e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24f7fff640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a7f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 -- 172.21.15.33:0/3639339569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 msgr2=0x55c8d3a72e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a72e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 -- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 msgr2=0x55c8d3a7f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a7f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 -- 172.21.15.33:0/3639339569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8d3a73430 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24f7fff640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a7f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24fcba1640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009610 tx=0x7f24e009c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd61bc8f41c1c606 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.883+0000 7f24d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e00ba070 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f24e00cdc20 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24e801b460 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f24e0020040 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 10 client.15963 fetch_fsmap learned FSMap version 29 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8d3a8e480 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e00a8080 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f24e009c4e0 con 0x55c8d3a67af0 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:28:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 20 client.15963 populate_metadata read hostname 'smithi033' 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3aed4e0 0x55c8d3aef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 4 client.15963 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8d39d4200 con 0x55c8d3aed4e0 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.884+0000 7f24f7fff640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3aed4e0 0x55c8d3aef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.885+0000 7f24f7fff640 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3aed4e0 0x55c8d3aef900 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a70e100e7c4ea10 server_cookie=704a8fe49d3140fa in_seq=0 out_seq=0 2024-04-23T18:28:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.885+0000 7f24d7fff640 10 client.15963 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f24d7fff640 1 -- 172.21.15.33:0/3639339569 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8d39d4200 con 0x55c8d3aed4e0 2024-04-23T18:28:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f24d7fff640 10 client.15963 handle_command_reply: tid=0 2024-04-23T18:28:56.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 client.15963 shutdown 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 msgr2=0x55c8d3a7ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009610 tx=0x7f24e009c070 comp rx=0 tx=0).stop 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 shutdown_connections 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8d3aed4e0 0x55c8d3aef900 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8d3a76950 0x55c8d3a72e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8d3a68cd0 0x55c8d3a7f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 --2- 172.21.15.33:0/3639339569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8d3a67af0 0x55c8d3a7ebf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 >> 172.21.15.33:0/3639339569 conn(0x55c8d3a63b30 msgr2=0x55c8d3a75620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f24d7fff640 0 client.15963 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 shutdown_connections 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 1 -- 172.21.15.33:0/3639339569 wait complete. 2024-04-23T18:28:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.892+0000 7f2501bf6740 20 client.15963 trim_cache size 0 max 16384 2024-04-23T18:28:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:28:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:28:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:28:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:28:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 msgr2=0x7f24e40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 0x7f24e40a6300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24f0006010 tx=0x7f24f0032b80 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 msgr2=0x7f24f8197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f24e000d770 tx=0x7f24e000dc30 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 shutdown_connections 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f24e40a3e40 0x7f24e40a6300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24f819be40 0x7f24f819c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24f8106170 0x7f24f8197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 --2- 172.21.15.33:0/1342159474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f81010f0 0x7f24f8197580 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 >> 172.21.15.33:0/1342159474 conn(0x7f24f8077fd0 msgr2=0x7f24f80fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 shutdown_connections 2024-04-23T18:28:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:28:56.893+0000 7f24fee05640 1 -- 172.21.15.33:0/1342159474 wait complete. 2024-04-23T18:28:57.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:56 smithi106 ceph-mon[32716]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:57.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:56 smithi122 ceph-mon[32546]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:57.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:56 smithi033 ceph-mon[29274]: pgmap v76: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-04-23T18:28:59.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:28:58 smithi106 ceph-mon[32716]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:28:59.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:28:58 smithi122 ceph-mon[32546]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:28:59.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:28:58 smithi033 ceph-mon[29274]: pgmap v77: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:29:00.692 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:29:00.692 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T18:29:00.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 -- 172.21.15.33:0/1054571137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c106190 msgr2=0x7ff63c10dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 --2- 172.21.15.33:0/1054571137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c106190 0x7ff63c10dd60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff634009ca0 tx=0x7ff63402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 -- 172.21.15.33:0/1054571137 shutdown_connections 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 --2- 172.21.15.33:0/1054571137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c106190 0x7ff63c10dd60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 --2- 172.21.15.33:0/1054571137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101ac0 0x7ff63c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 --2- 172.21.15.33:0/1054571137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c101110 0x7ff63c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 -- 172.21.15.33:0/1054571137 >> 172.21.15.33:0/1054571137 conn(0x7ff63c077fb0 msgr2=0x7ff63c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:00.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 -- 172.21.15.33:0/1054571137 shutdown_connections 2024-04-23T18:29:00.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.809+0000 7ff641b3d640 1 -- 172.21.15.33:0/1054571137 wait complete. 2024-04-23T18:29:00.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 Processor -- start 2024-04-23T18:29:00.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 -- start start 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 0x7ff63c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c106190 0x7ff63c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c1111d0 con 0x7ff63c19a190 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c111050 con 0x7ff63c106190 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff641b3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff63c111350 con 0x7ff63c101110 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34298/0 (socket says 172.21.15.33:34298) 2024-04-23T18:29:00.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 -- 172.21.15.33:0/465650891 learned_addr learned my addr 172.21.15.33:0/465650891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63b7fe640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 0x7ff63c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63affd640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c106190 0x7ff63c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 -- 172.21.15.33:0/465650891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 msgr2=0x7ff63c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 0x7ff63c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 -- 172.21.15.33:0/465650891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c106190 msgr2=0x7ff63c199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c106190 0x7ff63c199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63bfff640 1 -- 172.21.15.33:0/465650891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63c1a4f10 con 0x7ff63c19a190 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.810+0000 7ff63b7fe640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 0x7ff63c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff63bfff640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff634005cc0 tx=0x7ff634032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff634003c70 con 0x7ff63c19a190 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff634003e10 con 0x7ff63c19a190 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63c1a51a0 con 0x7ff63c19a190 2024-04-23T18:29:00.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63402fc90 con 0x7ff63c19a190 2024-04-23T18:29:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.811+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63c1a56f0 con 0x7ff63c19a190 2024-04-23T18:29:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.812+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63c0691c0 con 0x7ff63c19a190 2024-04-23T18:29:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.813+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff634004040 con 0x7ff63c19a190 2024-04-23T18:29:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.814+0000 7ff638ff9640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 0x7ff6240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:00.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.814+0000 7ff63b7fe640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 0x7ff6240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:00.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.814+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7ff6340f1da0 con 0x7ff63c19a190 2024-04-23T18:29:00.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.814+0000 7ff63b7fe640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 0x7ff6240a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009930 tx=0x7ff62c008040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:00.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:00.819+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6340be560 con 0x7ff63c19a190 2024-04-23T18:29:01.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.044+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff63c101ac0 con 0x7ff63c19a190 2024-04-23T18:29:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.045+0000 7ff638ff9640 1 -- 172.21.15.33:0/465650891 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7ff6340bde60 con 0x7ff63c19a190 2024-04-23T18:29:01.038 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:29:01.038 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:27:43.160044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:29:01.038 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 msgr2=0x7ff6240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 0x7ff6240a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009930 tx=0x7ff62c008040 comp rx=0 tx=0).stop 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 msgr2=0x7ff63c1a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff634005cc0 tx=0x7ff634032fd0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 shutdown_connections 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff6240a3d20 0x7ff6240a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c19a190 0x7ff63c1a46f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff63c106190 0x7ff63c199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 --2- 172.21.15.33:0/465650891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff63c101110 0x7ff63c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 >> 172.21.15.33:0/465650891 conn(0x7ff63c077fb0 msgr2=0x7ff63c0ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 shutdown_connections 2024-04-23T18:29:01.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.047+0000 7ff641b3d640 1 -- 172.21.15.33:0/465650891 wait complete. 2024-04-23T18:29:01.051 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T18:29:01.051 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T18:29:01.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- 172.21.15.33:0/4234987594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101ac0 msgr2=0x7f5e2c105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 --2- 172.21.15.33:0/4234987594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101ac0 0x7f5e2c105ab0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c009900 tx=0x7f5e1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- 172.21.15.33:0/4234987594 shutdown_connections 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 --2- 172.21.15.33:0/4234987594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 0x7f5e2c10dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 --2- 172.21.15.33:0/4234987594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101ac0 0x7f5e2c105ab0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 --2- 172.21.15.33:0/4234987594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c101110 0x7f5e2c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- 172.21.15.33:0/4234987594 >> 172.21.15.33:0/4234987594 conn(0x7f5e2c077fb0 msgr2=0x7f5e2c0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- 172.21.15.33:0/4234987594 shutdown_connections 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- 172.21.15.33:0/4234987594 wait complete. 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 Processor -- start 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.172+0000 7f5e34536640 1 -- start start 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 0x7f5e2c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c19cff0 0x7f5e2c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112210 con 0x7f5e2c101110 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112090 con 0x7f5e2c19cff0 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c112390 con 0x7f5e2c106190 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34316/0 (socket says 172.21.15.33:34316) 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 -- 172.21.15.33:0/262548639 learned_addr learned my addr 172.21.15.33:0/262548639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 0x7f5e2c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c19cff0 0x7f5e2c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 -- 172.21.15.33:0/262548639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 msgr2=0x7f5e2c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 0x7f5e2c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 -- 172.21.15.33:0/262548639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c19cff0 msgr2=0x7f5e2c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c19cff0 0x7f5e2c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 -- 172.21.15.33:0/262548639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c19dae0 con 0x7f5e2c101110 2024-04-23T18:29:01.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e322d2640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d950 tx=0x7f5e2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e237fe640 1 -- 172.21.15.33:0/262548639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e28017070 con 0x7f5e2c101110 2024-04-23T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e237fe640 1 -- 172.21.15.33:0/262548639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e28009c30 con 0x7f5e2c101110 2024-04-23T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c19ddd0 con 0x7f5e2c101110 2024-04-23T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.173+0000 7f5e237fe640 1 -- 172.21.15.33:0/262548639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800f620 con 0x7f5e2c101110 2024-04-23T18:29:01.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.174+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1a5590 con 0x7f5e2c101110 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e237fe640 1 -- 172.21.15.33:0/262548639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e28010460 con 0x7f5e2c101110 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 Processor -- start 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 -- start start 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a6e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a78b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555aa39d3370 con 0x555aa3a79090 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555aa39702e0 con 0x555aa3a6e9c0 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.176+0000 7f5e37245740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555aa39cf330 con 0x555aa3a6e0a0 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a6e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34330/0 (socket says 172.21.15.33:34330) 2024-04-23T18:29:01.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 -- 172.21.15.33:0/1113526389 learned_addr learned my addr 172.21.15.33:0/1113526389 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a6e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:49840/0 (socket says 172.21.15.33:49840) 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e322d2640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a78b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e237fe640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 -- 172.21.15.33:0/1113526389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 msgr2=0x555aa3a6e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a6e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 -- 172.21.15.33:0/1113526389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 msgr2=0x555aa3a78b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a78b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 -- 172.21.15.33:0/1113526389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa37b1d80 con 0x555aa3a79090 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e322d2640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a78b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e237fe640 1 -- 172.21.15.33:0/262548639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5e280d0430 con 0x7f5e2c101110 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c06a420 tx=0x7f5e1c032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d7b74581254695 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e20bfe640 1 -- 172.21.15.33:0/1113526389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c040070 con 0x555aa3a79090 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e20bfe640 1 -- 172.21.15.33:0/1113526389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c0316a0 con 0x555aa3a79090 2024-04-23T18:29:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 0x7f5e180a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007f20 tx=0x7f5e24004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.177+0000 7f5e20bfe640 1 -- 172.21.15.33:0/1113526389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c035040 con 0x555aa3a79090 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 -- 172.21.15.33:0/1113526389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 msgr2=0x555aa3a7b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c06a420 tx=0x7f5e1c032df0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 -- 172.21.15.33:0/1113526389 shutdown_connections 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7b480 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a78b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 --2- 172.21.15.33:0/1113526389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a6e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 -- 172.21.15.33:0/1113526389 >> 172.21.15.33:0/1113526389 conn(0x555aa3a6a3c0 msgr2=0x555aa3a6a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 -- 172.21.15.33:0/1113526389 shutdown_connections 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.178+0000 7f5e37245740 1 -- 172.21.15.33:0/1113526389 wait complete. 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 -- start start 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 Processor -- start 2024-04-23T18:29:01.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a76010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a76550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555aa3a7be20 con 0x555aa3a79090 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555aa396dfb0 con 0x555aa3a6e9c0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555aa3a7bfa0 con 0x555aa3a6e0a0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e37245740 10 client.0 _ll_register_callbacks cb 0x555aa396d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34338/0 (socket says 172.21.15.33:34338) 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 -- 172.21.15.33:0/809523347 learned_addr learned my addr 172.21.15.33:0/809523347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a76550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e322d2640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a76010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 -- 172.21.15.33:0/809523347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 msgr2=0x555aa3a76010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a76010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 -- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 msgr2=0x555aa3a76550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a76550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.179+0000 7f5e32ad3640 1 -- 172.21.15.33:0/809523347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa3812510 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007a90 tx=0x7f5e24004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f0710a8f946b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24010d80 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5e2401f430 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e1c0316a0 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e2402d050 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 10 client.15981 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555aa36a5670 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e1c0316a0 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5e2400e610 con 0x555aa3a79090 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 10 client.15981 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:01.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 10 client.15981 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 20 client.15981 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3a89490 0x555aa3a8b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 4 client.15981 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.180+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa39e0d90 con 0x555aa3a89490 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.181+0000 7f5e322d2640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3a89490 0x555aa3a8b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.181+0000 7f5e322d2640 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3a89490 0x555aa3a8b910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9ebcc8fe7e213f server_cookie=9add3cf47a272692 in_seq=0 out_seq=0 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.181+0000 7f5e15ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e15ffb640 1 -- 172.21.15.33:0/809523347 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555aa39e0d90 con 0x555aa3a89490 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e15ffb640 10 client.15981 handle_command_reply: tid=0 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 client.15981 shutdown 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 msgr2=0x555aa3a76a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007a90 tx=0x7f5e24004ab0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 shutdown_connections 2024-04-23T18:29:01.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3a89490 0x555aa3a8b910 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a76a90 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a76550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e15ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 --2- 172.21.15.33:0/809523347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a76010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 >> 172.21.15.33:0/809523347 conn(0x555aa3a6a3c0 msgr2=0x555aa3a701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 shutdown_connections 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 1 -- 172.21.15.33:0/809523347 wait complete. 2024-04-23T18:29:01.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.182+0000 7f5e37245740 20 client.15981 trim_cache size 0 max 16384 2024-04-23T18:29:01.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 Processor -- start 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 -- start start 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e0a0 0x555aa3a70800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 0x555aa3a69480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1adb40 con 0x555aa3a79090 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1af6b0 con 0x555aa3a6e0a0 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e37245740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ae210 con 0x555aa3a6e9c0 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34342/0 (socket says 172.21.15.33:34342) 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 -- 172.21.15.33:0/4081732379 learned_addr learned my addr 172.21.15.33:0/4081732379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 0x555aa3a69480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 -- 172.21.15.33:0/4081732379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 msgr2=0x555aa3a69480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 0x555aa3a69480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 -- 172.21.15.33:0/4081732379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e0a0 msgr2=0x555aa3a70800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e0a0 0x555aa3a70800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 -- 172.21.15.33:0/4081732379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa3a6f330 con 0x555aa3a79090 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 0x555aa3a69480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e240044b0 tx=0x7f5e24000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112a085f786f9d5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.200+0000 7f5e17fff640 1 -- 172.21.15.33:0/4081732379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400ec50 con 0x555aa3a79090 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e17fff640 1 -- 172.21.15.33:0/4081732379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e24036030 con 0x555aa3a79090 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e17fff640 1 -- 172.21.15.33:0/4081732379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24029870 con 0x555aa3a79090 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- 172.21.15.33:0/4081732379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 msgr2=0x555aa3a911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5e240044b0 tx=0x7f5e24000f40 comp rx=0 tx=0).stop 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- 172.21.15.33:0/4081732379 shutdown_connections 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a911e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e9c0 0x555aa3a69480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 --2- 172.21.15.33:0/4081732379 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e0a0 0x555aa3a70800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- 172.21.15.33:0/4081732379 >> 172.21.15.33:0/4081732379 conn(0x555aa3a6a3c0 msgr2=0x555aa3a68d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- 172.21.15.33:0/4081732379 shutdown_connections 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- 172.21.15.33:0/4081732379 wait complete. 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 -- start start 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.201+0000 7f5e37245740 1 Processor -- start 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a811f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a81730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1aefb0 con 0x555aa3a79090 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1ae8e0 con 0x555aa3a6e9c0 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1abb40 con 0x555aa3a6e0a0 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 10 client.0 _ll_register_callbacks cb 0x7f5e24012680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e322d2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a811f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34344/0 (socket says 172.21.15.33:34344) 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 -- 172.21.15.33:0/3621217101 learned_addr learned my addr 172.21.15.33:0/3621217101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e31ad1640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a81730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 -- 172.21.15.33:0/3621217101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 msgr2=0x555aa3a811f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a811f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 -- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 msgr2=0x555aa3a81730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a81730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 -- 172.21.15.33:0/3621217101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa3a7e880 con 0x555aa3a79090 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e32ad3640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e24012010 tx=0x7f5e24007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fa681cc5ac6fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400edd0 con 0x555aa3a79090 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f5e2400e3e0 con 0x555aa3a79090 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.202+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e24036030 con 0x555aa3a79090 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e2400e5e0 con 0x555aa3a79090 2024-04-23T18:29:01.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 10 client.15987 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555aa3af0370 con 0x555aa3a79090 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e24036030 con 0x555aa3a79090 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f5e240464b0 con 0x555aa3a79090 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 10 client.15987 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 20 client.15987 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3af0a70 0x555aa3af2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 4 client.15987 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x555aa39dac00 con 0x555aa3af0a70 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.203+0000 7f5e322d2640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3af0a70 0x555aa3af2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.204+0000 7f5e15ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:01.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.204+0000 7f5e322d2640 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3af0a70 0x555aa3af2e90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff0c7f091ea917e server_cookie=68e7ed080157502a in_seq=0 out_seq=0 2024-04-23T18:29:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e15ffb640 1 -- 172.21.15.33:0/3621217101 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555aa39dac00 con 0x555aa3af0a70 2024-04-23T18:29:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e15ffb640 10 client.15987 handle_command_reply: tid=0 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 client.15987 shutdown 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 msgr2=0x555aa3a7e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5e24012010 tx=0x7f5e24007790 comp rx=0 tx=0).stop 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 shutdown_connections 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555aa3af0a70 0x555aa3af2e90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555aa3a79090 0x555aa3a7e1c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e15ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555aa3a6e9c0 0x555aa3a81730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 --2- 172.21.15.33:0/3621217101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555aa3a6e0a0 0x555aa3a811f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 >> 172.21.15.33:0/3621217101 conn(0x555aa3a6a3c0 msgr2=0x555aa3a76860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.218+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 shutdown_connections 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e37245740 1 -- 172.21.15.33:0/3621217101 wait complete. 2024-04-23T18:29:01.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e37245740 20 client.15987 trim_cache size 0 max 16384 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 10, 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "1611aa4b": { 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 10, 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:29:01.213 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:01.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:01.215 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:29:01.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 msgr2=0x7f5e180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 0x7f5e180a61e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e24007f20 tx=0x7f5e24004a40 comp rx=0 tx=0).stop 2024-04-23T18:29:01.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 msgr2=0x7f5e2c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d950 tx=0x7f5e2800de10 comp rx=0 tx=0).stop 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 shutdown_connections 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e2c19cff0 0x7f5e2c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e2c106190 0x7f5e2c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 --2- 172.21.15.33:0/262548639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e2c101110 0x7f5e2c198730 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 >> 172.21.15.33:0/262548639 conn(0x7f5e2c077fb0 msgr2=0x7f5e2c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 shutdown_connections 2024-04-23T18:29:01.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:01.219+0000 7f5e34536640 1 -- 172.21.15.33:0/262548639 wait complete. 2024-04-23T18:29:01.223 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1611aa4b', quiesce took 0.5 seconds. Will release after: 30.0 2024-04-23T18:29:01.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:00 smithi106 ceph-mon[32716]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:29:01.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:00 smithi122 ceph-mon[32546]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:29:01.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:00 smithi033 ceph-mon[29274]: pgmap v78: 97 pgs: 97 active+clean; 2.1 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 12 op/s 2024-04-23T18:29:02.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:02 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/465650891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:02.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:02 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/465650891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:02.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:02 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/465650891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:03 smithi106 ceph-mon[32716]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:29:03.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:03 smithi122 ceph-mon[32546]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:29:03.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:03 smithi033 ceph-mon[29274]: pgmap v79: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-04-23T18:29:05.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:05 smithi106 ceph-mon[32716]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:29:05.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:05 smithi122 ceph-mon[32546]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:29:05.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:05 smithi033 ceph-mon[29274]: pgmap v80: 97 pgs: 97 active+clean; 2.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-23T18:29:07.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:07 smithi033 ceph-mon[29274]: pgmap v81: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-23T18:29:07.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:07 smithi106 ceph-mon[32716]: pgmap v81: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-23T18:29:07.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:07 smithi122 ceph-mon[32546]: pgmap v81: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 KiB/s wr, 34 op/s 2024-04-23T18:29:09.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:09 smithi106 ceph-mon[32716]: pgmap v82: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:09.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:09 smithi122 ceph-mon[32546]: pgmap v82: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:09.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:09 smithi033 ceph-mon[29274]: pgmap v82: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.992+0000 7f8e343e5640 1 -- 172.21.15.33:0/2587684441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c101a90 msgr2=0x7f8e2c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.992+0000 7f8e343e5640 1 --2- 172.21.15.33:0/2587684441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c101a90 0x7f8e2c105ad0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009900 tx=0x7f8e1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:29:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 -- 172.21.15.33:0/2587684441 shutdown_connections 2024-04-23T18:29:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 --2- 172.21.15.33:0/2587684441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 --2- 172.21.15.33:0/2587684441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c101a90 0x7f8e2c105ad0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 --2- 172.21.15.33:0/2587684441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 0x7f8e2c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 -- 172.21.15.33:0/2587684441 >> 172.21.15.33:0/2587684441 conn(0x7f8e2c0771d0 msgr2=0x7f8e2c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 -- 172.21.15.33:0/2587684441 shutdown_connections 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 -- 172.21.15.33:0/2587684441 wait complete. 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 Processor -- start 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.993+0000 7f8e343e5640 1 -- start start 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 0x7f8e2c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 0x7f8e2c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1121f0 con 0x7f8e2c1061b0 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c112070 con 0x7f8e2c19cf50 2024-04-23T18:29:09.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e343e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c112370 con 0x7f8e2c1010e0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57352/0 (socket says 172.21.15.33:57352) 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 -- 172.21.15.33:0/3155127883 learned_addr learned my addr 172.21.15.33:0/3155127883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e32181640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 0x7f8e2c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e32982640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 0x7f8e2c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 -- 172.21.15.33:0/3155127883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 msgr2=0x7f8e2c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 0x7f8e2c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 -- 172.21.15.33:0/3155127883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 msgr2=0x7f8e2c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 0x7f8e2c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 -- 172.21.15.33:0/3155127883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c19da40 con 0x7f8e2c1061b0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e32982640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 0x7f8e2c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e31980640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c032fd0 tx=0x7f8e1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.994+0000 7f8e237fe640 1 -- 172.21.15.33:0/3155127883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c03f030 con 0x7f8e2c1061b0 2024-04-23T18:29:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.995+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c19dcd0 con 0x7f8e2c1061b0 2024-04-23T18:29:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.995+0000 7f8e237fe640 1 -- 172.21.15.33:0/3155127883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e1c004490 con 0x7f8e2c1061b0 2024-04-23T18:29:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.995+0000 7f8e237fe640 1 -- 172.21.15.33:0/3155127883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c035070 con 0x7f8e2c1061b0 2024-04-23T18:29:09.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.995+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c1a54a0 con 0x7f8e2c1061b0 2024-04-23T18:29:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.997+0000 7f8e237fe640 1 -- 172.21.15.33:0/3155127883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e1c042030 con 0x7f8e2c1061b0 2024-04-23T18:29:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 Processor -- start 2024-04-23T18:29:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 -- start start 2024-04-23T18:29:09.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a74a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a74d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55894a6a0970 con 0x55894a74ac10 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55894a63d4a0 con 0x55894a73be00 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.998+0000 7f8e371db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55894a69c930 con 0x55894a73b4e0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e31980640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a74d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:56510/0 (socket says 172.21.15.33:56510) 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 -- 172.21.15.33:0/2171436720 learned_addr learned my addr 172.21.15.33:0/2171436720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e237fe640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 0x7f8e180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32181640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a74a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 0x7f8e180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e237fe640 1 -- 172.21.15.33:0/3155127883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f8e1c0f0d00 con 0x7f8e2c1061b0 2024-04-23T18:29:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 -- 172.21.15.33:0/2171436720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 msgr2=0x55894a74a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a74a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 -- 172.21.15.33:0/2171436720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 msgr2=0x55894a74d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a74d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 -- 172.21.15.33:0/2171436720 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55894a3b3210 con 0x55894a73b4e0 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e31980640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a74d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32181640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a74a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 0x7f8e180a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8e24012c60 tx=0x7f8e2400c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:09.999+0000 7f8e32982640 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e24014ec0 tx=0x7f8e24012760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb75598d1cc2cd6a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e20bfe640 1 -- 172.21.15.33:0/2171436720 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24023070 con 0x55894a73b4e0 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e20bfe640 1 -- 172.21.15.33:0/2171436720 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24014020 con 0x55894a73b4e0 2024-04-23T18:29:09.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e20bfe640 1 -- 172.21.15.33:0/2171436720 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2401f970 con 0x55894a73b4e0 2024-04-23T18:29:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 -- 172.21.15.33:0/2171436720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 msgr2=0x55894a73b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8e24014ec0 tx=0x7f8e24012760 comp rx=0 tx=0).stop 2024-04-23T18:29:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 -- 172.21.15.33:0/2171436720 shutdown_connections 2024-04-23T18:29:09.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a74d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a74a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 --2- 172.21.15.33:0/2171436720 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a73b8c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.000+0000 7f8e371db740 1 -- 172.21.15.33:0/2171436720 >> 172.21.15.33:0/2171436720 conn(0x55894a737b30 msgr2=0x55894a737f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- 172.21.15.33:0/2171436720 shutdown_connections 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- 172.21.15.33:0/2171436720 wait complete. 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- start start 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 Processor -- start 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a74a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 0x55894a7478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a747e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55894a74d780 con 0x55894a73be00 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55894a63b0b0 con 0x55894a74ac10 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55894a74d900 con 0x55894a73b4e0 2024-04-23T18:29:10.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e371db740 10 client.0 _ll_register_callbacks cb 0x55894a628d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e32181640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a74a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e31980640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 0x55894a7478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e32181640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a74a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:56518/0 (socket says 172.21.15.33:56518) 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.001+0000 7f8e32181640 1 -- 172.21.15.33:0/2474951064 learned_addr learned my addr 172.21.15.33:0/2474951064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e32982640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a747e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 -- 172.21.15.33:0/2474951064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 msgr2=0x55894a74a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a74a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 -- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 msgr2=0x55894a747e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a747e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 -- 172.21.15.33:0/2474951064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55894a47f620 con 0x55894a73be00 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e31980640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 0x55894a7478b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0ee400 tx=0x7f8e1c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180e38973b123c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c049040 con 0x55894a73be00 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f8e1c0ee680 con 0x55894a73be00 2024-04-23T18:29:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e24014020 con 0x55894a73be00 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c0ee880 con 0x55894a73be00 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e371db740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55894a7597a0 con 0x55894a73be00 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.002+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e24014020 con 0x55894a73be00 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f8e1c0dd0b0 con 0x55894a73be00 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 20 client.15999 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a75ac50 0x55894a75d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 4 client.15999 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55894a6ae390 con 0x55894a75ac50 2024-04-23T18:29:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e32181640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a75ac50 0x55894a75d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e32181640 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a75ac50 0x55894a75d070 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf6b44c9d94eb37 server_cookie=f1a0f6e636e2da78 in_seq=0 out_seq=0 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.003+0000 7f8e15ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e15ffb640 1 -- 172.21.15.33:0/2474951064 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55894a6ae390 con 0x55894a75ac50 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e15ffb640 10 client.15999 handle_command_reply: tid=0 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 client.15999 shutdown 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 msgr2=0x55894a7478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 0x55894a7478b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0ee400 tx=0x7f8e1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 shutdown_connections 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a75ac50 0x55894a75d070 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a747e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73be00 0x55894a7478b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e15ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 --2- 172.21.15.33:0/2474951064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a74a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 >> 172.21.15.33:0/2474951064 conn(0x55894a737b30 msgr2=0x55894a73d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 shutdown_connections 2024-04-23T18:29:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.004+0000 7f8e371db740 1 -- 172.21.15.33:0/2474951064 wait complete. 2024-04-23T18:29:10.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.005+0000 7f8e371db740 20 client.15999 trim_cache size 0 max 16384 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 Processor -- start 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 -- start start 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 0x55894a736cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a74a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ab900 con 0x55894a73b4e0 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ad470 con 0x55894a74ac10 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e371db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1abfd0 con 0x55894a73be00 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e32181640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e32181640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57378/0 (socket says 172.21.15.33:57378) 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.022+0000 7f8e32181640 1 -- 172.21.15.33:0/3954425885 learned_addr learned my addr 172.21.15.33:0/3954425885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e31980640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 0x55894a736cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32982640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a74a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 -- 172.21.15.33:0/3954425885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 msgr2=0x55894a736cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 0x55894a736cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 -- 172.21.15.33:0/3954425885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 msgr2=0x55894a74a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a74a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 -- 172.21.15.33:0/3954425885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55894a746900 con 0x55894a73b4e0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32982640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a74a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e31980640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 0x55894a736cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e32181640 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005d00 tx=0x7f8e2800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287d159e3638c680 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e17fff640 1 -- 172.21.15.33:0/3954425885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e28005220 con 0x55894a73b4e0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e17fff640 1 -- 172.21.15.33:0/3954425885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e28024030 con 0x55894a73b4e0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.023+0000 7f8e17fff640 1 -- 172.21.15.33:0/3954425885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e280204d0 con 0x55894a73b4e0 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- 172.21.15.33:0/3954425885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 msgr2=0x55894a7566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:10.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8e28005d00 tx=0x7f8e2800b410 comp rx=0 tx=0).stop 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- 172.21.15.33:0/3954425885 shutdown_connections 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a74ac10 0x55894a74a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73be00 0x55894a736cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 --2- 172.21.15.33:0/3954425885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a73b4e0 0x55894a7566b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- 172.21.15.33:0/3954425885 >> 172.21.15.33:0/3954425885 conn(0x55894a737b30 msgr2=0x55894a737f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- 172.21.15.33:0/3954425885 shutdown_connections 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- 172.21.15.33:0/3954425885 wait complete. 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 -- start start 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.024+0000 7f8e371db740 1 Processor -- start 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a744660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a744ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1acd70 con 0x55894a74ac10 2024-04-23T18:29:10.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1ac6a0 con 0x55894a73be00 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c1a9920 con 0x55894a73b4e0 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e371db740 10 client.0 _ll_register_callbacks cb 0x7f8e1c003ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57388/0 (socket says 172.21.15.33:57388) 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 -- 172.21.15.33:0/1379902471 learned_addr learned my addr 172.21.15.33:0/1379902471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e31980640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a744ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32181640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a744660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 -- 172.21.15.33:0/1379902471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 msgr2=0x55894a744660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a744660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 -- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 msgr2=0x55894a744ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a744ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 -- 172.21.15.33:0/1379902471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55894a7c0250 con 0x55894a74ac10 2024-04-23T18:29:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32181640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a744660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e32982640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e240096d0 tx=0x7f8e24014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f202ff257e1451 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.025+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24037060 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f8e24014050 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e28024030 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e24023040 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55894a7c14b0 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e28024030 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7f8e2400f040 con 0x55894a74ac10 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 20 client.16005 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a7c1a60 0x55894a7c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 4 client.16005 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55894a6a8200 con 0x55894a7c1a60 2024-04-23T18:29:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.026+0000 7f8e32181640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a7c1a60 0x55894a7c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.027+0000 7f8e32181640 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a7c1a60 0x55894a7c3e80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aefad1236d885f70 server_cookie=c0437b361b919550 in_seq=0 out_seq=0 2024-04-23T18:29:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.027+0000 7f8e15ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e15ffb640 1 -- 172.21.15.33:0/1379902471 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55894a6a8200 con 0x55894a7c1a60 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e15ffb640 10 client.16005 handle_command_reply: tid=0 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 client.16005 shutdown 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 msgr2=0x55894a7450e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e240096d0 tx=0x7f8e24014aa0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 shutdown_connections 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55894a7c1a60 0x55894a7c3e80 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55894a74ac10 0x55894a7450e0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55894a73be00 0x55894a744ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 --2- 172.21.15.33:0/1379902471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55894a73b4e0 0x55894a744660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e15ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 >> 172.21.15.33:0/1379902471 conn(0x55894a737b30 msgr2=0x55894a742f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.035+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 shutdown_connections 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e371db740 1 -- 172.21.15.33:0/1379902471 wait complete. 2024-04-23T18:29:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e371db740 20 client.16005 trim_cache size 0 max 16384 2024-04-23T18:29:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:29:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:29:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:29:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 msgr2=0x7f8e180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 0x7f8e180a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8e24012c60 tx=0x7f8e2400c000 comp rx=0 tx=0).stop 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 msgr2=0x7f8e2c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c032fd0 tx=0x7f8e1c004270 comp rx=0 tx=0).stop 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 shutdown_connections 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8e180a3dc0 0x7f8e180a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e2c19cf50 0x7f8e2c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e2c1061b0 0x7f8e2c198bd0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 --2- 172.21.15.33:0/3155127883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e2c1010e0 0x7f8e2c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 >> 172.21.15.33:0/3155127883 conn(0x7f8e2c0771d0 msgr2=0x7f8e2c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 shutdown_connections 2024-04-23T18:29:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:10.036+0000 7f8e343e5640 1 -- 172.21.15.33:0/3155127883 wait complete. 2024-04-23T18:29:11.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:11 smithi106 ceph-mon[32716]: pgmap v83: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:11.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:11 smithi122 ceph-mon[32546]: pgmap v83: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:11 smithi033 ceph-mon[29274]: pgmap v83: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 31 KiB/s wr, 28 op/s 2024-04-23T18:29:13.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:13 smithi106 ceph-mon[32716]: pgmap v84: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-04-23T18:29:13.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:13 smithi122 ceph-mon[32546]: pgmap v84: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-04-23T18:29:13.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:13 smithi033 ceph-mon[29274]: pgmap v84: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 34 op/s 2024-04-23T18:29:15.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:15 smithi106 ceph-mon[32716]: pgmap v85: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:29:15.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:15 smithi122 ceph-mon[32546]: pgmap v85: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:29:15.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:15 smithi033 ceph-mon[29274]: pgmap v85: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 28 KiB/s wr, 26 op/s 2024-04-23T18:29:17.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:17 smithi106 ceph-mon[32716]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-23T18:29:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:17 smithi122 ceph-mon[32546]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-23T18:29:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:17 smithi033 ceph-mon[29274]: pgmap v86: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 28 KiB/s wr, 30 op/s 2024-04-23T18:29:19.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:19 smithi106 ceph-mon[32716]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:19 smithi122 ceph-mon[32546]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:19.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:19 smithi033 ceph-mon[29274]: pgmap v87: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:21.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:21 smithi106 ceph-mon[32716]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:21.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:21 smithi122 ceph-mon[32546]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:21.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:21 smithi033 ceph-mon[29274]: pgmap v88: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-23T18:29:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- 172.21.15.33:0/3321038290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc334102e40 msgr2=0x7fc33406a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3321038290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc334102e40 0x7fc33406a7c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc324009900 tx=0x7fc32402a410 comp rx=0 tx=0).stop 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- 172.21.15.33:0/3321038290 shutdown_connections 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3321038290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33410e7a0 0x7fc334110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3321038290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc33406ad00 0x7fc33406b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3321038290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc334102e40 0x7fc33406a7c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- 172.21.15.33:0/3321038290 >> 172.21.15.33:0/3321038290 conn(0x7fc334077ba0 msgr2=0x7fc334077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- 172.21.15.33:0/3321038290 shutdown_connections 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- 172.21.15.33:0/3321038290 wait complete. 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 Processor -- start 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.136+0000 7fc33b19b640 1 -- start start 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33406ad00 0x7fc3341a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 0x7fc3341a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341143b0 con 0x7fc33410e7a0 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc334114230 con 0x7fc33406ad00 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc33b19b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc334114530 con 0x7fc334102e40 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51610/0 (socket says 172.21.15.33:51610) 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 -- 172.21.15.33:0/3969097269 learned_addr learned my addr 172.21.15.33:0/3969097269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc338f37640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33406ad00 0x7fc3341a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc333fff640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 0x7fc3341a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 -- 172.21.15.33:0/3969097269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 msgr2=0x7fc3341a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 0x7fc3341a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 -- 172.21.15.33:0/3969097269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33406ad00 msgr2=0x7fc3341a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33406ad00 0x7fc3341a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 -- 172.21.15.33:0/3969097269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33419af70 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc333fff640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 0x7fc3341a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc339738640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00eb10 tx=0x7fc32c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.137+0000 7fc331ffb640 1 -- 172.21.15.33:0/3969097269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004b20 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.138+0000 7fc331ffb640 1 -- 172.21.15.33:0/3969097269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32c004cc0 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.138+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33419b260 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.138+0000 7fc331ffb640 1 -- 172.21.15.33:0/3969097269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c010670 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.138+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3341a7570 con 0x7fc33410e7a0 2024-04-23T18:29:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.140+0000 7fc331ffb640 1 -- 172.21.15.33:0/3969097269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc32c005020 con 0x7fc33410e7a0 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.140+0000 7fc33df8f740 1 Processor -- start 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.140+0000 7fc33df8f740 1 -- start start 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528589bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625284e1970 con 0x56252857c620 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56252847e4a0 con 0x56252857cf40 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc33df8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5625284dd930 con 0x562528587800 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc331ffb640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 0x7fc31c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51620/0 (socket says 172.21.15.33:51620) 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 -- 172.21.15.33:0/2634389887 learned_addr learned my addr 172.21.15.33:0/2634389887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc338f37640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc339738640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 0x7fc31c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528589bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc331ffb640 1 -- 172.21.15.33:0/3969097269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc32c0d0d50 con 0x7fc33410e7a0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 -- 172.21.15.33:0/2634389887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 msgr2=0x562528589bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528589bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 -- 172.21.15.33:0/2634389887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 msgr2=0x5625285872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 -- 172.21.15.33:0/2634389887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625281f4210 con 0x56252857c620 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc338f37640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.141+0000 7fc333fff640 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc328009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f40e5d25e5dda1e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc339738640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 0x7fc31c0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc32c099670 tx=0x7fc32c0bd040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc31affd640 1 -- 172.21.15.33:0/2634389887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280104d0 con 0x56252857c620 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc31affd640 1 -- 172.21.15.33:0/2634389887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32801b070 con 0x56252857c620 2024-04-23T18:29:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc31affd640 1 -- 172.21.15.33:0/2634389887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc328013780 con 0x56252857c620 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 -- 172.21.15.33:0/2634389887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 msgr2=0x56252857ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc328005fd0 tx=0x7fc328009650 comp rx=0 tx=0).stop 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 -- 172.21.15.33:0/2634389887 shutdown_connections 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528589bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 --2- 172.21.15.33:0/2634389887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x56252857ca00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 -- 172.21.15.33:0/2634389887 >> 172.21.15.33:0/2634389887 conn(0x562528578b30 msgr2=0x562528578f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 -- 172.21.15.33:0/2634389887 shutdown_connections 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.142+0000 7fc33df8f740 1 -- 172.21.15.33:0/2634389887 wait complete. 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 -- start start 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 Processor -- start 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285843c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528584900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56252858a3d0 con 0x56252857c620 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56252847c0b0 con 0x56252857cf40 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56252858a550 con 0x562528587800 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc33df8f740 10 client.0 _ll_register_callbacks cb 0x562528469d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc333fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc333fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51628/0 (socket says 172.21.15.33:51628) 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.143+0000 7fc333fff640 1 -- 172.21.15.33:0/1809816616 learned_addr learned my addr 172.21.15.33:0/1809816616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc338f37640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285843c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528584900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 -- 172.21.15.33:0/1809816616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 msgr2=0x562528584900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528584900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 -- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 msgr2=0x5625285843c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285843c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 -- 172.21.15.33:0/1809816616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625282c0620 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc333fff640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc328009fe0 tx=0x7fc3280095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb76249ae521dc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc313fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3280109e0 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc328010b80 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc32801b070 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc328010d80 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc33df8f740 10 client.16017 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.144+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625285962b0 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc32801b070 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc32800c070 con 0x56252857c620 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 20 client.16017 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562528597650 0x562528599a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 4 client.16017 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625284ef390 con 0x562528597650 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.145+0000 7fc338f37640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562528597650 0x562528599a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc338f37640 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562528597650 0x562528599a40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c2f6dd8eb22048 server_cookie=5872998835b29c48 in_seq=0 out_seq=0 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc313fff640 10 client.16017 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc313fff640 1 -- 172.21.15.33:0/1809816616 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625284ef390 con 0x562528597650 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc313fff640 10 client.16017 handle_command_reply: tid=0 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 client.16017 shutdown 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 msgr2=0x5625285870d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc328009fe0 tx=0x7fc3280095b0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 shutdown_connections 2024-04-23T18:29:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562528597650 0x562528599a40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x562528584900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857cf40 0x5625285843c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc313fff640 0 client.16017 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1809816616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857c620 0x5625285870d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.146+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 >> 172.21.15.33:0/1809816616 conn(0x562528578b30 msgr2=0x56252857e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.147+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 shutdown_connections 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.147+0000 7fc33df8f740 1 -- 172.21.15.33:0/1809816616 wait complete. 2024-04-23T18:29:23.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.147+0000 7fc33df8f740 20 client.16017 trim_cache size 0 max 16384 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 Processor -- start 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 -- start start 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x562528593290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252859f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341afd10 con 0x56252857cf40 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341b1880 con 0x56252857c620 2024-04-23T18:29:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc33df8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341b03e0 con 0x562528587800 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51630/0 (socket says 172.21.15.33:51630) 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 -- 172.21.15.33:0/1482948079 learned_addr learned my addr 172.21.15.33:0/1482948079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc333fff640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252859f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc333fff640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x562528593290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 -- 172.21.15.33:0/1482948079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 msgr2=0x56252859f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252859f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 -- 172.21.15.33:0/1482948079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 msgr2=0x562528593290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x562528593290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 -- 172.21.15.33:0/1482948079 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562528584c60 con 0x56252857cf40 2024-04-23T18:29:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc333fff640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x562528593290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.162+0000 7fc338f37640 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc32402a930 tx=0x7fc324005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aada77472e51d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc31a7fc640 1 -- 172.21.15.33:0/1482948079 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32404b070 con 0x56252857cf40 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc31a7fc640 1 -- 172.21.15.33:0/1482948079 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3240044b0 con 0x56252857cf40 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc31a7fc640 1 -- 172.21.15.33:0/1482948079 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324035040 con 0x56252857cf40 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 -- 172.21.15.33:0/1482948079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 msgr2=0x562528586c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc32402a930 tx=0x7fc324005e60 comp rx=0 tx=0).stop 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 -- 172.21.15.33:0/1482948079 shutdown_connections 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252859f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x562528586c00 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 --2- 172.21.15.33:0/1482948079 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x562528593290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 -- 172.21.15.33:0/1482948079 >> 172.21.15.33:0/1482948079 conn(0x562528578b30 msgr2=0x56252857f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 -- 172.21.15.33:0/1482948079 shutdown_connections 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.163+0000 7fc33df8f740 1 -- 172.21.15.33:0/1482948079 wait complete. 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 -- start start 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 Processor -- start 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x56252858f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252858ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341b1180 con 0x56252857cf40 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341b0ab0 con 0x56252857c620 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341adcf0 con 0x562528587800 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc33df8f740 10 client.0 _ll_register_callbacks cb 0x7fc328009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:23.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc338f37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc338f37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51642/0 (socket says 172.21.15.33:51642) 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc338f37640 1 -- 172.21.15.33:0/3119920258 learned_addr learned my addr 172.21.15.33:0/3119920258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.164+0000 7fc333fff640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x56252858f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc333fff640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252858ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 -- 172.21.15.33:0/3119920258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 msgr2=0x56252858ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252858ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 -- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 msgr2=0x56252858f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x56252858f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 -- 172.21.15.33:0/3119920258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252858b5c0 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc333fff640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252858ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc338f37640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc32402a450 tx=0x7fc3240095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5557101fb86c6763 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc313fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324053020 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc324005350 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3240044b0 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc324035050 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc33df8f740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5625285fd4d0 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.165+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3240044b0 con 0x56252857cf40 2024-04-23T18:29:23.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc32404e070 con 0x56252857cf40 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 20 client.16023 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625285fdd00 0x562528600120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 4 client.16023 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625284e9200 con 0x5625285fdd00 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc333fff640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625285fdd00 0x562528600120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc333fff640 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625285fdd00 0x562528600120 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986734da5de37e88 server_cookie=2d2392d5406c222b in_seq=0 out_seq=0 2024-04-23T18:29:23.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.166+0000 7fc313fff640 10 client.16023 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.170+0000 7fc313fff640 1 -- 172.21.15.33:0/3119920258 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625284e9200 con 0x5625285fdd00 2024-04-23T18:29:23.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.170+0000 7fc313fff640 10 client.16023 handle_command_reply: tid=0 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.170+0000 7fc33df8f740 1 client.16023 shutdown 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.170+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 msgr2=0x56252858fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc32402a450 tx=0x7fc3240095a0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 shutdown_connections 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5625285fdd00 0x562528600120 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562528587800 0x56252858ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56252857cf40 0x56252858fca0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 --2- 172.21.15.33:0/3119920258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56252857c620 0x56252858f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc313fff640 0 client.16023 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 >> 172.21.15.33:0/3119920258 conn(0x562528578b30 msgr2=0x56252857f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 shutdown_connections 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 1 -- 172.21.15.33:0/3119920258 wait complete. 2024-04-23T18:29:23.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33df8f740 20 client.16023 trim_cache size 0 max 16384 2024-04-23T18:29:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:29:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:29:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:29:23.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 msgr2=0x7fc31c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 0x7fc31c0a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc32c099670 tx=0x7fc32c0bd040 comp rx=0 tx=0).stop 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 msgr2=0x7fc33419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00eb10 tx=0x7fc32c00c2d0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 shutdown_connections 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc31c0a3e40 0x7fc31c0a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc33410e7a0 0x7fc33419a6b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc334102e40 0x7fc3341a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 --2- 172.21.15.33:0/3969097269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc33406ad00 0x7fc3341a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.171+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 >> 172.21.15.33:0/3969097269 conn(0x7fc334077ba0 msgr2=0x7fc33410f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.172+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 shutdown_connections 2024-04-23T18:29:23.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:23.172+0000 7fc33b19b640 1 -- 172.21.15.33:0/3969097269 wait complete. 2024-04-23T18:29:23.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:23 smithi106 ceph-mon[32716]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T18:29:23.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:23 smithi122 ceph-mon[32546]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T18:29:23.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:23 smithi033 ceph-mon[29274]: pgmap v89: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-04-23T18:29:25.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:25 smithi106 ceph-mon[32716]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2024-04-23T18:29:25.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:25 smithi122 ceph-mon[32546]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2024-04-23T18:29:25.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:25 smithi033 ceph-mon[29274]: pgmap v90: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 8 op/s 2024-04-23T18:29:27.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:27 smithi106 ceph-mon[32716]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:27.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:27 smithi122 ceph-mon[32546]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:27.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:27 smithi033 ceph-mon[29274]: pgmap v91: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:29.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:29 smithi106 ceph-mon[32716]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:29.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:29 smithi122 ceph-mon[32546]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:29.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:29 smithi033 ceph-mon[29274]: pgmap v92: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:31.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- 172.21.15.33:0/3764069900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48641031e0 msgr2=0x7f48641035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 --2- 172.21.15.33:0/3764069900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48641031e0 0x7f48641035c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4854009900 tx=0x7f485402a510 comp rx=0 tx=0).stop 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- 172.21.15.33:0/3764069900 shutdown_connections 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 --2- 172.21.15.33:0/3764069900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f486410c520 0x7f486410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 --2- 172.21.15.33:0/3764069900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4864103b00 0x7f486410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 --2- 172.21.15.33:0/3764069900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48641031e0 0x7f48641035c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- 172.21.15.33:0/3764069900 >> 172.21.15.33:0/3764069900 conn(0x7f4864077f50 msgr2=0x7f4864078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- 172.21.15.33:0/3764069900 shutdown_connections 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- 172.21.15.33:0/3764069900 wait complete. 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 Processor -- start 2024-04-23T18:29:31.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.346+0000 7f486a3af640 1 -- start start 2024-04-23T18:29:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48641031e0 0x7f4864198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4864103b00 0x7f4864198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48641121e0 con 0x7f486410c520 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4864112060 con 0x7f48641031e0 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486a3af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4864112360 con 0x7f4864103b00 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43200/0 (socket says 172.21.15.33:43200) 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 -- 172.21.15.33:0/616994678 learned_addr learned my addr 172.21.15.33:0/616994678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f4863fff640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48641031e0 0x7f4864198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f48637fe640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4864103b00 0x7f4864198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 -- 172.21.15.33:0/616994678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4864103b00 msgr2=0x7f4864198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4864103b00 0x7f4864198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 -- 172.21.15.33:0/616994678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48641031e0 msgr2=0x7f4864198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48641031e0 0x7f4864198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 -- 172.21.15.33:0/616994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486419d720 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.347+0000 7f486894c640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f485c00eaa0 tx=0x7f485c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.348+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c00cdb0 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.348+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486419da10 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.348+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f485c005020 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.348+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0116d0 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.348+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48641a52a0 con 0x7f486410c520 2024-04-23T18:29:31.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.349+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4864068d70 con 0x7f486410c520 2024-04-23T18:29:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.351+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f485c0051d0 con 0x7f486410c520 2024-04-23T18:29:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.351+0000 7f48617fa640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 0x7f48500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.351+0000 7f4863fff640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 0x7f48500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.352+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f485c0d0d30 con 0x7f486410c520 2024-04-23T18:29:31.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.352+0000 7f4863fff640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 0x7f48500a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4854005ab0 tx=0x7f4854005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.355+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f485c016030 con 0x7f486410c520 2024-04-23T18:29:31.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:31 smithi106 ceph-mon[32716]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:31.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:31 smithi122 ceph-mon[32546]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:31.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:31 smithi033 ceph-mon[29274]: pgmap v93: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.585+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4864107700 con 0x7f486410c520 2024-04-23T18:29:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.586+0000 7f48617fa640 1 -- 172.21.15.33:0/616994678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7f485c09d3e0 con 0x7f486410c520 2024-04-23T18:29:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:29:31.579 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:27:43.160044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:29:31.579 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-04-23T18:29:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.588+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 msgr2=0x7f48500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.588+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 0x7f48500a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4854005ab0 tx=0x7f4854005a20 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.588+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 msgr2=0x7f486419cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.588+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f485c00eaa0 tx=0x7f485c00ef60 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.588+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 shutdown_connections 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f48500a3d20 0x7f48500a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f486410c520 0x7f486419cff0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4864103b00 0x7f4864198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 --2- 172.21.15.33:0/616994678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48641031e0 0x7f4864198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 >> 172.21.15.33:0/616994678 conn(0x7f4864077f50 msgr2=0x7f48640fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 shutdown_connections 2024-04-23T18:29:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.589+0000 7f486a3af640 1 -- 172.21.15.33:0/616994678 wait complete. 2024-04-23T18:29:31.593 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 1611aa4b --release --await' 2024-04-23T18:29:31.593 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 1611aa4b --release --await 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 -- 172.21.15.33:0/1268957183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c102b40 msgr2=0x7fb93c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 --2- 172.21.15.33:0/1268957183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c102b40 0x7fb93c106b80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009900 tx=0x7fb92c02a410 comp rx=0 tx=0).stop 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 -- 172.21.15.33:0/1268957183 shutdown_connections 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 --2- 172.21.15.33:0/1268957183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c107260 0x7fb93c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 --2- 172.21.15.33:0/1268957183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c102b40 0x7fb93c106b80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 --2- 172.21.15.33:0/1268957183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c102190 0x7fb93c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 -- 172.21.15.33:0/1268957183 >> 172.21.15.33:0/1268957183 conn(0x7fb93c0fd990 msgr2=0x7fb93c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 -- 172.21.15.33:0/1268957183 shutdown_connections 2024-04-23T18:29:31.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.709+0000 7fb941ff7640 1 -- 172.21.15.33:0/1268957183 wait complete. 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 Processor -- start 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 -- start start 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 0x7fb93c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112180 con 0x7fb93c107260 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112000 con 0x7fb93c102190 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb941ff7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c112300 con 0x7fb93c074cd0 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43204/0 (socket says 172.21.15.33:43204) 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 -- 172.21.15.33:0/3438563922 learned_addr learned my addr 172.21.15.33:0/3438563922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 -- 172.21.15.33:0/3438563922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c074cd0 msgr2=0x7fb93c1a6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 -- 172.21.15.33:0/3438563922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 msgr2=0x7fb93c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 0x7fb93c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 0x7fb93c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93affd640 1 -- 172.21.15.33:0/3438563922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93c1a7010 con 0x7fb93c107260 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.710+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 0x7fb93c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb93affd640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006010 tx=0x7fb92c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb938ff9640 1 -- 172.21.15.33:0/3438563922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03f030 con 0x7fb93c107260 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93c1a72a0 con 0x7fb93c107260 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb938ff9640 1 -- 172.21.15.33:0/3438563922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92c003aa0 con 0x7fb93c107260 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb938ff9640 1 -- 172.21.15.33:0/3438563922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c035070 con 0x7fb93c107260 2024-04-23T18:29:31.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.711+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93c1a7790 con 0x7fb93c107260 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.713+0000 7fb938ff9640 1 -- 172.21.15.33:0/3438563922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb92c042030 con 0x7fb93c107260 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.713+0000 7fb944de9740 1 Processor -- start 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.713+0000 7fb944de9740 1 -- start start 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.713+0000 7fb944de9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.713+0000 7fb944de9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a304f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb944de9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a307870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb944de9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f64a25f6d0 con 0x55f64a2fa490 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb944de9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f64a1fc1a0 con 0x55f64a305480 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb944de9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f64a25b690 con 0x55f64a2fadb0 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb938ff9640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 0x7fb9240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43208/0 (socket says 172.21.15.33:43208) 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 -- 172.21.15.33:0/4195465350 learned_addr learned my addr 172.21.15.33:0/4195465350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a304f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93affd640 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a307870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 0x7fb9240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb938ff9640 1 -- 172.21.15.33:0/3438563922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb92c0f0d30 con 0x7fb93c107260 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 -- 172.21.15.33:0/4195465350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 msgr2=0x55f64a304f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a304f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 -- 172.21.15.33:0/4195465350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 msgr2=0x55f64a307870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a307870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 -- 172.21.15.33:0/4195465350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64a2065b0 con 0x55f64a2fa490 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb9340049d0 tx=0x7fb934012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29fb2e23ab715ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 0x7fb9240a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb93401b000 tx=0x7fb93401d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.714+0000 7fb921ffb640 1 -- 172.21.15.33:0/4195465350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93401c9d0 con 0x55f64a2fa490 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb921ffb640 1 -- 172.21.15.33:0/4195465350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934020030 con 0x55f64a2fa490 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb921ffb640 1 -- 172.21.15.33:0/4195465350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934011d00 con 0x55f64a2fa490 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 -- 172.21.15.33:0/4195465350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 msgr2=0x55f64a2fa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb9340049d0 tx=0x7fb934012d10 comp rx=0 tx=0).stop 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 -- 172.21.15.33:0/4195465350 shutdown_connections 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a307870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a304f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 --2- 172.21.15.33:0/4195465350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fa490 0x55f64a2fa870 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 -- 172.21.15.33:0/4195465350 >> 172.21.15.33:0/4195465350 conn(0x55f64a2f67b0 msgr2=0x55f64a2f6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 -- 172.21.15.33:0/4195465350 shutdown_connections 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.715+0000 7fb944de9740 1 -- 172.21.15.33:0/4195465350 wait complete. 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 -- start start 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 Processor -- start 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a304f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a302900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f64a3082b0 con 0x55f64a2fadb0 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f64a1f9b20 con 0x55f64a305480 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f64a308430 con 0x55f64a2fa490 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb944de9740 10 client.0 _ll_register_callbacks cb 0x55f64a1f9530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43214/0 (socket says 172.21.15.33:43214) 2024-04-23T18:29:31.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 -- 172.21.15.33:0/3173087449 learned_addr learned my addr 172.21.15.33:0/3173087449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a304f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a302900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 -- 172.21.15.33:0/3173087449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 msgr2=0x55f64a304f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a304f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 -- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 msgr2=0x55f64a302900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a302900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93affd640 1 -- 172.21.15.33:0/3173087449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f649f71920 con 0x55f64a2fadb0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.716+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a304f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb93affd640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0b90e0 tx=0x7fb92c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7680619e7312151d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c049040 con 0x55f64a2fadb0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb92c0b92f0 con 0x55f64a2fadb0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb934020030 con 0x55f64a2fadb0 2024-04-23T18:29:31.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb92c0c1030 con 0x55f64a2fadb0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f64a03dd30 con 0x55f64a2fadb0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934020030 con 0x55f64a2fadb0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb92c0bcee0 con 0x55f64a2fadb0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 10 client.16041 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 20 client.16041 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a315510 0x55f64a317960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 4 client.16041 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.717+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64a21d500 con 0x55f64a315510 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.718+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a315510 0x55f64a317960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.718+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a315510 0x55f64a317960 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61e93e52a88ad60 server_cookie=c2e4b350dde15f2b in_seq=0 out_seq=0 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.718+0000 7fb91affd640 10 client.16041 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:31.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb91affd640 1 -- 172.21.15.33:0/3173087449 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f64a21d500 con 0x55f64a315510 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb91affd640 10 client.16041 handle_command_reply: tid=0 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 client.16041 shutdown 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 msgr2=0x55f64a302210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0b90e0 tx=0x7fb92c003f50 comp rx=0 tx=0).stop 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 shutdown_connections 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a315510 0x55f64a317960 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a305480 0x55f64a302900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb91affd640 0 client.16041 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a2fadb0 0x55f64a302210 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 --2- 172.21.15.33:0/3173087449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a304f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 >> 172.21.15.33:0/3173087449 conn(0x55f64a2f67b0 msgr2=0x55f64a2fc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 shutdown_connections 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 1 -- 172.21.15.33:0/3173087449 wait complete. 2024-04-23T18:29:31.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.719+0000 7fb944de9740 20 client.16041 trim_cache size 0 max 16384 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 Processor -- start 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 -- start start 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a311170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 0x55f64a304a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1afd70 con 0x55f64a305480 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b18e0 con 0x55f64a2fadb0 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb944de9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b0440 con 0x55f64a2fa490 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.736+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43224/0 (socket says 172.21.15.33:43224) 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 -- 172.21.15.33:0/3599583213 learned_addr learned my addr 172.21.15.33:0/3599583213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93affd640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 0x55f64a304a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a311170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 -- 172.21.15.33:0/3599583213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 msgr2=0x55f64a311170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a311170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 -- 172.21.15.33:0/3599583213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 msgr2=0x55f64a304a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 0x55f64a304a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 -- 172.21.15.33:0/3599583213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64a317fe0 con 0x55f64a305480 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93affd640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 0x55f64a304a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb93bfff640 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb9340049a0 tx=0x7fb934005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c028f9cf03b0786f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb9217fa640 1 -- 172.21.15.33:0/3599583213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934035020 con 0x55f64a305480 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb9217fa640 1 -- 172.21.15.33:0/3599583213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934004b20 con 0x55f64a305480 2024-04-23T18:29:31.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.737+0000 7fb9217fa640 1 -- 172.21.15.33:0/3599583213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93401c7d0 con 0x55f64a305480 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- 172.21.15.33:0/3599583213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 msgr2=0x55f64a2fc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb9340049a0 tx=0x7fb934005bd0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- 172.21.15.33:0/3599583213 shutdown_connections 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a2fc330 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fadb0 0x55f64a304a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- 172.21.15.33:0/3599583213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fa490 0x55f64a311170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- 172.21.15.33:0/3599583213 >> 172.21.15.33:0/3599583213 conn(0x55f64a2f67b0 msgr2=0x55f64a2fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- 172.21.15.33:0/3599583213 shutdown_connections 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- 172.21.15.33:0/3599583213 wait complete. 2024-04-23T18:29:31.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- start start 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 Processor -- start 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fa490 0x55f64a3068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a306e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b11e0 con 0x55f64a305480 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1b0b10 con 0x55f64a2fa490 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.738+0000 7fb944de9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb93c1add70 con 0x55f64a2fadb0 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb944de9740 10 client.0 _ll_register_callbacks cb 0x7fb92c0bc620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43230/0 (socket says 172.21.15.33:43230) 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 -- 172.21.15.33:0/1965993953 learned_addr learned my addr 172.21.15.33:0/1965993953 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93affd640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a306e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fa490 0x55f64a3068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 -- 172.21.15.33:0/1965993953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 msgr2=0x55f64a306e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a306e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 -- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fa490 msgr2=0x55f64a3068e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fa490 0x55f64a3068e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 -- 172.21.15.33:0/1965993953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64a30c210 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb93bfff640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb934009cf0 tx=0x7fb9340099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ccf69d177572c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb91affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93403e030 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fb93401c7d0 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb934004b20 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb934035040 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb944de9740 10 client.16047 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.739+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f64a37c290 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934004b20 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fb934011980 con 0x55f64a305480 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 10 client.16047 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 10 client.16047 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:31.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 20 client.16047 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a37c7e0 0x55f64a37ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 4 client.16047 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1611aa4b", "release": true, "await": true}] 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1611aa4b", "release": true, "await": true}) -- 0x55f64a26d0f0 con 0x55f64a37c7e0 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.740+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a37c7e0 0x55f64a37ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.741+0000 7fb93b7fe640 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a37c7e0 0x55f64a37ec00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbfdec614770b88 server_cookie=b1bb261ba49bd9f2 in_seq=0 out_seq=0 2024-04-23T18:29:31.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.741+0000 7fb91affd640 10 client.16047 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb91affd640 1 -- 172.21.15.33:0/1965993953 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f64a26d0f0 con 0x55f64a37c7e0 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb91affd640 10 client.16047 handle_command_reply: tid=0 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 client.16047 shutdown 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 msgr2=0x55f64a30bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb934009cf0 tx=0x7fb9340099b0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 shutdown_connections 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f64a37c7e0 0x55f64a37ec00 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f64a305480 0x55f64a30bcd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f64a2fadb0 0x55f64a306e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb91affd640 0 client.16047 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 --2- 172.21.15.33:0/1965993953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f64a2fa490 0x55f64a3068e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 >> 172.21.15.33:0/1965993953 conn(0x55f64a2f67b0 msgr2=0x55f64a2fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 shutdown_connections 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 1 -- 172.21.15.33:0/1965993953 wait complete. 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.743+0000 7fb944de9740 20 client.16047 trim_cache size 0 max 16384 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:29:31.740 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 12, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "1611aa4b": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 12, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:31.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:31.742 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:29:31.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 msgr2=0x7fb9240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 0x7fb9240a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb93401b000 tx=0x7fb93401d000 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 msgr2=0x7fb93c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb92c006010 tx=0x7fb92c004060 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 shutdown_connections 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb9240a3d20 0x7fb9240a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c074cd0 0x7fb93c1a6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb93c107260 0x7fb93c074790 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 --2- 172.21.15.33:0/3438563922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb93c102190 0x7fb93c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 >> 172.21.15.33:0/3438563922 conn(0x7fb93c0fd990 msgr2=0x7fb93c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 shutdown_connections 2024-04-23T18:29:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.744+0000 7fb941ff7640 1 -- 172.21.15.33:0/3438563922 wait complete. 2024-04-23T18:29:31.748 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1611aa4b', seconds elapsed: 31.1 2024-04-23T18:29:31.748 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T18:29:31.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:29:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.857+0000 7fec0c8ae640 1 -- 172.21.15.33:0/3601249466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec0810c580 msgr2=0x7fec0810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.857+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/3601249466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec0810c580 0x7fec0810e9b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fec00009900 tx=0x7fec0002a480 comp rx=0 tx=0).stop 2024-04-23T18:29:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 -- 172.21.15.33:0/3601249466 shutdown_connections 2024-04-23T18:29:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/3601249466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec0810c580 0x7fec0810e9b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/3601249466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec08103ad0 0x7fec0810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/3601249466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 0x7fec08103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 -- 172.21.15.33:0/3601249466 >> 172.21.15.33:0/3601249466 conn(0x7fec080771d0 msgr2=0x7fec080775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 -- 172.21.15.33:0/3601249466 shutdown_connections 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 -- 172.21.15.33:0/3601249466 wait complete. 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 Processor -- start 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 -- start start 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.858+0000 7fec0c8ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 0x7fec08198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec0810c580 0x7fec0819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fec081121a0 con 0x7fec08103ad0 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec08112020 con 0x7fec0810c580 2024-04-23T18:29:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec08112320 con 0x7fec081031b0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43242/0 (socket says 172.21.15.33:43242) 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 -- 172.21.15.33:0/1081255805 learned_addr learned my addr 172.21.15.33:0/1081255805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec06d9d640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec0810c580 0x7fec0819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0659c640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 0x7fec08198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 -- 172.21.15.33:0/1081255805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 msgr2=0x7fec08198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 0x7fec08198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 -- 172.21.15.33:0/1081255805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec0810c580 msgr2=0x7fec0819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec0810c580 0x7fec0819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 -- 172.21.15.33:0/1081255805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0819d6f0 con 0x7fec08103ad0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec05d9b640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7febf000e9f0 tx=0x7febf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000cd80 con 0x7fec08103ad0 2024-04-23T18:29:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febf00108e0 con 0x7fec08103ad0 2024-04-23T18:29:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0819d9e0 con 0x7fec08103ad0 2024-04-23T18:29:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0018980 con 0x7fec08103ad0 2024-04-23T18:29:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.859+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec081a5270 con 0x7fec08103ad0 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.860+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0806add0 con 0x7fec08103ad0 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.862+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7febf0010430 con 0x7fec08103ad0 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.862+0000 7febff7fe640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 0x7febf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.862+0000 7fec0659c640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 0x7febf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.862+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7febf00d0fb0 con 0x7fec08103ad0 2024-04-23T18:29:31.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.863+0000 7fec0659c640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 0x7febf40a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febf8005fd0 tx=0x7febf8009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:31.866+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febf009d660 con 0x7fec08103ad0 2024-04-23T18:29:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.091+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec080691c0 con 0x7fec08103ad0 2024-04-23T18:29:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.091+0000 7febff7fe640 1 -- 172.21.15.33:0/1081255805 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11930 (secure 0 0 0) 0x7fec080691c0 con 0x7fec08103ad0 2024-04-23T18:29:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:29:32.084 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:27:43.160044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:29:32.084 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-04-23T18:29:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.093+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 msgr2=0x7febf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.093+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 0x7febf40a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febf8005fd0 tx=0x7febf8009500 comp rx=0 tx=0).stop 2024-04-23T18:29:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 msgr2=0x7fec08198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7febf000e9f0 tx=0x7febf000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 shutdown_connections 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7febf40a3d20 0x7febf40a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec0810c580 0x7fec0819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec08103ad0 0x7fec08198c40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 --2- 172.21.15.33:0/1081255805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec081031b0 0x7fec08198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 >> 172.21.15.33:0/1081255805 conn(0x7fec080771d0 msgr2=0x7fec080fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 shutdown_connections 2024-04-23T18:29:32.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.094+0000 7fec0c8ae640 1 -- 172.21.15.33:0/1081255805 wait complete. 2024-04-23T18:29:32.097 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T18:29:32.097 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T18:29:32.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 -- 172.21.15.33:0/2837235774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 msgr2=0x7fab74102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 --2- 172.21.15.33:0/2837235774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74102620 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fab64009900 tx=0x7fab6402a410 comp rx=0 tx=0).stop 2024-04-23T18:29:32.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 -- 172.21.15.33:0/2837235774 shutdown_connections 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 --2- 172.21.15.33:0/2837235774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab7410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 --2- 172.21.15.33:0/2837235774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab74102bf0 0x7fab74106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 --2- 172.21.15.33:0/2837235774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74102620 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 -- 172.21.15.33:0/2837235774 >> 172.21.15.33:0/2837235774 conn(0x7fab740790e0 msgr2=0x7fab740794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 -- 172.21.15.33:0/2837235774 shutdown_connections 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.217+0000 7fab78eea640 1 -- 172.21.15.33:0/2837235774 wait complete. 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 Processor -- start 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 -- start start 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab74198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab7419cfc0 0x7fab7419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab741121e0 con 0x7fab74102240 2024-04-23T18:29:32.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab74112060 con 0x7fab74107280 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab78eea640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab74112360 con 0x7fab7419cfc0 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab7259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab7259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43268/0 (socket says 172.21.15.33:43268) 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.218+0000 7fab7259c640 1 -- 172.21.15.33:0/290819166 learned_addr learned my addr 172.21.15.33:0/290819166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab72d9d640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab7419cfc0 0x7fab7419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab6bfff640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab74198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 -- 172.21.15.33:0/290819166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab7419cfc0 msgr2=0x7fab7419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab7419cfc0 0x7fab7419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 -- 172.21.15.33:0/290819166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 msgr2=0x7fab74198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab74198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 -- 172.21.15.33:0/290819166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7419dab0 con 0x7fab74102240 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab6bfff640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab74198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab7259c640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fab64005940 tx=0x7fab64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab6b7fe640 1 -- 172.21.15.33:0/290819166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6403f030 con 0x7fab74102240 2024-04-23T18:29:32.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7419dd40 con 0x7fab74102240 2024-04-23T18:29:32.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab6b7fe640 1 -- 172.21.15.33:0/290819166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab64003aa0 con 0x7fab74102240 2024-04-23T18:29:32.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab6b7fe640 1 -- 172.21.15.33:0/290819166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64035070 con 0x7fab74102240 2024-04-23T18:29:32.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.219+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab741a5500 con 0x7fab74102240 2024-04-23T18:29:32.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.221+0000 7fab6b7fe640 1 -- 172.21.15.33:0/290819166 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab64042030 con 0x7fab74102240 2024-04-23T18:29:32.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 Processor -- start 2024-04-23T18:29:32.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 -- start start 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c2643480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636c25a8370 con 0x5636c264e090 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636c25452e0 con 0x5636c26439c0 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636c25a4330 con 0x5636c26430a0 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab6b7fe640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 0x7fab600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43270/0 (socket says 172.21.15.33:43270) 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab6bfff640 1 -- 172.21.15.33:0/2306496140 learned_addr learned my addr 172.21.15.33:0/2306496140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab72d9d640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 0x7fab600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab6b7fe640 1 -- 172.21.15.33:0/290819166 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fab640f0ff0 con 0x7fab74102240 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab7259c640 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.222+0000 7fab72d9d640 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c2643480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 -- 172.21.15.33:0/2306496140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 msgr2=0x5636c2643480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c2643480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 -- 172.21.15.33:0/2306496140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 msgr2=0x5636c264db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 -- 172.21.15.33:0/2306496140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c2386d80 con 0x5636c264e090 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab72d9d640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 0x7fab600a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009270 tx=0x7fab6c012b10 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab6bfff640 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00bb80 tx=0x7fab5c00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc4aab46a357ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab68ff9640 1 -- 172.21.15.33:0/2306496140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c017420 con 0x5636c264e090 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab68ff9640 1 -- 172.21.15.33:0/2306496140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5c017a70 con 0x5636c264e090 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.223+0000 7fab68ff9640 1 -- 172.21.15.33:0/2306496140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00fa50 con 0x5636c264e090 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- 172.21.15.33:0/2306496140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 msgr2=0x5636c2650480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00bb80 tx=0x7fab5c00bf80 comp rx=0 tx=0).stop 2024-04-23T18:29:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- 172.21.15.33:0/2306496140 shutdown_connections 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c2650480 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/2306496140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c2643480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- 172.21.15.33:0/2306496140 >> 172.21.15.33:0/2306496140 conn(0x5636c263f3c0 msgr2=0x5636c263f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- 172.21.15.33:0/2306496140 shutdown_connections 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- 172.21.15.33:0/2306496140 wait complete. 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- start start 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 Processor -- start 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c264af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636c2650e80 con 0x5636c264e090 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636c2542fb0 con 0x5636c26439c0 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.224+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636c2651000 con 0x5636c26430a0 2024-04-23T18:29:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab7bbfd740 10 client.0 _ll_register_callbacks cb 0x5636c25429a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43272/0 (socket says 172.21.15.33:43272) 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 -- 172.21.15.33:0/1443582829 learned_addr learned my addr 172.21.15.33:0/1443582829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab7259c640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c264af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 -- 172.21.15.33:0/1443582829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 msgr2=0x5636c264af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c264af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 -- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 msgr2=0x5636c264b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 -- 172.21.15.33:0/1443582829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c23e7510 con 0x5636c264e090 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab72d9d640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fab6c003ba0 tx=0x7fab6c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59b870a95123002 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c027400 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fab6c0275a0 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c017a70 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.225+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab6c0277a0 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 10 client.16065 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c227a670 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5c017a70 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fab6c01e070 con 0x5636c264e090 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 10 client.16065 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 10 client.16065 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 20 client.16065 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c265e2f0 0x5636c2660770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 4 client.16065 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c25b5d90 con 0x5636c265e2f0 2024-04-23T18:29:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.226+0000 7fab7259c640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c265e2f0 0x5636c2660770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab59ffb640 10 client.16065 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7259c640 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c265e2f0 0x5636c2660770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6955d9ed61573b72 server_cookie=bc1060d7be4d5489 in_seq=0 out_seq=0 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab59ffb640 1 -- 172.21.15.33:0/1443582829 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636c25b5d90 con 0x5636c265e2f0 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab59ffb640 10 client.16065 handle_command_reply: tid=0 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 client.16065 shutdown 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 msgr2=0x5636c264ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fab6c003ba0 tx=0x7fab6c009500 comp rx=0 tx=0).stop 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 shutdown_connections 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c265e2f0 0x5636c2660770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c264e090 0x5636c264ba10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c264b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab59ffb640 0 client.16065 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1443582829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c26430a0 0x5636c264af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.227+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 >> 172.21.15.33:0/1443582829 conn(0x5636c263f3c0 msgr2=0x5636c26451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.228+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 shutdown_connections 2024-04-23T18:29:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.228+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1443582829 wait complete. 2024-04-23T18:29:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.228+0000 7fab7bbfd740 20 client.16065 trim_cache size 0 max 16384 2024-04-23T18:29:32.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 Processor -- start 2024-04-23T18:29:32.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 -- start start 2024-04-23T18:29:32.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26430a0 0x5636c2659d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2656660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab741adad0 con 0x5636c26439c0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab741af640 con 0x5636c26430a0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ae1a0 con 0x5636c264e090 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43278/0 (socket says 172.21.15.33:43278) 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 -- 172.21.15.33:0/1715610457 learned_addr learned my addr 172.21.15.33:0/1715610457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab72d9d640 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2656660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab7259c640 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26430a0 0x5636c2659d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 -- 172.21.15.33:0/1715610457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 msgr2=0x5636c2656660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2656660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 -- 172.21.15.33:0/1715610457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26430a0 msgr2=0x5636c2659d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26430a0 0x5636c2659d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.245+0000 7fab6bfff640 1 -- 172.21.15.33:0/1715610457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c2649f40 con 0x5636c26439c0 2024-04-23T18:29:32.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5636c2640550 tx=0x7fab5c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45327550d2d99e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab5bfff640 1 -- 172.21.15.33:0/1715610457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c028070 con 0x5636c26439c0 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab5bfff640 1 -- 172.21.15.33:0/1715610457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5c0054f0 con 0x5636c26439c0 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab5bfff640 1 -- 172.21.15.33:0/1715610457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00f5c0 con 0x5636c26439c0 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1715610457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 msgr2=0x5636c2645800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5636c2640550 tx=0x7fab5c00b8a0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1715610457 shutdown_connections 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2656660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26439c0 0x5636c2645800 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1715610457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26430a0 0x5636c2659d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1715610457 >> 172.21.15.33:0/1715610457 conn(0x5636c263f3c0 msgr2=0x5636c264c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1715610457 shutdown_connections 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.246+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1715610457 wait complete. 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 -- start start 2024-04-23T18:29:32.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 Processor -- start 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c2647200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2647740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab741aef40 con 0x5636c26430a0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab741ae870 con 0x5636c26439c0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab741abab0 con 0x5636c264e090 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7bbfd740 10 client.0 _ll_register_callbacks cb 0x7fab6c010e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43290/0 (socket says 172.21.15.33:43290) 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 -- 172.21.15.33:0/1693448413 learned_addr learned my addr 172.21.15.33:0/1693448413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab72d9d640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2647740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c2647200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 -- 172.21.15.33:0/1693448413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 msgr2=0x5636c2647740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2647740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 -- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 msgr2=0x5636c2647200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c2647200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.247+0000 7fab7259c640 1 -- 172.21.15.33:0/1693448413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c2647de0 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7259c640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fab640bced0 tx=0x7fab64003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19590f3ce68afa49 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab640da070 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fab640b95b0 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5c0054f0 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab64049040 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c26c97c0 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab5c0054f0 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fab640eea00 con 0x5636c26430a0 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:32.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 10 client.16071 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 20 client.16071 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c26c9f30 0x5636c26cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 4 client.16071 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.248+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5636c25afc00 con 0x5636c26c9f30 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.249+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c26c9f30 0x5636c26cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.249+0000 7fab59ffb640 10 client.16071 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:32.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.249+0000 7fab6bfff640 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c26c9f30 0x5636c26cc350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f530986abe24ed0 server_cookie=7654107db8bf000b in_seq=0 out_seq=0 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.264+0000 7fab59ffb640 1 -- 172.21.15.33:0/1693448413 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5636c25afc00 con 0x5636c26c9f30 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.264+0000 7fab59ffb640 10 client.16071 handle_command_reply: tid=0 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.264+0000 7fab7bbfd740 1 client.16071 shutdown 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 msgr2=0x5636c2648a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fab640bced0 tx=0x7fab64003f10 comp rx=0 tx=0).stop 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 shutdown_connections 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636c26c9f30 0x5636c26cc350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636c264e090 0x5636c2647740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab59ffb640 0 client.16071 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:32.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636c26439c0 0x5636c2647200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 --2- 172.21.15.33:0/1693448413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636c26430a0 0x5636c2648a00 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 >> 172.21.15.33:0/1693448413 conn(0x5636c263f3c0 msgr2=0x5636c264d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 shutdown_connections 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 1 -- 172.21.15.33:0/1693448413 wait complete. 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab7bbfd740 20 client.16071 trim_cache size 0 max 16384 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 29, 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 14, 2024-04-23T18:29:32.259 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "93656ef1": { 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 14, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:29:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:29:32.261 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 msgr2=0x7fab600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 0x7fab600a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009270 tx=0x7fab6c012b10 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 msgr2=0x7fab74198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fab64005940 tx=0x7fab64004270 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 shutdown_connections 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab600a3d20 0x7fab600a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab7419cfc0 0x7fab7419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab74107280 0x7fab74198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 --2- 172.21.15.33:0/290819166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102240 0x7fab74198700 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 >> 172.21.15.33:0/290819166 conn(0x7fab740790e0 msgr2=0x7fab74108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 shutdown_connections 2024-04-23T18:29:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:29:32.265+0000 7fab78eea640 1 -- 172.21.15.33:0/290819166 wait complete. 2024-04-23T18:29:32.269 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '93656ef1', quiesce took 0.5 seconds. Will release after: 30.0 2024-04-23T18:29:32.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:32 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/616994678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:32.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:32 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/616994678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:32.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:32 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/616994678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:33.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:33 smithi106 ceph-mon[32716]: pgmap v94: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:33.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:33 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1081255805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:33.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:33 smithi122 ceph-mon[32546]: pgmap v94: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:33.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:33 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1081255805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:33.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:33 smithi033 ceph-mon[29274]: pgmap v94: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-23T18:29:33.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:33 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1081255805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:29:35.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:35 smithi106 ceph-mon[32716]: pgmap v95: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:35.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:35 smithi122 ceph-mon[32546]: pgmap v95: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:35.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:35 smithi033 ceph-mon[29274]: pgmap v95: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 -- 172.21.15.33:0/2029951452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc028106170 msgr2=0x7fc02810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 --2- 172.21.15.33:0/2029951452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc028106170 0x7fc02810dd40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc020009ca0 tx=0x7fc02002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 -- 172.21.15.33:0/2029951452 shutdown_connections 2024-04-23T18:29:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 --2- 172.21.15.33:0/2029951452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc028106170 0x7fc02810dd40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 --2- 172.21.15.33:0/2029951452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc028101aa0 0x7fc028105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 --2- 172.21.15.33:0/2029951452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0281010f0 0x7fc0281014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 -- 172.21.15.33:0/2029951452 >> 172.21.15.33:0/2029951452 conn(0x7fc028077fd0 msgr2=0x7fc0280783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 -- 172.21.15.33:0/2029951452 shutdown_connections 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.271+0000 7fc02ea35640 1 -- 172.21.15.33:0/2029951452 wait complete. 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 Processor -- start 2024-04-23T18:29:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 -- start start 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106170 0x7fc028198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 0x7fc02819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281110e0 con 0x7fc0281010f0 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc028110f60 con 0x7fc02819cfc0 2024-04-23T18:29:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02ea35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc028111260 con 0x7fc028106170 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc027fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc027fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43302/0 (socket says 172.21.15.33:43302) 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc027fff640 1 -- 172.21.15.33:0/752179810 learned_addr learned my addr 172.21.15.33:0/752179810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.272+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 0x7fc02819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc0277fe640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106170 0x7fc028198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 -- 172.21.15.33:0/752179810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106170 msgr2=0x7fc028198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106170 0x7fc028198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 -- 172.21.15.33:0/752179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 msgr2=0x7fc02819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 0x7fc02819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 -- 172.21.15.33:0/752179810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02819dab0 con 0x7fc0281010f0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 0x7fc02819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc027fff640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc01800d770 tx=0x7fc01800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc0257fa640 1 -- 172.21.15.33:0/752179810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018017070 con 0x7fc0281010f0 2024-04-23T18:29:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc0257fa640 1 -- 172.21.15.33:0/752179810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0180054d0 con 0x7fc0281010f0 2024-04-23T18:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02819dda0 con 0x7fc0281010f0 2024-04-23T18:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc0257fa640 1 -- 172.21.15.33:0/752179810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018004c30 con 0x7fc0281010f0 2024-04-23T18:29:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.273+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0281a5540 con 0x7fc0281010f0 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.275+0000 7fc0257fa640 1 -- 172.21.15.33:0/752179810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc018004e10 con 0x7fc0281010f0 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 Processor -- start 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 -- start start 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc0257fa640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 0x7fc0100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9fa4800 0x563bb9fa6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563bb9efe970 con 0x563bb9f99620 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bb9e9b4a0 con 0x563bb9f99f40 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc03182c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bb9efa930 con 0x563bb9fa4800 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 0x7fc0100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc0257fa640 1 -- 172.21.15.33:0/752179810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc0180d03d0 con 0x7fc0281010f0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc027fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc027fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43306/0 (socket says 172.21.15.33:43306) 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.276+0000 7fc027fff640 1 -- 172.21.15.33:0/2779175606 learned_addr learned my addr 172.21.15.33:0/2779175606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc0277fe640 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9fa4800 0x563bb9fa6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 -- 172.21.15.33:0/2779175606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9fa4800 msgr2=0x563bb9fa6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9fa4800 0x563bb9fa6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 -- 172.21.15.33:0/2779175606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 msgr2=0x563bb9fa42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 -- 172.21.15.33:0/2779175606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb9c11210 con 0x563bb9f99620 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 0x7fc0100a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc020006010 tx=0x7fc0200093f0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc027fff640 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc018005c20 tx=0x7fc018005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d528cdb980a393 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc00e7fc640 1 -- 172.21.15.33:0/2779175606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018020040 con 0x563bb9f99620 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc00e7fc640 1 -- 172.21.15.33:0/2779175606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0180a06d0 con 0x563bb9f99620 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.277+0000 7fc00e7fc640 1 -- 172.21.15.33:0/2779175606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01809a6b0 con 0x563bb9f99620 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- 172.21.15.33:0/2779175606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 msgr2=0x563bb9f99a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc018005c20 tx=0x7fc018005740 comp rx=0 tx=0).stop 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- 172.21.15.33:0/2779175606 shutdown_connections 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9fa4800 0x563bb9fa6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 --2- 172.21.15.33:0/2779175606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99620 0x563bb9f99a00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- 172.21.15.33:0/2779175606 >> 172.21.15.33:0/2779175606 conn(0x563bb9f95b30 msgr2=0x563bb9f95f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- 172.21.15.33:0/2779175606 shutdown_connections 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- 172.21.15.33:0/2779175606 wait complete. 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.278+0000 7fc03182c740 1 -- start start 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 Processor -- start 2024-04-23T18:29:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fa41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563bb9fa7430 con 0x563bb9fa4800 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bb9e990b0 con 0x563bb9f99f40 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bb9fa75b0 con 0x563bb9f99620 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc03182c740 10 client.0 _ll_register_callbacks cb 0x563bb9e86d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43314/0 (socket says 172.21.15.33:43314) 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 -- 172.21.15.33:0/1973471898 learned_addr learned my addr 172.21.15.33:0/1973471898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc0277fe640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc027fff640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fa41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 -- 172.21.15.33:0/1973471898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 msgr2=0x563bb9fa41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fa41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 -- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 msgr2=0x563bb9fa14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc02cfd2640 1 -- 172.21.15.33:0/1973471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb9cdd620 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.279+0000 7fc0277fe640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc020035000 tx=0x7fc02002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efdd7d7dad44de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0200047b0 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc02002f660 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0180a06d0 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc020058050 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc03182c740 10 client.16083 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bb9fb33a0 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0180a06d0 con 0x563bb9fa4800 2024-04-23T18:29:36.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc020049070 con 0x563bb9fa4800 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc03182c740 10 client.16083 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.280+0000 7fc03182c740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc03182c740 20 client.16083 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb9fb4980 0x563bb9fb6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc03182c740 4 client.16083 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb9f0c390 con 0x563bb9fb4980 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc027fff640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb9fb4980 0x563bb9fb6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc0077fe640 10 client.16083 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.281+0000 7fc027fff640 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb9fb4980 0x563bb9fb6dc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fbe7d14073aff0 server_cookie=5d9cab43baeeb447 in_seq=0 out_seq=0 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc0077fe640 1 -- 172.21.15.33:0/1973471898 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bb9f0c390 con 0x563bb9fb4980 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc0077fe640 10 client.16083 handle_command_reply: tid=0 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 client.16083 shutdown 2024-04-23T18:29:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 msgr2=0x563bb9fa1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc020035000 tx=0x7fc02002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 shutdown_connections 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb9fb4980 0x563bb9fb6dc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9fa1a40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99f40 0x563bb9fa14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 --2- 172.21.15.33:0/1973471898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fa41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 >> 172.21.15.33:0/1973471898 conn(0x563bb9f95b30 msgr2=0x563bb9f9b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc0077fe640 0 client.16083 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 shutdown_connections 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 1 -- 172.21.15.33:0/1973471898 wait complete. 2024-04-23T18:29:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.282+0000 7fc03182c740 20 client.16083 trim_cache size 0 max 16384 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.297+0000 7fc03182c740 1 Processor -- start 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.297+0000 7fc03182c740 1 -- start start 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.297+0000 7fc03182c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fb02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc03182c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc03182c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9fa4800 0x563bb9fbc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc03182c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281adae0 con 0x563bb9f99f40 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc03182c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281af650 con 0x563bb9fa4800 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc03182c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281ae1b0 con 0x563bb9f99620 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43330/0 (socket says 172.21.15.33:43330) 2024-04-23T18:29:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 -- 172.21.15.33:0/3908255114 learned_addr learned my addr 172.21.15.33:0/3908255114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc027fff640 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fb02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9fa4800 0x563bb9fbc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 -- 172.21.15.33:0/3908255114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 msgr2=0x563bb9fb02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fb02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 -- 172.21.15.33:0/3908255114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9fa4800 msgr2=0x563bb9fbc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9fa4800 0x563bb9fbc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 -- 172.21.15.33:0/3908255114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb9fa7310 con 0x563bb9f99f40 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc0277fe640 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc02806b730 tx=0x7fc01c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df8363ead8458cb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc00dffb640 1 -- 172.21.15.33:0/3908255114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00fe60 con 0x563bb9f99f40 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc00dffb640 1 -- 172.21.15.33:0/3908255114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01c004230 con 0x563bb9f99f40 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.298+0000 7fc00dffb640 1 -- 172.21.15.33:0/3908255114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c0118f0 con 0x563bb9f99f40 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- 172.21.15.33:0/3908255114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 msgr2=0x563bb9f94b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fc02806b730 tx=0x7fc01c013830 comp rx=0 tx=0).stop 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- 172.21.15.33:0/3908255114 shutdown_connections 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9fa4800 0x563bb9fbc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9f99f40 0x563bb9f94b30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 --2- 172.21.15.33:0/3908255114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99620 0x563bb9fb02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- 172.21.15.33:0/3908255114 >> 172.21.15.33:0/3908255114 conn(0x563bb9f95b30 msgr2=0x563bb9f94420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- 172.21.15.33:0/3908255114 shutdown_connections 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- 172.21.15.33:0/3908255114 wait complete. 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.299+0000 7fc03182c740 1 -- start start 2024-04-23T18:29:36.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 Processor -- start 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99620 0x563bb9faa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99f40 0x563bb9faa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281aef50 con 0x563bb9fa4800 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281ae880 con 0x563bb9f99620 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281abac0 con 0x563bb9f99f40 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc03182c740 10 client.0 _ll_register_callbacks cb 0x7fc020031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43342/0 (socket says 172.21.15.33:43342) 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 -- 172.21.15.33:0/3413454774 learned_addr learned my addr 172.21.15.33:0/3413454774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:36.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc0277fe640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99f40 0x563bb9faa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc027fff640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99620 0x563bb9faa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 -- 172.21.15.33:0/3413454774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99f40 msgr2=0x563bb9faa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99f40 0x563bb9faa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 -- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99620 msgr2=0x563bb9faa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99620 0x563bb9faa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 -- 172.21.15.33:0/3413454774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bba01a0e0 con 0x563bb9fa4800 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc02cfd2640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc020009620 tx=0x7fc020031960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fb2e376f432fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.300+0000 7fc0077fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020030d40 con 0x563bb9fa4800 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc020004600 con 0x563bb9fa4800 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c004230 con 0x563bb9fa4800 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc020005bd0 con 0x563bb9fa4800 2024-04-23T18:29:36.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563bba01b3b0 con 0x563bb9fa4800 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc01c004230 con 0x563bb9fa4800 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8358+0+0 (secure 0 0 0) 0x7fc020049040 con 0x563bb9fa4800 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 20 client.16089 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bba01bb20 0x563bba01df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 4 client.16089 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.301+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bb9f06200 con 0x563bba01bb20 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.302+0000 7fc027fff640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bba01bb20 0x563bba01df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.302+0000 7fc027fff640 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bba01bb20 0x563bba01df40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c977141babb94197 server_cookie=5a439ab52fb6fc27 in_seq=0 out_seq=0 2024-04-23T18:29:36.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.302+0000 7fc0077fe640 10 client.16089 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc0077fe640 1 -- 172.21.15.33:0/3413454774 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bb9f06200 con 0x563bba01bb20 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc0077fe640 10 client.16089 handle_command_reply: tid=0 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 client.16089 shutdown 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 msgr2=0x563bb9faaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc020009620 tx=0x7fc020031960 comp rx=0 tx=0).stop 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 shutdown_connections 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bba01bb20 0x563bba01df40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb9fa4800 0x563bb9faaea0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb9f99f40 0x563bb9faa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 --2- 172.21.15.33:0/3413454774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb9f99620 0x563bb9faa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 >> 172.21.15.33:0/3413454774 conn(0x563bb9f95b30 msgr2=0x563bb9f9f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc0077fe640 0 client.16089 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:36.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 shutdown_connections 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 1 -- 172.21.15.33:0/3413454774 wait complete. 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.306+0000 7fc03182c740 20 client.16089 trim_cache size 0 max 16384 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:29:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 msgr2=0x7fc0100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 0x7fc0100a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc020006010 tx=0x7fc0200093f0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 msgr2=0x7fc028198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc01800d770 tx=0x7fc01800dc30 comp rx=0 tx=0).stop 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 shutdown_connections 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0100a3dc0 0x7fc0100a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc02819cfc0 0x7fc02819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc028106170 0x7fc028198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 --2- 172.21.15.33:0/752179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0281010f0 0x7fc028198700 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 >> 172.21.15.33:0/752179810 conn(0x7fc028077fd0 msgr2=0x7fc028069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 shutdown_connections 2024-04-23T18:29:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:36.307+0000 7fc02ea35640 1 -- 172.21.15.33:0/752179810 wait complete. 2024-04-23T18:29:37.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:37 smithi106 ceph-mon[32716]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:29:37.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:37 smithi122 ceph-mon[32546]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:29:37.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:37 smithi033 ceph-mon[29274]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-04-23T18:29:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:38.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:29:38.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:38.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:38.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:38.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:29:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:29:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:29:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:29:38.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:29:39.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:39 smithi106 ceph-mon[32716]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:39.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:39 smithi122 ceph-mon[32546]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:39.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:39 smithi033 ceph-mon[29274]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:41.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:41 smithi106 ceph-mon[32716]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:41.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:41 smithi122 ceph-mon[32546]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:41.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:41 smithi033 ceph-mon[29274]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-23T18:29:43.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:43 smithi106 ceph-mon[32716]: pgmap v99: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:29:43.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:43 smithi122 ceph-mon[32546]: pgmap v99: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:29:43.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:43 smithi033 ceph-mon[29274]: pgmap v99: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-04-23T18:29:45.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:45 smithi106 ceph-mon[32716]: pgmap v100: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-04-23T18:29:45.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:45 smithi122 ceph-mon[32546]: pgmap v100: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-04-23T18:29:45.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:45 smithi033 ceph-mon[29274]: pgmap v100: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 9 op/s 2024-04-23T18:29:47.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:47 smithi106 ceph-mon[32716]: pgmap v101: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-23T18:29:47.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:47 smithi122 ceph-mon[32546]: pgmap v101: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-23T18:29:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:47 smithi033 ceph-mon[29274]: pgmap v101: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-04-23T18:29:49.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:49 smithi106 ceph-mon[32716]: pgmap v102: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- 172.21.15.33:0/3122024902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c102b70 msgr2=0x7f498c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- 172.21.15.33:0/3122024902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c102b70 0x7f498c106bb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402a410 comp rx=0 tx=0).stop 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- 172.21.15.33:0/3122024902 shutdown_connections 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- 172.21.15.33:0/3122024902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 0x7f498c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- 172.21.15.33:0/3122024902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c102b70 0x7f498c106bb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- 172.21.15.33:0/3122024902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- 172.21.15.33:0/3122024902 >> 172.21.15.33:0/3122024902 conn(0x7f498c0fda20 msgr2=0x7f498c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- 172.21.15.33:0/3122024902 shutdown_connections 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- 172.21.15.33:0/3122024902 wait complete. 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 Processor -- start 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- start start 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 0x7f498c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1121d0 con 0x7f498c19cf80 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f498c112050 con 0x7f498c107290 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.406+0000 7f4991907640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f498c112350 con 0x7f498c1021c0 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44248/0 (socket says 172.21.15.33:44248) 2024-04-23T18:29:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 -- 172.21.15.33:0/3739990212 learned_addr learned my addr 172.21.15.33:0/3739990212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4983fff640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 0x7f498c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498affd640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 -- 172.21.15.33:0/3739990212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 msgr2=0x7f498c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 -- 172.21.15.33:0/3739990212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 msgr2=0x7f498c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 0x7f498c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 -- 172.21.15.33:0/3739990212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c19da70 con 0x7f498c19cf80 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498affd640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f498b7fe640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f498400eaa0 tx=0x7f498400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4990905640 1 -- 172.21.15.33:0/3739990212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498400cdb0 con 0x7f498c19cf80 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c19dd60 con 0x7f498c19cf80 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4990905640 1 -- 172.21.15.33:0/3739990212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984005020 con 0x7f498c19cf80 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4990905640 1 -- 172.21.15.33:0/3739990212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840116d0 con 0x7f498c19cf80 2024-04-23T18:29:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.407+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a54d0 con 0x7f498c19cf80 2024-04-23T18:29:49.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4990905640 1 -- 172.21.15.33:0/3739990212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f49840051d0 con 0x7f498c19cf80 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 Processor -- start 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 -- start start 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecb3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a1ec0b970 con 0x558a1eca6620 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558a1eba84a0 con 0x558a1ecb1800 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4994616740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a1ec07930 con 0x558a1eca6f40 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44262/0 (socket says 172.21.15.33:44262) 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 -- 172.21.15.33:0/1717331143 learned_addr learned my addr 172.21.15.33:0/1717331143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f498affd640 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecb3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4990905640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 0x7f49780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 -- 172.21.15.33:0/1717331143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 msgr2=0x558a1ecb12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 -- 172.21.15.33:0/1717331143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 msgr2=0x558a1ecb3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecb3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 -- 172.21.15.33:0/1717331143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1e91e210 con 0x558a1eca6620 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f498b7fe640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 0x7f49780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4983fff640 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f49740096c0 tx=0x7f49740095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9334def09b9435e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4990905640 1 -- 172.21.15.33:0/3739990212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f49840d0990 con 0x7f498c19cf80 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4981ffb640 1 -- 172.21.15.33:0/1717331143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974044020 con 0x558a1eca6620 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4981ffb640 1 -- 172.21.15.33:0/1717331143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49740317a0 con 0x558a1eca6620 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.411+0000 7f4981ffb640 1 -- 172.21.15.33:0/1717331143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974035040 con 0x558a1eca6620 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- 172.21.15.33:0/1717331143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 msgr2=0x558a1eca6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f49740096c0 tx=0x7f49740095a0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f498b7fe640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 0x7f49780a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4984098da0 tx=0x7f49840bd040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- 172.21.15.33:0/1717331143 shutdown_connections 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecb3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 --2- 172.21.15.33:0/1717331143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1eca6a00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- 172.21.15.33:0/1717331143 >> 172.21.15.33:0/1717331143 conn(0x558a1eca2b30 msgr2=0x558a1eca2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- 172.21.15.33:0/1717331143 shutdown_connections 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- 172.21.15.33:0/1717331143 wait complete. 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.412+0000 7f4994616740 1 -- start start 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 Processor -- start 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecae970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558a1ecb43d0 con 0x558a1eca6620 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558a1eba60b0 con 0x558a1ecb1800 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558a1ecb4550 con 0x558a1eca6f40 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4994616740 10 client.0 _ll_register_callbacks cb 0x558a1eb93d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44272/0 (socket says 172.21.15.33:44272) 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 -- 172.21.15.33:0/2385573104 learned_addr learned my addr 172.21.15.33:0/2385573104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f498affd640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 -- 172.21.15.33:0/2385573104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 msgr2=0x558a1ecae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 -- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 msgr2=0x558a1ecae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.413+0000 7f4983fff640 1 -- 172.21.15.33:0/2385573104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a1e9ea620 con 0x558a1eca6620 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4983fff640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4974005e90 tx=0x7f4974009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bd894d320fc296 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974051020 con 0x558a1eca6620 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f49740035f0 con 0x558a1eca6620 2024-04-23T18:29:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740317a0 con 0x558a1eca6620 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4974044030 con 0x558a1eca6620 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a1ecc02e0 con 0x558a1eca6620 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49740317a0 con 0x558a1eca6620 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f497404a6a0 con 0x558a1eca6620 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.414+0000 7f4994616740 20 client.16101 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ecc1630 0x558a1ecc3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f4994616740 4 client.16101 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a1ec19390 con 0x558a1ecc1630 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f498affd640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ecc1630 0x558a1ecc3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f496affd640 10 client.16101 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:49.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.415+0000 7f498affd640 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ecc1630 0x558a1ecc3a20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bbef03a9a4459e server_cookie=9ca22a1bc26eec4e in_seq=0 out_seq=0 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f496affd640 1 -- 172.21.15.33:0/2385573104 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a1ec19390 con 0x558a1ecc1630 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f496affd640 10 client.16101 handle_command_reply: tid=0 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 client.16101 shutdown 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 msgr2=0x558a1ecb1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4974005e90 tx=0x7f4974009290 comp rx=0 tx=0).stop 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 shutdown_connections 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ecc1630 0x558a1ecc3a20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1ecb1800 0x558a1ecae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f496affd640 0 client.16101 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 --2- 172.21.15.33:0/2385573104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6620 0x558a1ecb1140 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 >> 172.21.15.33:0/2385573104 conn(0x558a1eca2b30 msgr2=0x558a1eca88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 shutdown_connections 2024-04-23T18:29:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 1 -- 172.21.15.33:0/2385573104 wait complete. 2024-04-23T18:29:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.416+0000 7f4994616740 20 client.16101 trim_cache size 0 max 16384 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.431+0000 7f4994616740 1 Processor -- start 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.431+0000 7f4994616740 1 -- start start 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1eca99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1ada40 con 0x558a1ecb1800 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1af580 con 0x558a1eca6620 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4994616740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1ae110 con 0x558a1eca6f40 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44276/0 (socket says 172.21.15.33:44276) 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 -- 172.21.15.33:0/621161802 learned_addr learned my addr 172.21.15.33:0/621161802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1eca99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f498affd640 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 -- 172.21.15.33:0/621161802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 msgr2=0x558a1ecb0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 -- 172.21.15.33:0/621161802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 msgr2=0x558a1eca99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1eca99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 -- 172.21.15.33:0/621161802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a1ecac980 con 0x558a1ecb1800 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f4983fff640 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4974005bd0 tx=0x7f497403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f63658a0fd6579 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f49817fa640 1 -- 172.21.15.33:0/621161802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974051020 con 0x558a1ecb1800 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f49817fa640 1 -- 172.21.15.33:0/621161802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497404f070 con 0x558a1ecb1800 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.432+0000 7f49817fa640 1 -- 172.21.15.33:0/621161802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974035040 con 0x558a1ecb1800 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- 172.21.15.33:0/621161802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 msgr2=0x558a1ecc6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4974005bd0 tx=0x7f497403f040 comp rx=0 tx=0).stop 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- 172.21.15.33:0/621161802 shutdown_connections 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1ecb1800 0x558a1ecc6a40 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1eca6f40 0x558a1ecb0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 --2- 172.21.15.33:0/621161802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1eca99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- 172.21.15.33:0/621161802 >> 172.21.15.33:0/621161802 conn(0x558a1eca2b30 msgr2=0x558a1ecbd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- 172.21.15.33:0/621161802 shutdown_connections 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- 172.21.15.33:0/621161802 wait complete. 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.433+0000 7f4994616740 1 -- start start 2024-04-23T18:29:49.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 Processor -- start 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1ecc6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1ecb1800 0x558a1ecc4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1aeeb0 con 0x558a1eca6f40 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1ae7e0 con 0x558a1eca6620 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1aba40 con 0x558a1ecb1800 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4994616740 10 client.0 _ll_register_callbacks cb 0x7f4974031630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:29:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44290/0 (socket says 172.21.15.33:44290) 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 -- 172.21.15.33:0/3677755260 learned_addr learned my addr 172.21.15.33:0/3677755260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4983fff640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1ecb1800 0x558a1ecc4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f4983fff640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1ecc6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 -- 172.21.15.33:0/3677755260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1ecb1800 msgr2=0x558a1ecc4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1ecb1800 0x558a1ecc4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 -- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 msgr2=0x558a1ecc6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1ecc6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.434+0000 7f498affd640 1 -- 172.21.15.33:0/3677755260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a1ecc4ba0 con 0x558a1eca6f40 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f498affd640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f497c00bfd0 tx=0x7f497c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b19c6c7c62c8dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c005250 con 0x558a1eca6f40 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f497c0053f0 con 0x558a1eca6f40 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497404f070 con 0x558a1eca6f40 2024-04-23T18:29:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f497c0055f0 con 0x558a1eca6f40 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 10 client.16107 fetch_fsmap learned FSMap version 30 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a1ed2baa0 con 0x558a1eca6f40 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497404f070 con 0x558a1eca6f40 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f497c004bf0 con 0x558a1eca6f40 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 20 client.16107 populate_metadata read hostname 'smithi033' 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ed2c250 0x558a1ed2e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 4 client.16107 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.435+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a1ec13200 con 0x558a1ed2c250 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.436+0000 7f4983fff640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ed2c250 0x558a1ed2e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.436+0000 7f4983fff640 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ed2c250 0x558a1ed2e670 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51dac1539a2af732 server_cookie=332ceaadc7139817 in_seq=0 out_seq=0 2024-04-23T18:29:49.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.436+0000 7f496affd640 10 client.16107 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f496affd640 1 -- 172.21.15.33:0/3677755260 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a1ec13200 con 0x558a1ed2c250 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f496affd640 10 client.16107 handle_command_reply: tid=0 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 client.16107 shutdown 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 msgr2=0x558a1ecc4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f497c00bfd0 tx=0x7f497c012820 comp rx=0 tx=0).stop 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 shutdown_connections 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558a1ed2c250 0x558a1ed2e670 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558a1ecb1800 0x558a1ecc4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558a1eca6f40 0x558a1ecc4020 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f496affd640 0 client.16107 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 --2- 172.21.15.33:0/3677755260 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558a1eca6620 0x558a1ecc6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 >> 172.21.15.33:0/3677755260 conn(0x558a1eca2b30 msgr2=0x558a1eca8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 shutdown_connections 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 1 -- 172.21.15.33:0/3677755260 wait complete. 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4994616740 20 client.16107 trim_cache size 0 max 16384 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:29:49.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 msgr2=0x7f49780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 0x7f49780a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4984098da0 tx=0x7f49840bd040 comp rx=0 tx=0).stop 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 msgr2=0x7f498c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f498400eaa0 tx=0x7f498400ef60 comp rx=0 tx=0).stop 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 shutdown_connections 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49780a3dc0 0x7f49780a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c19cf80 0x7f498c19d430 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f498c107290 0x7f498c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 --2- 172.21.15.33:0/3739990212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f498c1021c0 0x7f498c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:29:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 >> 172.21.15.33:0/3739990212 conn(0x7f498c0fda20 msgr2=0x7f498c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:29:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 shutdown_connections 2024-04-23T18:29:49.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:29:49.445+0000 7f4991907640 1 -- 172.21.15.33:0/3739990212 wait complete. 2024-04-23T18:29:49.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:49 smithi122 ceph-mon[32546]: pgmap v102: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:49.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:49 smithi033 ceph-mon[29274]: pgmap v102: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:50.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:50 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:29:50.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:50 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:29:50.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:50 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:29:51.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:51 smithi106 ceph-mon[32716]: pgmap v103: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:51.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:51 smithi122 ceph-mon[32546]: pgmap v103: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:51.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:51 smithi033 ceph-mon[29274]: pgmap v103: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.6 KiB/s wr, 9 op/s 2024-04-23T18:29:53.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:53 smithi106 ceph-mon[32716]: pgmap v104: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-23T18:29:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:53 smithi122 ceph-mon[32546]: pgmap v104: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-23T18:29:53.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:53 smithi033 ceph-mon[29274]: pgmap v104: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.0 KiB/s wr, 15 op/s 2024-04-23T18:29:55.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:55 smithi106 ceph-mon[32716]: pgmap v105: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-23T18:29:55.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:55 smithi122 ceph-mon[32546]: pgmap v105: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-23T18:29:55.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:55 smithi033 ceph-mon[29274]: pgmap v105: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-04-23T18:29:57.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:57 smithi106 ceph-mon[32716]: pgmap v106: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-23T18:29:57.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:57 smithi122 ceph-mon[32546]: pgmap v106: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-23T18:29:57.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:57 smithi033 ceph-mon[29274]: pgmap v106: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2024-04-23T18:29:59.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:29:59 smithi106 ceph-mon[32716]: pgmap v107: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:29:59.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:29:59 smithi122 ceph-mon[32546]: pgmap v107: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:29:59.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:29:59 smithi033 ceph-mon[29274]: pgmap v107: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:30:00.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:00 smithi106 ceph-mon[32716]: overall HEALTH_OK 2024-04-23T18:30:00.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:00 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:30:00.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:00 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:30:01.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:01 smithi106 ceph-mon[32716]: pgmap v108: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:30:01.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:01 smithi122 ceph-mon[32546]: pgmap v108: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:30:01.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:01 smithi033 ceph-mon[29274]: pgmap v108: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.4 KiB/s wr, 10 op/s 2024-04-23T18:30:02.270 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:30:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 -- 172.21.15.33:0/1241598683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f41010a0 msgr2=0x7fc5f4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/1241598683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f41010a0 0x7fc5f4101480 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009900 tx=0x7fc5dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 -- 172.21.15.33:0/1241598683 shutdown_connections 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/1241598683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f4106170 0x7fc5f410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/1241598683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4101a50 0x7fc5f4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/1241598683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f41010a0 0x7fc5f4101480 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 -- 172.21.15.33:0/1241598683 >> 172.21.15.33:0/1241598683 conn(0x7fc5f4077f50 msgr2=0x7fc5f4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 -- 172.21.15.33:0/1241598683 shutdown_connections 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.396+0000 7fc5fa94f640 1 -- 172.21.15.33:0/1241598683 wait complete. 2024-04-23T18:30:02.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 Processor -- start 2024-04-23T18:30:02.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 -- start start 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 0x7fc5f41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4197e00 0x7fc5f41a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4111130 con 0x7fc5f4106170 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4110fb0 con 0x7fc5f4197e00 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.397+0000 7fc5fa94f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41112b0 con 0x7fc5f41010a0 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 0x7fc5f41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:34064/0 (socket says 172.21.15.33:34064) 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 -- 172.21.15.33:0/3421983802 learned_addr learned my addr 172.21.15.33:0/3421983802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f8eec640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4197e00 0x7fc5f41a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 -- 172.21.15.33:0/3421983802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4197e00 msgr2=0x7fc5f41a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4197e00 0x7fc5f41a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 -- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 msgr2=0x7fc5f41978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 0x7fc5f41978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 -- 172.21.15.33:0/3421983802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f41a6f90 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f37fe640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 0x7fc5f41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f3fff640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005ab0 tx=0x7fc5dc003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc03f070 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f41a7220 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.398+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5dc031d60 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.399+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f41a7730 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.399+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc049060 con 0x7fc5f41010a0 2024-04-23T18:30:02.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.399+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f4068d70 con 0x7fc5f41010a0 2024-04-23T18:30:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.401+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5dc042070 con 0x7fc5f41010a0 2024-04-23T18:30:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.401+0000 7fc5f17fa640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 0x7fc5e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.401+0000 7fc5f37fe640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 0x7fc5e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.402+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc5dc0f16b0 con 0x7fc5f41010a0 2024-04-23T18:30:02.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.402+0000 7fc5f37fe640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 0x7fc5e00a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5f41994e0 tx=0x7fc5e400a2d0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.405+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5dc0bdd60 con 0x7fc5f41010a0 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2042275854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 msgr2=0x7fc0341035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2042275854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc0341035c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc028009900 tx=0x7fc02802a510 comp rx=0 tx=0).stop 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2042275854 shutdown_connections 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2042275854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 0x7fc03410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2042275854 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 0x7fc03410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2042275854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc0341035c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2042275854 >> 172.21.15.33:0/2042275854 conn(0x7fc034077fd0 msgr2=0x7fc0340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2042275854 shutdown_connections 2024-04-23T18:30:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.535+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2042275854 wait complete. 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 Processor -- start 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 -- start start 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 0x7fc034101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 0x7fc0341018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112190 con 0x7fc0341031e0 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112010 con 0x7fc034103b00 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc03ab4b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc034112310 con 0x7fc03410c520 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc0388e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc0388e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49136/0 (socket says 172.21.15.33:49136) 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc0388e7640 1 -- 172.21.15.33:0/2923043289 learned_addr learned my addr 172.21.15.33:0/2923043289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 0x7fc0341018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.536+0000 7fc033fff640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 0x7fc034101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 -- 172.21.15.33:0/2923043289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 msgr2=0x7fc0341018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 0x7fc0341018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 -- 172.21.15.33:0/2923043289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 msgr2=0x7fc034101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 0x7fc034101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 -- 172.21.15.33:0/2923043289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0341a6bb0 con 0x7fc0341031e0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc028005940 tx=0x7fc028004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc031ffb640 1 -- 172.21.15.33:0/2923043289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02803f030 con 0x7fc0341031e0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0341a6e40 con 0x7fc0341031e0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc031ffb640 1 -- 172.21.15.33:0/2923043289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc028003aa0 con 0x7fc0341031e0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc031ffb640 1 -- 172.21.15.33:0/2923043289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028035070 con 0x7fc0341031e0 2024-04-23T18:30:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.537+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a72a0 con 0x7fc0341031e0 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.539+0000 7fc03d861740 1 Processor -- start 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 -- start start 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc031ffb640 1 -- 172.21.15.33:0/2923043289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc028042030 con 0x7fc0341031e0 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1f40 0x55c8091ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c809106970 con 0x55c8091ac800 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c8090a34a0 con 0x55c8091a1620 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc03d861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c809102930 con 0x55c8091a1f40 2024-04-23T18:30:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49142/0 (socket says 172.21.15.33:49142) 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 -- 172.21.15.33:0/603355233 learned_addr learned my addr 172.21.15.33:0/603355233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc0390e8640 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc0388e7640 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1f40 0x55c8091ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 -- 172.21.15.33:0/603355233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1f40 msgr2=0x55c8091ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1f40 0x55c8091ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 -- 172.21.15.33:0/603355233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 msgr2=0x55c8091a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc031ffb640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 0x7fc0240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.540+0000 7fc033fff640 1 -- 172.21.15.33:0/603355233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c808e19210 con 0x55c8091ac800 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 0x7fc0240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc031ffb640 1 -- 172.21.15.33:0/2923043289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc0280f0e20 con 0x7fc0341031e0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc033fff640 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc020009c90 tx=0x7fc020011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23e71deba556f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc01effd640 1 -- 172.21.15.33:0/603355233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02001e070 con 0x55c8091ac800 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc01effd640 1 -- 172.21.15.33:0/603355233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc020019ae0 con 0x55c8091ac800 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc01effd640 1 -- 172.21.15.33:0/603355233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02000e940 con 0x55c8091ac800 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc03d861740 1 -- 172.21.15.33:0/603355233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 msgr2=0x55c8091aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 0x7fc0240a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c009600 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.541+0000 7fc03d861740 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc020009c90 tx=0x7fc020011810 comp rx=0 tx=0).stop 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- 172.21.15.33:0/603355233 shutdown_connections 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091ac800 0x55c8091aebf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1f40 0x55c8091ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- 172.21.15.33:0/603355233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- 172.21.15.33:0/603355233 >> 172.21.15.33:0/603355233 conn(0x55c80919db30 msgr2=0x55c80919df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- 172.21.15.33:0/603355233 shutdown_connections 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- 172.21.15.33:0/603355233 wait complete. 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- start start 2024-04-23T18:30:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 Processor -- start 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091ac800 0x55c8091a9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c8091af430 con 0x55c8091a1f40 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c8090a10b0 con 0x55c8091a1620 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c8091af5b0 con 0x55c8091ac800 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.542+0000 7fc03d861740 10 client.0 _ll_register_callbacks cb 0x55c80908ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49146/0 (socket says 172.21.15.33:49146) 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 -- 172.21.15.33:0/1493343832 learned_addr learned my addr 172.21.15.33:0/1493343832 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc0388e7640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc0390e8640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091ac800 0x55c8091a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 -- 172.21.15.33:0/1493343832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091ac800 msgr2=0x55c8091a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091ac800 0x55c8091a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 -- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 msgr2=0x55c8091ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 -- 172.21.15.33:0/1493343832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c808ee5620 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc033fff640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc020019a30 tx=0x7fc020009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a42cdb306b6b4a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc013fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02001e050 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc020005450 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc020019ae0 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc020032050 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.543+0000 7fc03d861740 10 client.16125 fetch_fsmap learned FSMap version 30 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8091bb2d0 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc020019ae0 con 0x55c8091a1f40 2024-04-23T18:30:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc02002d030 con 0x55c8091a1f40 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 10 client.16125 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 20 client.16125 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091bc6d0 0x55c8091beac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 4 client.16125 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c809114390 con 0x55c8091bc6d0 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.544+0000 7fc0388e7640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091bc6d0 0x55c8091beac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.545+0000 7fc0388e7640 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091bc6d0 0x55c8091beac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c2e4db31764620 server_cookie=db5503146729d2e5 in_seq=0 out_seq=0 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.545+0000 7fc013fff640 10 client.16125 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.545+0000 7fc013fff640 1 -- 172.21.15.33:0/1493343832 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c809114390 con 0x55c8091bc6d0 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.545+0000 7fc013fff640 10 client.16125 handle_command_reply: tid=0 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 client.16125 shutdown 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 msgr2=0x55c8091a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc020019a30 tx=0x7fc020009850 comp rx=0 tx=0).stop 2024-04-23T18:30:02.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 shutdown_connections 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091bc6d0 0x55c8091beac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091ac800 0x55c8091a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1f40 0x55c8091a93c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc013fff640 0 client.16125 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 --2- 172.21.15.33:0/1493343832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1620 0x55c8091ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 >> 172.21.15.33:0/1493343832 conn(0x55c80919db30 msgr2=0x55c8091a3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 shutdown_connections 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 1 -- 172.21.15.33:0/1493343832 wait complete. 2024-04-23T18:30:02.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.546+0000 7fc03d861740 20 client.16125 trim_cache size 0 max 16384 2024-04-23T18:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 Processor -- start 2024-04-23T18:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 -- start start 2024-04-23T18:30:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 0x55c80919cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341afd50 con 0x55c80919db30 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b1890 con 0x55c8091a1f40 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc03d861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b0420 con 0x55c8091a1620 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49156/0 (socket says 172.21.15.33:49156) 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 -- 172.21.15.33:0/2895613938 learned_addr learned my addr 172.21.15.33:0/2895613938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc033fff640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 0x55c80919cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 -- 172.21.15.33:0/2895613938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 msgr2=0x55c80919cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 0x55c80919cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 -- 172.21.15.33:0/2895613938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 msgr2=0x55c8091c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0388e7640 1 -- 172.21.15.33:0/2895613938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8091bee30 con 0x55c80919db30 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc033fff640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 0x55c80919cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.561+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc0280b92e0 tx=0x7fc0280dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d0199a4b44e1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc01e7fc640 1 -- 172.21.15.33:0/2895613938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028049040 con 0x55c80919db30 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc01e7fc640 1 -- 172.21.15.33:0/2895613938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0280bce80 con 0x55c80919db30 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc01e7fc640 1 -- 172.21.15.33:0/2895613938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0280b94f0 con 0x55c80919db30 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 -- 172.21.15.33:0/2895613938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 msgr2=0x55c8091b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc0280b92e0 tx=0x7fc0280dd040 comp rx=0 tx=0).stop 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 -- 172.21.15.33:0/2895613938 shutdown_connections 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c8091a1620 0x55c80919cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 --2- 172.21.15.33:0/2895613938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c80919db30 0x55c8091b82f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 -- 172.21.15.33:0/2895613938 >> 172.21.15.33:0/2895613938 conn(0x55c8091a9a60 msgr2=0x55c80919c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.562+0000 7fc03d861740 1 -- 172.21.15.33:0/2895613938 shutdown_connections 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 -- 172.21.15.33:0/2895613938 wait complete. 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 -- start start 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 Processor -- start 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 0x55c8091b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b11c0 con 0x55c8091a1620 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341b0af0 con 0x55c8091a1f40 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0341add30 con 0x55c80919db30 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc03d861740 10 client.0 _ll_register_callbacks cb 0x7fc020009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49166/0 (socket says 172.21.15.33:49166) 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.563+0000 7fc033fff640 1 -- 172.21.15.33:0/2187997529 learned_addr learned my addr 172.21.15.33:0/2187997529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc0390e8640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 0x55c8091b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 -- 172.21.15.33:0/2187997529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 msgr2=0x55c8091b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 0x55c8091b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 -- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 msgr2=0x55c8091b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 -- 172.21.15.33:0/2187997529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8091b2820 con 0x55c8091a1620 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 0x55c8091b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc033fff640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c8091aa990 tx=0x7fc020004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0358addb5a6149 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc013fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020015020 con 0x55c8091a1620 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7fc020005450 con 0x55c8091a1620 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0280bce80 con 0x55c8091a1620 2024-04-23T18:30:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc02001e030 con 0x55c8091a1620 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc03d861740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8092228a0 con 0x55c8091a1620 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.564+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0280bce80 con 0x55c8091a1620 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc02002d040 con 0x55c8091a1620 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 20 client.16131 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091ac800 0x55c8092250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 4 client.16131 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c80910e200 con 0x55c8091ac800 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091ac800 0x55c8092250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc013fff640 10 client.16131 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:02.565+0000 7fc0388e7640 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091ac800 0x55c8092250b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13045a4ab9ab61d server_cookie=3619c0a8d108e6a5 in_seq=0 out_seq=0 2024-04-23T18:30:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.640+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5f4069680 con 0x7fc5f41010a0 2024-04-23T18:30:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.641+0000 7fc5f17fa640 1 -- 172.21.15.33:0/3421983802 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7fc5f4069680 con 0x7fc5f41010a0 2024-04-23T18:30:02.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:30:02.634 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:29:48.162758+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3209242154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3209242154},{"type":"v1","addr":"172.21.15.122:6835","nonce":3209242154}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24466,24482,24551,24490]},"id":1}]} 2024-04-23T18:30:02.634 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 msgr2=0x7fc5e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 0x7fc5e00a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5f41994e0 tx=0x7fc5e400a2d0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 msgr2=0x7fc5f4197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc005ab0 tx=0x7fc5dc003cd0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 shutdown_connections 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5e00a3d20 0x7fc5e00a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5f4197e00 0x7fc5f41a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f4106170 0x7fc5f41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 --2- 172.21.15.33:0/3421983802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5f41010a0 0x7fc5f4197380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 >> 172.21.15.33:0/3421983802 conn(0x7fc5f4077f50 msgr2=0x7fc5f40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 shutdown_connections 2024-04-23T18:30:02.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.643+0000 7fc5fa94f640 1 -- 172.21.15.33:0/3421983802 wait complete. 2024-04-23T18:30:02.647 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 93656ef1 --release --await' 2024-04-23T18:30:02.647 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 93656ef1 --release --await 2024-04-23T18:30:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 -- 172.21.15.33:0/884678277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f4107270 msgr2=0x7f70f410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 --2- 172.21.15.33:0/884678277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f4107270 0x7f70f410ee40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009ca0 tx=0x7f70ec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 -- 172.21.15.33:0/884678277 shutdown_connections 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 --2- 172.21.15.33:0/884678277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f4107270 0x7f70f410ee40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 --2- 172.21.15.33:0/884678277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406b1f0 0x7f70f4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 --2- 172.21.15.33:0/884678277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f406a840 0x7f70f406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 -- 172.21.15.33:0/884678277 >> 172.21.15.33:0/884678277 conn(0x7f70f4077eb0 msgr2=0x7f70f40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 -- 172.21.15.33:0/884678277 shutdown_connections 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.764+0000 7f70f887c640 1 -- 172.21.15.33:0/884678277 wait complete. 2024-04-23T18:30:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.765+0000 7f70f887c640 1 Processor -- start 2024-04-23T18:30:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.765+0000 7f70f887c640 1 -- start start 2024-04-23T18:30:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406a840 0x7f70f4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f4107270 0x7f70f4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41121b0 con 0x7f70f419cf30 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4112030 con 0x7f70f406a840 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70f4112330 con 0x7f70f4107270 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49192/0 (socket says 172.21.15.33:49192) 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 -- 172.21.15.33:0/2809285357 learned_addr learned my addr 172.21.15.33:0/2809285357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f4107270 0x7f70f4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f259c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406a840 0x7f70f4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 -- 172.21.15.33:0/2809285357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f4107270 msgr2=0x7f70f4198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f4107270 0x7f70f4198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 -- 172.21.15.33:0/2809285357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406a840 msgr2=0x7f70f4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406a840 0x7f70f4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 -- 172.21.15.33:0/2809285357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f419da20 con 0x7f70f419cf30 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f2d9d640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005cc0 tx=0x7f70ec031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70ebfff640 1 -- 172.21.15.33:0/2809285357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec003b70 con 0x7f70f419cf30 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f419dcb0 con 0x7f70f419cf30 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.766+0000 7f70ebfff640 1 -- 172.21.15.33:0/2809285357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70ec0318f0 con 0x7f70f419cf30 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.767+0000 7f70ebfff640 1 -- 172.21.15.33:0/2809285357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec02f950 con 0x7f70f419cf30 2024-04-23T18:30:02.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.767+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f41a5500 con 0x7f70f419cf30 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70ebfff640 1 -- 172.21.15.33:0/2809285357 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70ec02fb50 con 0x7f70f419cf30 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 Processor -- start 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 -- start start 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb337f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb33a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562deb2926d0 con 0x562deb32d490 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562deb22f1a0 con 0x562deb338480 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70fb591740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562deb28e690 con 0x562deb32ddb0 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70ebfff640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 0x7f70e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49206/0 (socket says 172.21.15.33:49206) 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.769+0000 7f70e9d9b640 1 -- 172.21.15.33:0/846611488 learned_addr learned my addr 172.21.15.33:0/846611488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70f2d9d640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 0x7f70e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70f259c640 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb337f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70ebfff640 1 -- 172.21.15.33:0/2809285357 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f70ec0f19c0 con 0x7f70f419cf30 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb33a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 -- 172.21.15.33:0/846611488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 msgr2=0x562deb337f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb337f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 -- 172.21.15.33:0/846611488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 msgr2=0x562deb33a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb33a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 -- 172.21.15.33:0/846611488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562deb2395b0 con 0x562deb32d490 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70f2d9d640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 0x7f70e00a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0ef2e0 tx=0x7f70ec02a8c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009f70 tx=0x7f70dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97287f15eabf64f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e8d99640 1 -- 172.21.15.33:0/846611488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc004050 con 0x562deb32d490 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e8d99640 1 -- 172.21.15.33:0/846611488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0041f0 con 0x562deb32d490 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.770+0000 7f70e8d99640 1 -- 172.21.15.33:0/846611488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc00f500 con 0x562deb32d490 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- 172.21.15.33:0/846611488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 msgr2=0x562deb32d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009f70 tx=0x7f70dc005850 comp rx=0 tx=0).stop 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- 172.21.15.33:0/846611488 shutdown_connections 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb33a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb337f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 --2- 172.21.15.33:0/846611488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32d490 0x562deb32d870 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- 172.21.15.33:0/846611488 >> 172.21.15.33:0/846611488 conn(0x562deb3297b0 msgr2=0x562deb329b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- 172.21.15.33:0/846611488 shutdown_connections 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- 172.21.15.33:0/846611488 wait complete. 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 -- start start 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.771+0000 7f70fb591740 1 Processor -- start 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32d490 0x562deb337e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb335770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562deb33b310 con 0x562deb32ddb0 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562deb22cb20 con 0x562deb338480 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562deb33b490 con 0x562deb32d490 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70fb591740 10 client.0 _ll_register_callbacks cb 0x562deb22c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49220/0 (socket says 172.21.15.33:49220) 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 -- 172.21.15.33:0/3025055393 learned_addr learned my addr 172.21.15.33:0/3025055393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32d490 0x562deb337e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 -- 172.21.15.33:0/3025055393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32d490 msgr2=0x562deb337e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32d490 0x562deb337e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 -- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 msgr2=0x562deb335770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb335770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb335770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 -- 172.21.15.33:0/3025055393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562deafa4920 con 0x562deb32ddb0 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb335770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70f259c640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f70e4012740 tx=0x7f70e4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30591d2813ae9a74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400f920 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.772+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f70e400fac0 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc0041f0 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70e400fcc0 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 10 client.16143 fetch_fsmap learned FSMap version 30 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562deb070d30 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0041f0 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f70e401c420 con 0x562deb32ddb0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 10 client.16143 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 10 client.16143 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 20 client.16143 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3487a0 0x562deb34abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 4 client.16143 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562deb250500 con 0x562deb3487a0 2024-04-23T18:30:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.773+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3487a0 0x562deb34abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.774+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3487a0 0x562deb34abe0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42aa5f06740cf04 server_cookie=8a2e093380065cab in_seq=0 out_seq=0 2024-04-23T18:30:02.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:02.774+0000 7f70d9ffb640 10 client.16143 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.165+0000 7fc013fff640 1 -- 172.21.15.33:0/2187997529 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c80910e200 con 0x55c8091ac800 2024-04-23T18:30:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.165+0000 7fc013fff640 10 client.16131 handle_command_reply: tid=0 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.165+0000 7fc03d861740 1 client.16131 shutdown 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.165+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 msgr2=0x55c8091b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.165+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x55c8091aa990 tx=0x7fc020004e10 comp rx=0 tx=0).stop 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 shutdown_connections 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c8091ac800 0x55c8092250b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c8091a1f40 0x55c8091b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c8091a1620 0x55c8091b1d20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc013fff640 0 client.16131 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 --2- 172.21.15.33:0/2187997529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c80919db30 0x55c8091b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 >> 172.21.15.33:0/2187997529 conn(0x55c8091a9a60 msgr2=0x55c80919ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70d9ffb640 1 -- 172.21.15.33:0/3025055393 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562deb250500 con 0x562deb3487a0 2024-04-23T18:30:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70d9ffb640 10 client.16143 handle_command_reply: tid=0 2024-04-23T18:30:03.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 client.16143 shutdown 2024-04-23T18:30:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 shutdown_connections 2024-04-23T18:30:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 1 -- 172.21.15.33:0/2187997529 wait complete. 2024-04-23T18:30:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.166+0000 7fc03d861740 20 client.16131 trim_cache size 0 max 16384 2024-04-23T18:30:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 msgr2=0x562deb335160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f70e4012740 tx=0x7f70e4012c00 comp rx=0 tx=0).stop 2024-04-23T18:30:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 shutdown_connections 2024-04-23T18:30:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3487a0 0x562deb34abe0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb338480 0x562deb335770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb335160 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 --2- 172.21.15.33:0/3025055393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32d490 0x562deb337e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 >> 172.21.15.33:0/3025055393 conn(0x562deb3297b0 msgr2=0x562deb32f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70d9ffb640 0 client.16143 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:03.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 shutdown_connections 2024-04-23T18:30:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:30:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:30:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:30:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:30:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 1 -- 172.21.15.33:0/3025055393 wait complete. 2024-04-23T18:30:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.166+0000 7f70fb591740 20 client.16143 trim_cache size 0 max 16384 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 msgr2=0x7fc0240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 0x7fc0240a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc02c005fd0 tx=0x7fc02c009600 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 msgr2=0x7fc034100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc028005940 tx=0x7fc028004270 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 shutdown_connections 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc0240a3dc0 0x7fc0240a6280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc03410c520 0x7fc0341018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc034103b00 0x7fc034101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 --2- 172.21.15.33:0/2923043289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0341031e0 0x7fc034100e30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 >> 172.21.15.33:0/2923043289 conn(0x7fc034077fd0 msgr2=0x7fc0340fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 shutdown_connections 2024-04-23T18:30:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:03.167+0000 7fc03ab4b640 1 -- 172.21.15.33:0/2923043289 wait complete. 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 Processor -- start 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 -- start start 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb328850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb3379a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41ada40 con 0x562deb338480 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41af580 con 0x562deb32d490 2024-04-23T18:30:03.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.184+0000 7f70fb591740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41ae110 con 0x562deb32ddb0 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49230/0 (socket says 172.21.15.33:49230) 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 -- 172.21.15.33:0/707670251 learned_addr learned my addr 172.21.15.33:0/707670251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb328850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70f259c640 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb3379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 -- 172.21.15.33:0/707670251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 msgr2=0x562deb3379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb3379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 -- 172.21.15.33:0/707670251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 msgr2=0x562deb328850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb328850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 -- 172.21.15.33:0/707670251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562deb335ae0 con 0x562deb338480 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005c20 tx=0x7f70dc003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6019c578f7816926 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70dbfff640 1 -- 172.21.15.33:0/707670251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc028040 con 0x562deb338480 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70dbfff640 1 -- 172.21.15.33:0/707670251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0055a0 con 0x562deb338480 2024-04-23T18:30:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.185+0000 7f70dbfff640 1 -- 172.21.15.33:0/707670251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc02b6c0 con 0x562deb338480 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- 172.21.15.33:0/707670251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 msgr2=0x562deb3504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f70dc005c20 tx=0x7f70dc003c40 comp rx=0 tx=0).stop 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- 172.21.15.33:0/707670251 shutdown_connections 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb338480 0x562deb3504b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb32ddb0 0x562deb3379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 --2- 172.21.15.33:0/707670251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb328850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- 172.21.15.33:0/707670251 >> 172.21.15.33:0/707670251 conn(0x562deb3297b0 msgr2=0x562deb328140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- 172.21.15.33:0/707670251 shutdown_connections 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- 172.21.15.33:0/707670251 wait complete. 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.186+0000 7f70fb591740 1 -- start start 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 Processor -- start 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb340bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 0x562deb332b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41aeeb0 con 0x562deb32ddb0 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41ae7e0 con 0x562deb32d490 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70f41aba20 con 0x562deb338480 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70fb591740 10 client.0 _ll_register_callbacks cb 0x7f70e4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49236/0 (socket says 172.21.15.33:49236) 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 -- 172.21.15.33:0/556390928 learned_addr learned my addr 172.21.15.33:0/556390928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 0x562deb332b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 -- 172.21.15.33:0/556390928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 msgr2=0x562deb332b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 0x562deb332b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 -- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 msgr2=0x562deb340bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb340bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 -- 172.21.15.33:0/556390928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562deb339190 con 0x562deb32ddb0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 0x562deb332b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70f259c640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e401c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dffc7368ee221eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.187+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e400fe60 con 0x562deb32ddb0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8498+0+0 (secure 0 0 0) 0x7f70e401cd10 con 0x562deb32ddb0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70dc0055a0 con 0x562deb32ddb0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70e4033050 con 0x562deb32ddb0 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 10 client.16149 fetch_fsmap learned FSMap version 30 2024-04-23T18:30:03.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562deb3af010 con 0x562deb32ddb0 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70dc0055a0 con 0x562deb32ddb0 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f70e402b020 con 0x562deb32ddb0 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 10 client.16149 fetch_fsmap finished waiting for FSMap version 30 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 10 client.16149 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 20 client.16149 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3af810 0x562deb3b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 4 client.16149 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "93656ef1", "release": true, "await": true}] 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "93656ef1", "release": true, "await": true}) -- 0x562deb2a00f0 con 0x562deb3af810 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.188+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3af810 0x562deb3b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.189+0000 7f70d9ffb640 10 client.16149 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:03.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.189+0000 7f70e9d9b640 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3af810 0x562deb3b1c30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1143f8dec87a8e server_cookie=c72a0e13975f68eb in_seq=0 out_seq=0 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70d9ffb640 1 -- 172.21.15.33:0/556390928 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562deb2a00f0 con 0x562deb3af810 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70d9ffb640 10 client.16149 handle_command_reply: tid=0 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 client.16149 shutdown 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 msgr2=0x562deb3325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f70e4000c00 tx=0x7f70e401c6d0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 shutdown_connections 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562deb3af810 0x562deb3b1c30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562deb338480 0x562deb332b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70d9ffb640 0 client.16149 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562deb32ddb0 0x562deb3325c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 --2- 172.21.15.33:0/556390928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562deb32d490 0x562deb340bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 >> 172.21.15.33:0/556390928 conn(0x562deb3297b0 msgr2=0x562deb3289d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.191+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 shutdown_connections 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70fb591740 1 -- 172.21.15.33:0/556390928 wait complete. 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70fb591740 20 client.16149 trim_cache size 0 max 16384 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 30, 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 16, 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "93656ef1": { 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 16, 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:30:03.189 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:30:03.190 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:30:03.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:30:03.191 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:30:03.191 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 msgr2=0x7f70e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 0x7f70e00a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f70ec0ef2e0 tx=0x7f70ec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 msgr2=0x7f70f419d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005cc0 tx=0x7f70ec031700 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 shutdown_connections 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70e00a3d20 0x7f70e00a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70f419cf30 0x7f70f419d3e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70f4107270 0x7f70f4198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 --2- 172.21.15.33:0/2809285357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70f406a840 0x7f70f4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 >> 172.21.15.33:0/2809285357 conn(0x7f70f4077eb0 msgr2=0x7f70f40fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 shutdown_connections 2024-04-23T18:30:03.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:30:03.192+0000 7f70f887c640 1 -- 172.21.15.33:0/2809285357 wait complete. 2024-04-23T18:30:03.196 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '93656ef1', seconds elapsed: 31.4 2024-04-23T18:30:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:03 smithi106 ceph-mon[32716]: pgmap v109: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-23T18:30:03.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:03 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3421983802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:03.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-mon[32546]: pgmap v109: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-23T18:30:03.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3421983802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:03.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:03 smithi033 ceph-mon[29274]: pgmap v109: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.4 KiB/s wr, 14 op/s 2024-04-23T18:30:03.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:03 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3421983802' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: in thread 7f325d123700 thread_name:md_log_replay 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f326e4e4d20] 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2: /usr/bin/ceph-mds(+0x549372) [0x564d4273a372] 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564d4273db23] 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 4: (MDLog::_replay_thread()+0x617) [0x564d42798647] 2024-04-23T18:30:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564d427a1d31] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f326e4da1ca] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 7: clone() 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:03.878+0000 7f325d123700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: in thread 7f325d123700 thread_name:md_log_replay 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f326e4e4d20] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2: /usr/bin/ceph-mds(+0x549372) [0x564d4273a372] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564d4273db23] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 4: (MDLog::_replay_thread()+0x617) [0x564d42798647] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564d427a1d31] 2024-04-23T18:30:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f326e4da1ca] 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 7: clone() 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 0> 2024-04-23T18:30:03.878+0000 7f325d123700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: in thread 7f325d123700 thread_name:md_log_replay 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f326e4e4d20] 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2: /usr/bin/ceph-mds(+0x549372) [0x564d4273a372] 2024-04-23T18:30:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564d4273db23] 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 4: (MDLog::_replay_thread()+0x617) [0x564d42798647] 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564d427a1d31] 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f326e4da1ca] 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 7: clone() 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 0> 2024-04-23T18:30:03.878+0000 7f325d123700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: in thread 7f325d123700 thread_name:md_log_replay 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:30:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f326e4e4d20] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2: /usr/bin/ceph-mds(+0x549372) [0x564d4273a372] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x564d4273db23] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 4: (MDLog::_replay_thread()+0x617) [0x564d42798647] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 5: (MDLog::ReplayThread::entry()+0x11) [0x564d427a1d31] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f326e4da1ca] 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 7: clone() 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[62951]: 2024-04-23T18:30:04.203 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:03 smithi122 podman[67172]: 2024-04-23 18:30:03.945143752 +0000 UTC m=+0.022972317 container died bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:30:05.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:05 smithi122 ceph-mon[32546]: pgmap v110: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:30:05.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:04 smithi122 podman[67172]: 2024-04-23 18:30:04.882475725 +0000 UTC m=+0.960304292 container remove bfb12c9012e27c49446cd2b38a10f846f2a6a59675b37506574ffd84b2881b80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T18:30:05.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:30:05.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:05 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:30:05.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:05 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.192s CPU time. 2024-04-23T18:30:05.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:05 smithi106 ceph-mon[32716]: pgmap v110: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:30:05.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:05 smithi033 ceph-mon[29274]: pgmap v110: 97 pgs: 97 active+clean; 2.2 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:30:07.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:07 smithi106 ceph-mon[32716]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 29 op/s 2024-04-23T18:30:07.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:07 smithi122 ceph-mon[32546]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 29 op/s 2024-04-23T18:30:07.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:07 smithi033 ceph-mon[29274]: pgmap v111: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 27 KiB/s wr, 29 op/s 2024-04-23T18:30:09.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:09 smithi122 ceph-mon[32546]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:09.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:09 smithi033 ceph-mon[29274]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:09.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:09 smithi106 ceph-mon[32716]: pgmap v112: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:11.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:11 smithi122 ceph-mon[32546]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:11.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:11 smithi033 ceph-mon[29274]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:11.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:11 smithi106 ceph-mon[32716]: pgmap v113: 97 pgs: 97 active+clean; 2.3 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-04-23T18:30:13.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:13 smithi122 ceph-mon[32546]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:30:13.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:13 smithi033 ceph-mon[29274]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:30:13.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:13 smithi106 ceph-mon[32716]: pgmap v114: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:30:15.431 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:15 smithi122 ceph-mon[32546]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-23T18:30:15.431 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-04-23T18:30:15.431 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:30:15.431 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Consumed 1.192s CPU time. 2024-04-23T18:30:15.431 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:30:15.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:15 smithi033 ceph-mon[29274]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-23T18:30:15.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:15 smithi106 ceph-mon[32716]: pgmap v115: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 34 KiB/s wr, 26 op/s 2024-04-23T18:30:15.698 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 podman[67282]: 2024-04-23 18:30:15.3414768 +0000 UTC m=+0.017812679 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:30:15.698 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 podman[67282]: 2024-04-23 18:30:15.450401153 +0000 UTC m=+0.126737059 container create 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-23T18:30:15.698 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 podman[67282]: 2024-04-23 18:30:15.686404199 +0000 UTC m=+0.362740081 container init 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:30:15.698 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 podman[67282]: 2024-04-23 18:30:15.689018881 +0000 UTC m=+0.365354747 container start 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:30:16.142 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 ceph-mds[67301]: -- 172.21.15.122:0/3637268056 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a490acda40 con 0x55a490aa0800 2024-04-23T18:30:16.142 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: starting mds.f at 2024-04-23T18:30:16.142 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 bash[67282]: 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 2024-04-23T18:30:16.142 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:30:15 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 -- 172.21.15.33:0/40222860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 msgr2=0x7f382410c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 --2- 172.21.15.33:0/40222860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f382410c040 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3814009900 tx=0x7f381402a480 comp rx=0 tx=0).stop 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 -- 172.21.15.33:0/40222860 shutdown_connections 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 --2- 172.21.15.33:0/40222860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 0x7f382410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 --2- 172.21.15.33:0/40222860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f382410c040 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 --2- 172.21.15.33:0/40222860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 0x7f3824103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 -- 172.21.15.33:0/40222860 >> 172.21.15.33:0/40222860 conn(0x7f3824077fd0 msgr2=0x7f38240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 -- 172.21.15.33:0/40222860 shutdown_connections 2024-04-23T18:30:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.265+0000 7f382b2bb640 1 -- 172.21.15.33:0/40222860 wait complete. 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 Processor -- start 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 -- start start 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 0x7f3824100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 0x7f3824101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38241121a0 con 0x7f3824103980 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3824112020 con 0x7f3824103060 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f382b2bb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3824112320 con 0x7f382410c580 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33048/0 (socket says 172.21.15.33:33048) 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.266+0000 7f3828856640 1 -- 172.21.15.33:0/2720940226 learned_addr learned my addr 172.21.15.33:0/2720940226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3829858640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 0x7f3824101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3829057640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 0x7f3824100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 -- 172.21.15.33:0/2720940226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 msgr2=0x7f3824101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 0x7f3824101910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 -- 172.21.15.33:0/2720940226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 msgr2=0x7f3824100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 0x7f3824100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 -- 172.21.15.33:0/2720940226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38241a6be0 con 0x7f3824103980 2024-04-23T18:30:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f3828856640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3814005940 tx=0x7f38140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f381a7fc640 1 -- 172.21.15.33:0/2720940226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381403f030 con 0x7f3824103980 2024-04-23T18:30:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f381a7fc640 1 -- 172.21.15.33:0/2720940226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3814003cb0 con 0x7f3824103980 2024-04-23T18:30:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38241a6e70 con 0x7f3824103980 2024-04-23T18:30:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f381a7fc640 1 -- 172.21.15.33:0/2720940226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814035070 con 0x7f3824103980 2024-04-23T18:30:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.267+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38241a72e0 con 0x7f3824103980 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.270+0000 7f381a7fc640 1 -- 172.21.15.33:0/2720940226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3814042030 con 0x7f3824103980 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.270+0000 7f382dfcf740 1 Processor -- start 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.270+0000 7f382dfcf740 1 -- start start 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638acbc0970 con 0x5638acc5b620 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5638acb5d4a0 con 0x5638acc5bf40 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f382dfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5638acbbc930 con 0x5638acc66800 2024-04-23T18:30:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f381a7fc640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 0x7f38100a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33050/0 (socket says 172.21.15.33:33050) 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 -- 172.21.15.33:0/1405199698 learned_addr learned my addr 172.21.15.33:0/1405199698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829057640 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3828856640 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 0x7f38100a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f381a7fc640 1 -- 172.21.15.33:0/2720940226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f38140f0c40 con 0x7f3824103980 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 -- 172.21.15.33:0/1405199698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 msgr2=0x5638acc68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc68bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 -- 172.21.15.33:0/1405199698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 msgr2=0x5638acc662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 -- 172.21.15.33:0/1405199698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638ac8d3210 con 0x5638acc5b620 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.271+0000 7f3829858640 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f381c014a60 tx=0x7f381c014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256fa2f6f2aa2a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f380f7fe640 1 -- 172.21.15.33:0/1405199698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c01ec50 con 0x5638acc5b620 2024-04-23T18:30:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f380f7fe640 1 -- 172.21.15.33:0/1405199698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381c023070 con 0x5638acc5b620 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f380f7fe640 1 -- 172.21.15.33:0/1405199698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c01dca0 con 0x5638acc5b620 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 -- 172.21.15.33:0/1405199698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 msgr2=0x5638acc5ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f3829858640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 0x7f38100a6330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f381c022000 tx=0x7f381c01e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f381c014a60 tx=0x7f381c014f20 comp rx=0 tx=0).stop 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 -- 172.21.15.33:0/1405199698 shutdown_connections 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1405199698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5b620 0x5638acc5ba00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 -- 172.21.15.33:0/1405199698 >> 172.21.15.33:0/1405199698 conn(0x5638acc57b30 msgr2=0x5638acc57f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 -- 172.21.15.33:0/1405199698 shutdown_connections 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.272+0000 7f382dfcf740 1 -- 172.21.15.33:0/1405199698 wait complete. 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 -- start start 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 Processor -- start 2024-04-23T18:30:16.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc66190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc66800 0x5638acc63a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638acc69430 con 0x5638acc5bf40 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5638acb5b0b0 con 0x5638acc66800 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5638acc695b0 con 0x5638acc5b620 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f382dfcf740 10 client.0 _ll_register_callbacks cb 0x5638acb48d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33052/0 (socket says 172.21.15.33:33052) 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.273+0000 7f3828856640 1 -- 172.21.15.33:0/1718090148 learned_addr learned my addr 172.21.15.33:0/1718090148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3829858640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc66800 0x5638acc63a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3829057640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc66190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 -- 172.21.15.33:0/1718090148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 msgr2=0x5638acc66190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc66190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 -- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc66800 msgr2=0x5638acc63a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc66800 0x5638acc63a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 -- 172.21.15.33:0/1718090148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638ac99f620 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3829057640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc66190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f3828856640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f38140bc8f0 tx=0x7f381408b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f9efa895e3296f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f380cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814049040 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f38140ee620 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381c023070 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3814005d50 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f382dfcf740 10 client.16161 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.274+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5638acc753c0 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381c023070 con 0x5638acc5bf40 2024-04-23T18:30:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f381408b1e0 con 0x5638acc5bf40 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 10 client.16161 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 20 client.16161 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638acc76820 0x5638acc78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 4 client.16161 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638acbce390 con 0x5638acc76820 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f3829057640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638acc76820 0x5638acc78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f3829057640 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638acc76820 0x5638acc78c10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e4faa936789c5d server_cookie=ee1f3e62d1f87802 in_seq=0 out_seq=0 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.275+0000 7f380cff9640 10 client.16161 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f380cff9640 1 -- 172.21.15.33:0/1718090148 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638acbce390 con 0x5638acc76820 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f380cff9640 10 client.16161 handle_command_reply: tid=0 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 client.16161 shutdown 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 msgr2=0x5638acc63480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f38140bc8f0 tx=0x7f381408b5a0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 shutdown_connections 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638acc76820 0x5638acc78c10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc66800 0x5638acc63a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc63480 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f380cff9640 0 client.16161 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 --2- 172.21.15.33:0/1718090148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc66190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.276+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 >> 172.21.15.33:0/1718090148 conn(0x5638acc57b30 msgr2=0x5638acc5d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.277+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 shutdown_connections 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.277+0000 7f382dfcf740 1 -- 172.21.15.33:0/1718090148 wait complete. 2024-04-23T18:30:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.277+0000 7f382dfcf740 20 client.16161 trim_cache size 0 max 16384 2024-04-23T18:30:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 Processor -- start 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 -- start start 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc7e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38241afc90 con 0x5638acc66800 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38241b17d0 con 0x5638acc5bf40 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f382dfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38241b0360 con 0x5638acc5b620 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f3829858640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f3829057640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f3829858640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33062/0 (socket says 172.21.15.33:33062) 2024-04-23T18:30:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f3829858640 1 -- 172.21.15.33:0/2756508088 learned_addr learned my addr 172.21.15.33:0/2756508088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.292+0000 7f3828856640 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc7e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 -- 172.21.15.33:0/2756508088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 msgr2=0x5638acc722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 -- 172.21.15.33:0/2756508088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 msgr2=0x5638acc7e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc7e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 -- 172.21.15.33:0/2756508088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638acc78f80 con 0x5638acc66800 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f3829858640 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f381c01ede0 tx=0x7f381c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab75b716cccd68c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f380effd640 1 -- 172.21.15.33:0/2756508088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0144f0 con 0x5638acc66800 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f380effd640 1 -- 172.21.15.33:0/2756508088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381c036070 con 0x5638acc66800 2024-04-23T18:30:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.293+0000 7f380effd640 1 -- 172.21.15.33:0/2756508088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c0049a0 con 0x5638acc66800 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- 172.21.15.33:0/2756508088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 msgr2=0x5638acc7ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f381c01ede0 tx=0x7f381c000f40 comp rx=0 tx=0).stop 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- 172.21.15.33:0/2756508088 shutdown_connections 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc66800 0x5638acc7ea60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5bf40 0x5638acc7e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- 172.21.15.33:0/2756508088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc5b620 0x5638acc722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- 172.21.15.33:0/2756508088 >> 172.21.15.33:0/2756508088 conn(0x5638acc57b30 msgr2=0x5638acc56cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- 172.21.15.33:0/2756508088 shutdown_connections 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- 172.21.15.33:0/2756508088 wait complete. 2024-04-23T18:30:16.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- start start 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 Processor -- start 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 0x5638acc6c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc6cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38241b1100 con 0x5638acc5bf40 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.294+0000 7f382dfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38241b0a30 con 0x5638acc5b620 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f382dfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38241adc70 con 0x5638acc66800 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f382dfcf740 10 client.0 _ll_register_callbacks cb 0x7f3814003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33068/0 (socket says 172.21.15.33:33068) 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 -- 172.21.15.33:0/3553648348 learned_addr learned my addr 172.21.15.33:0/3553648348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3829057640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 0x5638acc6c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3829858640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc6cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 -- 172.21.15.33:0/3553648348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 msgr2=0x5638acc6cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc6cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 -- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 msgr2=0x5638acc6c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 0x5638acc6c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 -- 172.21.15.33:0/3553648348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638accdc040 con 0x5638acc5bf40 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3829057640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 0x5638acc6c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f3828856640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f38140b95c0 tx=0x7f38140c5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986c85021770e078 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f380cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3814049040 con 0x5638acc5bf40 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f3814003800 con 0x5638acc5bf40 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f381c036070 con 0x5638acc5bf40 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f38140bc730 con 0x5638acc5bf40 2024-04-23T18:30:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f382dfcf740 10 client.16167 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.295+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5638accdd350 con 0x5638acc5bf40 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f381c036070 con 0x5638acc5bf40 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f38140bc040 con 0x5638acc5bf40 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 10 client.16167 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 20 client.16167 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638accdd8e0 0x5638accdfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 4 client.16167 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638acbc8200 con 0x5638accdd8e0 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f3829057640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638accdd8e0 0x5638accdfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f3829057640 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638accdd8e0 0x5638accdfcd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7510eec58115d5ea server_cookie=570abd9a28223e0b in_seq=0 out_seq=0 2024-04-23T18:30:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.296+0000 7f380cff9640 10 client.16167 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f380cff9640 1 -- 172.21.15.33:0/3553648348 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638acbc8200 con 0x5638accdd8e0 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f380cff9640 10 client.16167 handle_command_reply: tid=0 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 client.16167 shutdown 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 msgr2=0x5638acc6ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f38140b95c0 tx=0x7f38140c5710 comp rx=0 tx=0).stop 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 shutdown_connections 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5638accdd8e0 0x5638accdfcd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638acc66800 0x5638acc6cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638acc5bf40 0x5638acc6ca20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f380cff9640 0 client.16167 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 --2- 172.21.15.33:0/3553648348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5638acc5b620 0x5638acc6c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 >> 172.21.15.33:0/3553648348 conn(0x5638acc57b30 msgr2=0x5638acc56cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 shutdown_connections 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 1 -- 172.21.15.33:0/3553648348 wait complete. 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.305+0000 7f382dfcf740 20 client.16167 trim_cache size 0 max 16384 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:30:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:30:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:30:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 msgr2=0x7f38100a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 0x7f38100a6330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f381c022000 tx=0x7f381c01e000 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 msgr2=0x7f38241013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3814005940 tx=0x7f38140042f0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 shutdown_connections 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38100a3e70 0x7f38100a6330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f382410c580 0x7f3824101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3824103980 0x7f38241013d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 --2- 172.21.15.33:0/2720940226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3824103060 0x7f3824100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 >> 172.21.15.33:0/2720940226 conn(0x7f3824077fd0 msgr2=0x7f38240fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 shutdown_connections 2024-04-23T18:30:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:16.306+0000 7f382b2bb640 1 -- 172.21.15.33:0/2720940226 wait complete. 2024-04-23T18:30:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:16 smithi122 ceph-mon[32546]: daemon mds.f restarted 2024-04-23T18:30:16.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:16 smithi033 ceph-mon[29274]: daemon mds.f restarted 2024-04-23T18:30:16.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:16 smithi106 ceph-mon[32716]: daemon mds.f restarted 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: pgmap v116: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/111505116,v1:172.21.15.122:6835/111505116] up:boot 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:30:17.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:17 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: pgmap v116: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/111505116,v1:172.21.15.122:6835/111505116] up:boot 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:30:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:17 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: pgmap v116: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 40 KiB/s wr, 32 op/s 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: osdmap e89: 12 total, 12 up, 12 in 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/111505116,v1:172.21.15.122:6835/111505116] up:boot 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:30:17.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:17 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:30:19.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:19 smithi122 ceph-mon[32546]: pgmap v118: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:19.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:19 smithi033 ceph-mon[29274]: pgmap v118: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:19.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:19 smithi106 ceph-mon[32716]: pgmap v118: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:21.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:21 smithi122 ceph-mon[32546]: pgmap v119: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:21.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:21 smithi033 ceph-mon[29274]: pgmap v119: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:21.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:21 smithi106 ceph-mon[32716]: pgmap v119: 97 pgs: 97 active+clean; 2.4 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:30:22.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:22 smithi122 ceph-mon[32546]: pgmap v120: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:22.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:22 smithi033 ceph-mon[29274]: pgmap v120: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:22.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:22 smithi106 ceph-mon[32716]: pgmap v120: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:24.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:24 smithi106 ceph-mon[32716]: pgmap v121: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:24.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:24 smithi122 ceph-mon[32546]: pgmap v121: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:24.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:24 smithi033 ceph-mon[29274]: pgmap v121: 97 pgs: 97 active+clean; 2.5 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-23T18:30:26.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:26 smithi106 ceph-mon[32716]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:30:26.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:26 smithi122 ceph-mon[32546]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:30:26.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:26 smithi033 ceph-mon[29274]: pgmap v122: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-23T18:30:28.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:28 smithi106 ceph-mon[32716]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:30:28.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:28 smithi122 ceph-mon[32546]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:30:28.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:28 smithi033 ceph-mon[29274]: pgmap v123: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:30:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- 172.21.15.33:0/2958446352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4102b70 msgr2=0x7f1dc4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 --2- 172.21.15.33:0/2958446352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4102b70 0x7f1dc4106bb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009900 tx=0x7f1db802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- 172.21.15.33:0/2958446352 shutdown_connections 2024-04-23T18:30:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 --2- 172.21.15.33:0/2958446352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc4107290 0x7f1dc410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 --2- 172.21.15.33:0/2958446352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4102b70 0x7f1dc4106bb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 --2- 172.21.15.33:0/2958446352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 0x7f1dc41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- 172.21.15.33:0/2958446352 >> 172.21.15.33:0/2958446352 conn(0x7f1dc40fda20 msgr2=0x7f1dc40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- 172.21.15.33:0/2958446352 shutdown_connections 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- 172.21.15.33:0/2958446352 wait complete. 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 Processor -- start 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.406+0000 7f1dca810640 1 -- start start 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 0x7f1dc41986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc419cf70 0x7f1dc419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41121b0 con 0x7f1dc4107290 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4112030 con 0x7f1dc419cf70 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dca810640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4112330 con 0x7f1dc41021c0 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34032/0 (socket says 172.21.15.33:34032) 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 -- 172.21.15.33:0/3624999950 learned_addr learned my addr 172.21.15.33:0/3624999950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc419cf70 0x7f1dc419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 0x7f1dc41986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 -- 172.21.15.33:0/3624999950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 msgr2=0x7f1dc41986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 0x7f1dc41986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 -- 172.21.15.33:0/3624999950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc419cf70 msgr2=0x7f1dc419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc419cf70 0x7f1dc419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 -- 172.21.15.33:0/3624999950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc419da60 con 0x7f1dc4107290 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.407+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1db8032fd0 tx=0x7f1db8031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.408+0000 7f1dc17fa640 1 -- 172.21.15.33:0/3624999950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db803f030 con 0x7f1dc4107290 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.408+0000 7f1dc17fa640 1 -- 172.21.15.33:0/3624999950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db8004490 con 0x7f1dc4107290 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.408+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc419dcf0 con 0x7f1dc4107290 2024-04-23T18:30:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.408+0000 7f1dc17fa640 1 -- 172.21.15.33:0/3624999950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8035070 con 0x7f1dc4107290 2024-04-23T18:30:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.408+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a54b0 con 0x7f1dc4107290 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.410+0000 7f1dc17fa640 1 -- 172.21.15.33:0/3624999950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1db8042030 con 0x7f1dc4107290 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.410+0000 7f1dcd600740 1 Processor -- start 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 -- start start 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559363054970 con 0x5593630fa800 2024-04-23T18:30:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559362ff14a0 con 0x5593630eff40 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dcd600740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559363050930 con 0x5593630ef620 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc17fa640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 0x7f1dac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 0x7f1dac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34048/0 (socket says 172.21.15.33:34048) 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc37fe640 1 -- 172.21.15.33:0/3512634981 learned_addr learned my addr 172.21.15.33:0/3512634981 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.411+0000 7f1dc17fa640 1 -- 172.21.15.33:0/3624999950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1db80f0b10 con 0x7f1dc4107290 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 -- 172.21.15.33:0/3512634981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 msgr2=0x5593630efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 -- 172.21.15.33:0/3512634981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 msgr2=0x5593630fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 -- 172.21.15.33:0/3512634981 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559362d67210 con 0x5593630eff40 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 0x7f1dac0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005fd0 tx=0x7f1dbc009e80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630fa2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009c20 tx=0x7f1db4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bcecf29fdcc16f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1daa7fc640 1 -- 172.21.15.33:0/3512634981 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db401e070 con 0x5593630eff40 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1daa7fc640 1 -- 172.21.15.33:0/3512634981 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4009dd0 con 0x5593630eff40 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.412+0000 7f1daa7fc640 1 -- 172.21.15.33:0/3512634981 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4019410 con 0x5593630eff40 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- 172.21.15.33:0/3512634981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 msgr2=0x5593630fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630fa2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009c20 tx=0x7f1db4011810 comp rx=0 tx=0).stop 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- 172.21.15.33:0/3512634981 shutdown_connections 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630fa2c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3512634981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- 172.21.15.33:0/3512634981 >> 172.21.15.33:0/3512634981 conn(0x5593630ebb30 msgr2=0x5593630ebf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- 172.21.15.33:0/3512634981 shutdown_connections 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- 172.21.15.33:0/3512634981 wait complete. 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 -- start start 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.413+0000 7f1dcd600740 1 Processor -- start 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593630fd430 con 0x5593630fa800 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559362fef0b0 con 0x5593630eff40 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593630fd5b0 con 0x5593630ef620 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dcd600740 10 client.0 _ll_register_callbacks cb 0x559362fdcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34054/0 (socket says 172.21.15.33:34054) 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 -- 172.21.15.33:0/2719607238 learned_addr learned my addr 172.21.15.33:0/2719607238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 -- 172.21.15.33:0/2719607238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 msgr2=0x5593630fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 -- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 msgr2=0x5593630f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc8dad640 1 -- 172.21.15.33:0/2719607238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559362e33620 con 0x5593630fa800 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.414+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009dc0 tx=0x7f1dbc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95d9a485b0573f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc024070 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1dbc0047c0 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db4009dd0 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1dbc031450 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dcd600740 10 client.16179 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559363109300 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4009dd0 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1dbc01f4e0 con 0x5593630fa800 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dcd600740 10 client.16179 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.415+0000 7f1dcd600740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dcd600740 20 client.16179 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55936310a810 0x55936310cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dcd600740 4 client.16179 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559363062390 con 0x55936310a810 2024-04-23T18:30:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55936310a810 0x55936310cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1da37fe640 10 client.16179 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.416+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55936310a810 0x55936310cc20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a3c30af825ad1d server_cookie=963937d2b41e46a in_seq=0 out_seq=0 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1da37fe640 1 -- 172.21.15.33:0/2719607238 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559363062390 con 0x55936310a810 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1da37fe640 10 client.16179 handle_command_reply: tid=0 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 client.16179 shutdown 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 msgr2=0x5593630f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009dc0 tx=0x7f1dbc01d000 comp rx=0 tx=0).stop 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 shutdown_connections 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55936310a810 0x55936310cc20 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630fa800 0x5593630f7920 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x5593630f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1da37fe640 0 client.16179 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2719607238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 >> 172.21.15.33:0/2719607238 conn(0x5593630ebb30 msgr2=0x5593630f1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 shutdown_connections 2024-04-23T18:30:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 1 -- 172.21.15.33:0/2719607238 wait complete. 2024-04-23T18:30:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.417+0000 7f1dcd600740 20 client.16179 trim_cache size 0 max 16384 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.432+0000 7f1dcd600740 1 Processor -- start 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.432+0000 7f1dcd600740 1 -- start start 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ebb30 0x5593631062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 0x5593630eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41adaf0 con 0x5593630eff40 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41af630 con 0x5593630ef620 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dcd600740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41ae1c0 con 0x5593630ebb30 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34066/0 (socket says 172.21.15.33:34066) 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 -- 172.21.15.33:0/3869172335 learned_addr learned my addr 172.21.15.33:0/3869172335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 0x5593630eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ebb30 0x5593631062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 -- 172.21.15.33:0/3869172335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ebb30 msgr2=0x5593631062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ebb30 0x5593631062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 -- 172.21.15.33:0/3869172335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 msgr2=0x5593630eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 0x5593630eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 -- 172.21.15.33:0/3869172335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593630f09f0 con 0x5593630eff40 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 0x5593630eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.433+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009ba0 tx=0x7f1dbc00f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df73f1edec542e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1da9ffb640 1 -- 172.21.15.33:0/3869172335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc03f070 con 0x5593630eff40 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1da9ffb640 1 -- 172.21.15.33:0/3869172335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc004400 con 0x5593630eff40 2024-04-23T18:30:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1da9ffb640 1 -- 172.21.15.33:0/3869172335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc024070 con 0x5593630eff40 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 -- 172.21.15.33:0/3869172335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 msgr2=0x559363112530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009ba0 tx=0x7f1dbc00f9a0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 -- 172.21.15.33:0/3869172335 shutdown_connections 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630eff40 0x559363112530 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630ef620 0x5593630eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 --2- 172.21.15.33:0/3869172335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ebb30 0x5593631062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 -- 172.21.15.33:0/3869172335 >> 172.21.15.33:0/3869172335 conn(0x5593630f7a80 msgr2=0x5593630ea420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 -- 172.21.15.33:0/3869172335 shutdown_connections 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.434+0000 7f1dcd600740 1 -- 172.21.15.33:0/3869172335 wait complete. 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 -- start start 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 Processor -- start 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x559363100400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41aef60 con 0x5593630ebb30 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41ae890 con 0x5593630eff40 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41abaf0 con 0x5593630ef620 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dcd600740 10 client.0 _ll_register_callbacks cb 0x7f1dbc00ff20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dc3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34072/0 (socket says 172.21.15.33:34072) 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dc3fff640 1 -- 172.21.15.33:0/2706197650 learned_addr learned my addr 172.21.15.33:0/2706197650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.435+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 -- 172.21.15.33:0/2706197650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 msgr2=0x5593630ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x559363100400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 -- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 msgr2=0x559363100400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x559363100400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 -- 172.21.15.33:0/2706197650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559363100c00 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc8dad640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x559363100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dc3fff640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1db401c040 tx=0x7f1db4011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec67dc257bb3288 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1da37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db401e040 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1db4009dd0 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dbc004400 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1db40038b0 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dcd600740 10 client.16185 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559363170d30 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.436+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc004400 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1db4026620 con 0x5593630ebb30 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 20 client.16185 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593630fa800 0x559363173330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 4 client.16185 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936305c200 con 0x5593630fa800 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593630fa800 0x559363173330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1dc37fe640 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593630fa800 0x559363173330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01dc81e2605f483 server_cookie=224eff095757fa17 in_seq=0 out_seq=0 2024-04-23T18:30:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.437+0000 7f1da37fe640 10 client.16185 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.444+0000 7f1da37fe640 1 -- 172.21.15.33:0/2706197650 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936305c200 con 0x5593630fa800 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.444+0000 7f1da37fe640 10 client.16185 handle_command_reply: tid=0 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.444+0000 7f1dcd600740 1 client.16185 shutdown 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 msgr2=0x559363102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1db401c040 tx=0x7f1db4011e50 comp rx=0 tx=0).stop 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 shutdown_connections 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593630fa800 0x559363173330 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593630eff40 0x559363100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593630ef620 0x5593630ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1da37fe640 0 client.16185 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 --2- 172.21.15.33:0/2706197650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593630ebb30 0x559363102b30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 >> 172.21.15.33:0/2706197650 conn(0x5593630f7a80 msgr2=0x5593630ea420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 shutdown_connections 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 1 -- 172.21.15.33:0/2706197650 wait complete. 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dcd600740 20 client.16185 trim_cache size 0 max 16384 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:30:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 msgr2=0x7f1dac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 0x7f1dac0a6280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc005fd0 tx=0x7f1dbc009e80 comp rx=0 tx=0).stop 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 msgr2=0x7f1dc4198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.445+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1db8032fd0 tx=0x7f1db8031a20 comp rx=0 tx=0).stop 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 shutdown_connections 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1dac0a3dc0 0x7f1dac0a6280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dc419cf70 0x7f1dc419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dc4107290 0x7f1dc4198bf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 --2- 172.21.15.33:0/3624999950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1dc41021c0 0x7f1dc41986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 >> 172.21.15.33:0/3624999950 conn(0x7f1dc40fda20 msgr2=0x7f1dc40ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 shutdown_connections 2024-04-23T18:30:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:29.446+0000 7f1dca810640 1 -- 172.21.15.33:0/3624999950 wait complete. 2024-04-23T18:30:30.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:30 smithi106 ceph-mon[32716]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-23T18:30:30.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:30 smithi122 ceph-mon[32546]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-23T18:30:30.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:30 smithi033 ceph-mon[29274]: pgmap v124: 97 pgs: 97 active+clean; 2.6 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-04-23T18:30:32.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:32 smithi106 ceph-mon[32716]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:30:32.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:32 smithi122 ceph-mon[32546]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:30:32.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:32 smithi033 ceph-mon[29274]: pgmap v125: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-23T18:30:34.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:34 smithi106 ceph-mon[32716]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:34.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:34 smithi122 ceph-mon[32546]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:34.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:34 smithi033 ceph-mon[29274]: pgmap v126: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:36.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:36 smithi106 ceph-mon[32716]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:36.957 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:36 smithi122 ceph-mon[32546]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:36 smithi033 ceph-mon[29274]: pgmap v127: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:38.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:38 smithi106 ceph-mon[32716]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:38.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:38.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:38.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:38.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:30:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:38 smithi122 ceph-mon[32546]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:38.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:30:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:38 smithi033 ceph-mon[29274]: pgmap v128: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:30:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:30:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:30:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:30:40.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:40 smithi106 ceph-mon[32716]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:40.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:40 smithi122 ceph-mon[32546]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:40.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:40 smithi033 ceph-mon[29274]: pgmap v129: 97 pgs: 97 active+clean; 2.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 -- 172.21.15.33:0/2098234279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca8102b70 msgr2=0x7f4ca8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 --2- 172.21.15.33:0/2098234279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca8102b70 0x7f4ca8106bb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009900 tx=0x7f4c9802a410 comp rx=0 tx=0).stop 2024-04-23T18:30:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 -- 172.21.15.33:0/2098234279 shutdown_connections 2024-04-23T18:30:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 --2- 172.21.15.33:0/2098234279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 0x7f4ca810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 --2- 172.21.15.33:0/2098234279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca8102b70 0x7f4ca8106bb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 --2- 172.21.15.33:0/2098234279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca81021c0 0x7f4ca81025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.546+0000 7f4caf407640 1 -- 172.21.15.33:0/2098234279 >> 172.21.15.33:0/2098234279 conn(0x7f4ca80fda20 msgr2=0x7f4ca80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- 172.21.15.33:0/2098234279 shutdown_connections 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- 172.21.15.33:0/2098234279 wait complete. 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 Processor -- start 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- start start 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 0x7f4ca8198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca819cfa0 0x7f4ca819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca8112220 con 0x7f4ca81021c0 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81120a0 con 0x7f4ca8107290 2024-04-23T18:30:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.547+0000 7f4caf407640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81123a0 con 0x7f4ca819cfa0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52090/0 (socket says 172.21.15.33:52090) 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 -- 172.21.15.33:0/338534209 learned_addr learned my addr 172.21.15.33:0/338534209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 0x7f4ca8198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca819cfa0 0x7f4ca819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 -- 172.21.15.33:0/338534209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca819cfa0 msgr2=0x7f4ca819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca819cfa0 0x7f4ca819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 -- 172.21.15.33:0/338534209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 msgr2=0x7f4ca8198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 0x7f4ca8198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 -- 172.21.15.33:0/338534209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca819da90 con 0x7f4ca81021c0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400d770 tx=0x7f4ca400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4c9e7fc640 1 -- 172.21.15.33:0/338534209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4017070 con 0x7f4ca81021c0 2024-04-23T18:30:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.548+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca819dd80 con 0x7f4ca81021c0 2024-04-23T18:30:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.549+0000 7f4c9e7fc640 1 -- 172.21.15.33:0/338534209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca40054d0 con 0x7f4ca81021c0 2024-04-23T18:30:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.549+0000 7f4c9e7fc640 1 -- 172.21.15.33:0/338534209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4004c30 con 0x7f4ca81021c0 2024-04-23T18:30:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.549+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca81a5540 con 0x7f4ca81021c0 2024-04-23T18:30:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.551+0000 7f4c9e7fc640 1 -- 172.21.15.33:0/338534209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ca4004e10 con 0x7f4ca81021c0 2024-04-23T18:30:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.551+0000 7f4cb21f0740 1 Processor -- start 2024-04-23T18:30:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.551+0000 7f4cb21f0740 1 -- start start 2024-04-23T18:30:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397aba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563739710970 con 0x5637397abf40 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5637396ad4a0 con 0x5637397ab620 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56373970c930 con 0x5637397b6800 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4c9e7fc640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 0x7f4c940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cad9a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:59058/0 (socket says 172.21.15.33:59058) 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52104/0 (socket says 172.21.15.33:52104) 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 -- 172.21.15.33:0/1520150294 learned_addr learned my addr 172.21.15.33:0/1520150294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 0x7f4c940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4c9e7fc640 1 -- 172.21.15.33:0/338534209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4ca40d0210 con 0x7f4ca81021c0 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 -- 172.21.15.33:0/1520150294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 msgr2=0x5637397aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 -- 172.21.15.33:0/1520150294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 msgr2=0x5637397b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 -- 172.21.15.33:0/1520150294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563739423210 con 0x5637397b6800 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 0x7f4c940a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca0009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.552+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c98006010 tx=0x7f4c98004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62af298a28e54d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4c937fe640 1 -- 172.21.15.33:0/1520150294 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98035040 con 0x5637397b6800 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4c937fe640 1 -- 172.21.15.33:0/1520150294 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c980054f0 con 0x5637397b6800 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4c937fe640 1 -- 172.21.15.33:0/1520150294 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c980035f0 con 0x5637397b6800 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 -- 172.21.15.33:0/1520150294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 msgr2=0x5637397b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4c98006010 tx=0x7f4c98004170 comp rx=0 tx=0).stop 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 -- 172.21.15.33:0/1520150294 shutdown_connections 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x5637397b8bf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/1520150294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.553+0000 7f4cb21f0740 1 -- 172.21.15.33:0/1520150294 >> 172.21.15.33:0/1520150294 conn(0x5637397a7b30 msgr2=0x5637397a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- 172.21.15.33:0/1520150294 shutdown_connections 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- 172.21.15.33:0/1520150294 wait complete. 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- start start 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 Processor -- start 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 0x5637397b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5637397b93d0 con 0x5637397ab620 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5637396ab0b0 con 0x5637397b6800 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5637397b9550 con 0x5637397abf40 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.554+0000 7f4cb21f0740 10 client.0 _ll_register_callbacks cb 0x563739698d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52106/0 (socket says 172.21.15.33:52106) 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 -- 172.21.15.33:0/368366120 learned_addr learned my addr 172.21.15.33:0/368366120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 0x5637397b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 -- 172.21.15.33:0/368366120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 msgr2=0x5637397b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 -- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 msgr2=0x5637397b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 0x5637397b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 -- 172.21.15.33:0/368366120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637394ef620 con 0x5637397ab620 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 0x5637397b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4ca409bf70 tx=0x7f4ca40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1264140a87590a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4c90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40d0070 con 0x5637397ab620 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4ca40988e0 con 0x5637397ab620 2024-04-23T18:30:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c980054f0 con 0x5637397ab620 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ca4020040 con 0x5637397ab620 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cb21f0740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.555+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5637397c5390 con 0x5637397ab620 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c980054f0 con 0x5637397ab620 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4ca40a49a0 con 0x5637397ab620 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 20 client.16197 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5637397c6850 0x5637397c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 4 client.16197 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373971e390 con 0x5637397c6850 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.556+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5637397c6850 0x5637397c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.557+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5637397c6850 0x5637397c8c80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f20b4b9dbf12600 server_cookie=126362062be3bc3f in_seq=0 out_seq=0 2024-04-23T18:30:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.557+0000 7f4c90ff9640 10 client.16197 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.557+0000 7f4c90ff9640 1 -- 172.21.15.33:0/368366120 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56373971e390 con 0x5637397c6850 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.557+0000 7f4c90ff9640 10 client.16197 handle_command_reply: tid=0 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.557+0000 7f4cb21f0740 1 client.16197 shutdown 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 msgr2=0x5637397b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4ca409bf70 tx=0x7f4ca40057c0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 shutdown_connections 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5637397c6850 0x5637397c8c80 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397b6800 0x5637397b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4c90ff9640 0 client.16197 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/368366120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397ab620 0x5637397b61a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 >> 172.21.15.33:0/368366120 conn(0x5637397a7b30 msgr2=0x5637397ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 shutdown_connections 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 1 -- 172.21.15.33:0/368366120 wait complete. 2024-04-23T18:30:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.558+0000 7f4cb21f0740 20 client.16197 trim_cache size 0 max 16384 2024-04-23T18:30:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 Processor -- start 2024-04-23T18:30:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 -- start start 2024-04-23T18:30:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ada60 con 0x5637397b6800 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81af5a0 con 0x5637397ab620 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ae130 con 0x5637397abf40 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cad9a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cad9a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52108/0 (socket says 172.21.15.33:52108) 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cad9a4640 1 -- 172.21.15.33:0/3384160174 learned_addr learned my addr 172.21.15.33:0/3384160174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.573+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 -- 172.21.15.33:0/3384160174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 msgr2=0x5637397ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 -- 172.21.15.33:0/3384160174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 msgr2=0x5637397c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 -- 172.21.15.33:0/3384160174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637397b2440 con 0x5637397b6800 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d60 tx=0x7f4ca0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bece4e84b9d8301 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4c92ffd640 1 -- 172.21.15.33:0/3384160174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0004b20 con 0x5637397b6800 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4c92ffd640 1 -- 172.21.15.33:0/3384160174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca0004cc0 con 0x5637397b6800 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.574+0000 7f4c92ffd640 1 -- 172.21.15.33:0/3384160174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca002a690 con 0x5637397b6800 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- 172.21.15.33:0/3384160174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 msgr2=0x5637397ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009d60 tx=0x7f4ca0016000 comp rx=0 tx=0).stop 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- 172.21.15.33:0/3384160174 shutdown_connections 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397b6800 0x5637397ceaf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397abf40 0x5637397ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/3384160174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- 172.21.15.33:0/3384160174 >> 172.21.15.33:0/3384160174 conn(0x5637397a7b30 msgr2=0x5637397a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- 172.21.15.33:0/3384160174 shutdown_connections 2024-04-23T18:30:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- 172.21.15.33:0/3384160174 wait complete. 2024-04-23T18:30:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- start start 2024-04-23T18:30:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 Processor -- start 2024-04-23T18:30:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397bc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x56373982be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81aeed0 con 0x5637397abf40 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.575+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81ae800 con 0x5637397ab620 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cb21f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca81aba40 con 0x5637397b6800 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cb21f0740 10 client.0 _ll_register_callbacks cb 0x7f4ca409bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52110/0 (socket says 172.21.15.33:52110) 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 -- 172.21.15.33:0/2316740776 learned_addr learned my addr 172.21.15.33:0/2316740776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397bc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cad9a4640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x56373982be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 -- 172.21.15.33:0/2316740776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 msgr2=0x56373982be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x56373982be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 -- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 msgr2=0x5637397bc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397bc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 -- 172.21.15.33:0/2316740776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637397b3da0 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cac9a2640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802aa00 tx=0x7f4c98005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd4620b8dbee3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4c90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98035040 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4c98004260 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca0004cc0 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.576+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c98004460 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 10 client.16203 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56373982d480 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca0004cc0 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c98003860 con 0x5637397abf40 2024-04-23T18:30:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 20 client.16203 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56373982d810 0x56373982fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 4 client.16203 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563739718200 con 0x56373982d810 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.577+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56373982d810 0x56373982fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.578+0000 7f4c90ff9640 10 client.16203 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.578+0000 7f4cad1a3640 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56373982d810 0x56373982fc00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eecb9547345277b server_cookie=eca4e1eb474853cd in_seq=0 out_seq=0 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4c90ff9640 1 -- 172.21.15.33:0/2316740776 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563739718200 con 0x56373982d810 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4c90ff9640 10 client.16203 handle_command_reply: tid=0 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 client.16203 shutdown 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 msgr2=0x5637397bce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c9802aa00 tx=0x7f4c98005a20 comp rx=0 tx=0).stop 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 shutdown_connections 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56373982d810 0x56373982fc00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5637397b6800 0x56373982be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4c90ff9640 0 client.16203 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5637397abf40 0x5637397bce30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 --2- 172.21.15.33:0/2316740776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5637397ab620 0x5637397bc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 >> 172.21.15.33:0/2316740776 conn(0x5637397a7b30 msgr2=0x5637397c1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 shutdown_connections 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 1 -- 172.21.15.33:0/2316740776 wait complete. 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.585+0000 7f4cb21f0740 20 client.16203 trim_cache size 0 max 16384 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:30:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:30:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 msgr2=0x7f4c940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 0x7f4c940a6280 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009fd0 tx=0x7f4ca0009500 comp rx=0 tx=0).stop 2024-04-23T18:30:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 msgr2=0x7f4ca81986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400d770 tx=0x7f4ca400dc30 comp rx=0 tx=0).stop 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 shutdown_connections 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4c940a3dc0 0x7f4c940a6280 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ca819cfa0 0x7f4ca819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ca8107290 0x7f4ca8198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 --2- 172.21.15.33:0/338534209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ca81021c0 0x7f4ca81986e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 >> 172.21.15.33:0/338534209 conn(0x7f4ca80fda20 msgr2=0x7f4ca80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 shutdown_connections 2024-04-23T18:30:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:42.586+0000 7f4caf407640 1 -- 172.21.15.33:0/338534209 wait complete. 2024-04-23T18:30:42.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:42 smithi106 ceph-mon[32716]: pgmap v130: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-23T18:30:42.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:42 smithi122 ceph-mon[32546]: pgmap v130: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-23T18:30:42.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:42 smithi033 ceph-mon[29274]: pgmap v130: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-23T18:30:44.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:44 smithi106 ceph-mon[32716]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T18:30:44.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:44 smithi122 ceph-mon[32546]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T18:30:44.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:44 smithi033 ceph-mon[29274]: pgmap v131: 97 pgs: 97 active+clean; 2.8 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 13 op/s 2024-04-23T18:30:46.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:46 smithi122 ceph-mon[32546]: pgmap v132: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:46.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:46 smithi033 ceph-mon[29274]: pgmap v132: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:47.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:46 smithi106 ceph-mon[32716]: pgmap v132: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:47.488 INFO:tasks.workunit.client.0.smithi033.stdout:64/600: open fd = 3 2024-04-23T18:30:47.488 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:65/600: open fd = 3 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:66/600: open fd = 3 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:67/600: open fd = 3 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.490 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:68/600: open fd = 3 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:69/600: open fd = 3 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.491 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:70/600: open fd = 3 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:71/600: open fd = 3 2024-04-23T18:30:47.492 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:72/600: open fd = 3 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:73/600: open fd = 3 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:74/600: open fd = 3 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.494 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:75/600: open fd = 3 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:76/600: open fd = 3 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:77/600: open fd = 3 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:78/600: open fd = 3 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:79/600: open fd = 3 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.497 INFO:tasks.workunit.client.0.smithi033.stdout:80/600: open fd = 3 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:81/600: open fd = 3 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:82/600: open fd = 3 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:83/600: open fd = 3 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.499 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:84/600: open fd = 3 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:85/600: open fd = 3 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:86/600: open fd = 3 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:87/600: open fd = 3 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:88/600: open fd = 3 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:89/600: open fd = 3 2024-04-23T18:30:47.502 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:90/600: open fd = 3 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.503 INFO:tasks.workunit.client.0.smithi033.stdout:91/600: open fd = 3 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:92/600: open fd = 3 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:93/600: open fd = 3 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:94/600: open fd = 3 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:95/600: open fd = 3 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.506 INFO:tasks.workunit.client.0.smithi033.stdout:96/600: open fd = 3 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:97/600: open fd = 3 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:98/600: open fd = 3 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:99/600: open fd = 3 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.508 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:100/600: open fd = 3 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:101/600: open fd = 3 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.509 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:102/600: open fd = 3 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.510 INFO:tasks.workunit.client.0.smithi033.stdout:103/600: open fd = 3 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:104/600: open fd = 3 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:105/600: open fd = 3 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:106/600: open fd = 3 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.512 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:107/600: open fd = 3 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:108/600: open fd = 3 2024-04-23T18:30:47.513 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:109/600: open fd = 3 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.514 INFO:tasks.workunit.client.0.smithi033.stdout:110/600: open fd = 3 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:111/600: open fd = 3 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.515 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:112/600: open fd = 3 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:113/600: open fd = 3 2024-04-23T18:30:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:114/600: open fd = 3 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.517 INFO:tasks.workunit.client.0.smithi033.stdout:115/600: open fd = 3 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:116/600: open fd = 3 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.518 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:117/600: open fd = 3 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:118/600: open fd = 3 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.519 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:119/600: open fd = 3 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:120/600: open fd = 3 2024-04-23T18:30:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:121/600: open fd = 3 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:122/600: open fd = 3 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:123/600: open fd = 3 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.522 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:124/600: open fd = 3 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:125/600: open fd = 3 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:47.523 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:30:47.524 INFO:tasks.workunit.client.0.smithi033.stdout:126/600: open fd = 3 2024-04-23T18:30:47.524 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:30:48.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:48 smithi122 ceph-mon[32546]: pgmap v133: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:48.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:48 smithi033 ceph-mon[29274]: pgmap v133: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:49.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:48 smithi106 ceph-mon[32716]: pgmap v133: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:50.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:50 smithi122 ceph-mon[32546]: pgmap v134: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:50.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:50 smithi033 ceph-mon[29274]: pgmap v134: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:51.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:50 smithi106 ceph-mon[32716]: pgmap v134: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:30:52.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:52 smithi122 ceph-mon[32546]: pgmap v135: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:52.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:52 smithi033 ceph-mon[29274]: pgmap v135: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:53.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:52 smithi106 ceph-mon[32716]: pgmap v135: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 21 op/s 2024-04-23T18:30:54.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:54 smithi122 ceph-mon[32546]: pgmap v136: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-23T18:30:54.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:54 smithi033 ceph-mon[29274]: pgmap v136: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-23T18:30:55.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:54 smithi106 ceph-mon[32716]: pgmap v136: 97 pgs: 97 active+clean; 2.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 14 op/s 2024-04-23T18:30:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 -- 172.21.15.33:0/1643316108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc1021b0 msgr2=0x7f4dfc102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 --2- 172.21.15.33:0/1643316108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc102590 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009900 tx=0x7f4dec02a410 comp rx=0 tx=0).stop 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 -- 172.21.15.33:0/1643316108 shutdown_connections 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 --2- 172.21.15.33:0/1643316108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc107280 0x7f4dfc10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 --2- 172.21.15.33:0/1643316108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc102b60 0x7f4dfc106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 --2- 172.21.15.33:0/1643316108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc102590 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.686+0000 7f4e02220640 1 -- 172.21.15.33:0/1643316108 >> 172.21.15.33:0/1643316108 conn(0x7f4dfc0fd9f0 msgr2=0x7f4dfc0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.687+0000 7f4e02220640 1 -- 172.21.15.33:0/1643316108 shutdown_connections 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.687+0000 7f4e02220640 1 -- 172.21.15.33:0/1643316108 wait complete. 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.687+0000 7f4e02220640 1 Processor -- start 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.687+0000 7f4e02220640 1 -- start start 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 0x7f4dfc19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1121e0 con 0x7f4dfc107280 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc112060 con 0x7f4dfc19cfb0 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4e02220640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc112360 con 0x7f4dfc1021b0 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41330/0 (socket says 172.21.15.33:41330) 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1594329179 learned_addr learned my addr 172.21.15.33:0/1594329179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 0x7f4dfc19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1594329179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc1021b0 msgr2=0x7f4dfc198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1594329179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 msgr2=0x7f4dfc19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 0x7f4dfc19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1594329179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19daa0 con 0x7f4dfc107280 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 0x7f4dfc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d740 tx=0x7f4df000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.688+0000 7f4df8ff9640 1 -- 172.21.15.33:0/1594329179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0017070 con 0x7f4dfc107280 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.689+0000 7f4df8ff9640 1 -- 172.21.15.33:0/1594329179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4df000de50 con 0x7f4dfc107280 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.689+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19dd90 con 0x7f4dfc107280 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.689+0000 7f4df8ff9640 1 -- 172.21.15.33:0/1594329179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004ce0 con 0x7f4dfc107280 2024-04-23T18:30:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.689+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a5530 con 0x7f4dfc107280 2024-04-23T18:30:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.691+0000 7f4df8ff9640 1 -- 172.21.15.33:0/1594329179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4df0004e80 con 0x7f4dfc107280 2024-04-23T18:30:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.691+0000 7f4e0500b740 1 Processor -- start 2024-04-23T18:30:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.691+0000 7f4e0500b740 1 -- start start 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.691+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b7b620 0x55a775b7ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.691+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4e0500b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a775ae0970 con 0x55a775b86800 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4e0500b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a775a7d4a0 con 0x55a775b7b620 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4e0500b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a775adc930 con 0x55a775b7bf40 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4df8ff9640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 0x7f4de40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41334/0 (socket says 172.21.15.33:41334) 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 0x7f4de40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1380590693 learned_addr learned my addr 172.21.15.33:0/1380590693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b7b620 0x55a775b7ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4df8ff9640 1 -- 172.21.15.33:0/1594329179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4df00d0290 con 0x7f4dfc107280 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1380590693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 msgr2=0x55a775b862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1380590693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b7b620 msgr2=0x55a775b7ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b7b620 0x55a775b7ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 -- 172.21.15.33:0/1380590693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7757f3210 con 0x55a775b86800 2024-04-23T18:30:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 0x7f4de40a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4df4007fd0 tx=0x7f4df4007d00 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.692+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4df00a4980 tx=0x7f4df009bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b047722186a7ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4de1ffb640 1 -- 172.21.15.33:0/1380590693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0020070 con 0x55a775b86800 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4de1ffb640 1 -- 172.21.15.33:0/1380590693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4df009c240 con 0x55a775b86800 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4de1ffb640 1 -- 172.21.15.33:0/1380590693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df00988b0 con 0x55a775b86800 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 -- 172.21.15.33:0/1380590693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 msgr2=0x55a775b88bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4df00a4980 tx=0x7f4df009bc20 comp rx=0 tx=0).stop 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 -- 172.21.15.33:0/1380590693 shutdown_connections 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b86800 0x55a775b88bf0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b862c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 --2- 172.21.15.33:0/1380590693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b7b620 0x55a775b7ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 -- 172.21.15.33:0/1380590693 >> 172.21.15.33:0/1380590693 conn(0x55a775b77b30 msgr2=0x55a775b77f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 -- 172.21.15.33:0/1380590693 shutdown_connections 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.693+0000 7f4e0500b740 1 -- 172.21.15.33:0/1380590693 wait complete. 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 -- start start 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 Processor -- start 2024-04-23T18:30:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b83450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a775b893d0 con 0x55a775b7b620 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a775a7b0b0 con 0x55a775b86800 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a775b89550 con 0x55a775b7bf40 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4e0500b740 10 client.0 _ll_register_callbacks cb 0x55a775a68d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41350/0 (socket says 172.21.15.33:41350) 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.694+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/197720239 learned_addr learned my addr 172.21.15.33:0/197720239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b83450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/197720239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 msgr2=0x55a775b83450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b83450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 msgr2=0x55a775b839b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b839b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/197720239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7758bf620 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4dec02a930 tx=0x7f4dec0317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3713a5ae7976ebbc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec035040 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4dec004240 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4df009c240 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4dec004440 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4e0500b740 10 client.16215 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.695+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a775b95350 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4df009c240 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4dec004aa0 con 0x55a775b7b620 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 10 client.16215 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 20 client.16215 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775b96810 0x55a775b98c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 4 client.16215 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a775aee390 con 0x55a775b96810 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.696+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775b96810 0x55a775b98c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775b96810 0x55a775b98c20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4fd1ac0b5dcf392 server_cookie=c54bd795d7412c94 in_seq=0 out_seq=0 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4ddaffd640 10 client.16215 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4ddaffd640 1 -- 172.21.15.33:0/197720239 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a775aee390 con 0x55a775b96810 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4ddaffd640 10 client.16215 handle_command_reply: tid=0 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4e0500b740 1 client.16215 shutdown 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 msgr2=0x55a775b86160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.697+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4dec02a930 tx=0x7f4dec0317a0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 shutdown_connections 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775b96810 0x55a775b98c20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b839b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b83450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4ddaffd640 0 client.16215 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 --2- 172.21.15.33:0/197720239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b86160 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 >> 172.21.15.33:0/197720239 conn(0x55a775b77b30 msgr2=0x55a775b7d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 shutdown_connections 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 1 -- 172.21.15.33:0/197720239 wait complete. 2024-04-23T18:30:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.698+0000 7f4e0500b740 20 client.16215 trim_cache size 0 max 16384 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 Processor -- start 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 -- start start 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b9e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1adab0 con 0x55a775b7b620 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1af5f0 con 0x55a775b86800 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4e0500b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ae180 con 0x55a775b7bf40 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41366/0 (socket says 172.21.15.33:41366) 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/822192753 learned_addr learned my addr 172.21.15.33:0/822192753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b9e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/822192753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 msgr2=0x55a775b76b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b76b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/822192753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 msgr2=0x55a775b9e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b9e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.713+0000 7f4dfb7fe640 1 -- 172.21.15.33:0/822192753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a775b7bd70 con 0x55a775b7b620 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be86ad45fdbf9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4de17fa640 1 -- 172.21.15.33:0/822192753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec052070 con 0x55a775b7b620 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4de17fa640 1 -- 172.21.15.33:0/822192753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dec004fc0 con 0x55a775b7b620 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4de17fa640 1 -- 172.21.15.33:0/822192753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dec035040 con 0x55a775b7b620 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 -- 172.21.15.33:0/822192753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 msgr2=0x55a775b92290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4dec005ab0 tx=0x7f4dec02a410 comp rx=0 tx=0).stop 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 -- 172.21.15.33:0/822192753 shutdown_connections 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b9e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7bf40 0x55a775b76b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 --2- 172.21.15.33:0/822192753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7b620 0x55a775b92290 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 -- 172.21.15.33:0/822192753 >> 172.21.15.33:0/822192753 conn(0x55a775b77b30 msgr2=0x55a775b76420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 -- 172.21.15.33:0/822192753 shutdown_connections 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.714+0000 7f4e0500b740 1 -- 172.21.15.33:0/822192753 wait complete. 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 -- start start 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 Processor -- start 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7b620 0x55a775b8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b8cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1aef20 con 0x55a775b7bf40 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1ae850 con 0x55a775b86800 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1abab0 con 0x55a775b7b620 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4e0500b740 10 client.0 _ll_register_callbacks cb 0x7f4dec032ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4dfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41370/0 (socket says 172.21.15.33:41370) 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4dfaffd640 1 -- 172.21.15.33:0/459813159 learned_addr learned my addr 172.21.15.33:0/459813159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.715+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7b620 0x55a775b8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b8cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 -- 172.21.15.33:0/459813159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7b620 msgr2=0x55a775b8c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7b620 0x55a775b8c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 -- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 msgr2=0x55a775b8cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:30:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b8cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 -- 172.21.15.33:0/459813159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a775bfbdb0 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfbfff640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b8cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4dfaffd640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4df009bf40 tx=0x7f4df009c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1437237c0f46f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4ddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df0020040 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4df00cdc70 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dec004fc0 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4df009c240 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4e0500b740 10 client.16221 fetch_fsmap learned FSMap version 32 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a775bfd040 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.716+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dec004fc0 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4df00b0ce0 con 0x55a775b7bf40 2024-04-23T18:30:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 10 client.16221 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 20 client.16221 populate_metadata read hostname 'smithi033' 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775bfd380 0x55a775bff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 4 client.16221 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a775ae8200 con 0x55a775bfd380 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775bfd380 0x55a775bff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4dfb7fe640 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775bfd380 0x55a775bff770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7841be23674ee45c server_cookie=d6ab0e7f95e0dc0a in_seq=0 out_seq=0 2024-04-23T18:30:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.717+0000 7f4ddaffd640 10 client.16221 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4ddaffd640 1 -- 172.21.15.33:0/459813159 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a775ae8200 con 0x55a775bfd380 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4ddaffd640 10 client.16221 handle_command_reply: tid=0 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 client.16221 shutdown 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 msgr2=0x55a775b8c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4df009bf40 tx=0x7f4df009c010 comp rx=0 tx=0).stop 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 shutdown_connections 2024-04-23T18:30:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a775bfd380 0x55a775bff770 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a775b86800 0x55a775b8cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a775b7bf40 0x55a775b8c6b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 --2- 172.21.15.33:0/459813159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a775b7b620 0x55a775b8c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4ddaffd640 0 client.16221 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 >> 172.21.15.33:0/459813159 conn(0x55a775b77b30 msgr2=0x55a775b76cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 shutdown_connections 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 1 -- 172.21.15.33:0/459813159 wait complete. 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.725+0000 7f4e0500b740 20 client.16221 trim_cache size 0 max 16384 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:30:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:30:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:30:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:30:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 msgr2=0x7f4de40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 0x7f4de40a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4df4007fd0 tx=0x7f4df4007d00 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 msgr2=0x7f4dfc198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4df000d740 tx=0x7f4df000dc00 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 shutdown_connections 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4de40a3dc0 0x7f4de40a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4dfc19cfb0 0x7f4dfc19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dfc107280 0x7f4dfc198c80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 --2- 172.21.15.33:0/1594329179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4dfc1021b0 0x7f4dfc198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 >> 172.21.15.33:0/1594329179 conn(0x7f4dfc0fd9f0 msgr2=0x7f4dfc0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 shutdown_connections 2024-04-23T18:30:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:30:55.726+0000 7f4e02220640 1 -- 172.21.15.33:0/1594329179 wait complete. 2024-04-23T18:30:56.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:56 smithi122 ceph-mon[32546]: pgmap v137: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-23T18:30:56.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:56 smithi033 ceph-mon[29274]: pgmap v137: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-23T18:30:57.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:56 smithi106 ceph-mon[32716]: pgmap v137: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 20 op/s 2024-04-23T18:30:58.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:30:58 smithi122 ceph-mon[32546]: pgmap v138: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:30:58.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:30:58 smithi033 ceph-mon[29274]: pgmap v138: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:30:59.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:30:58 smithi106 ceph-mon[32716]: pgmap v138: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:31:00.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:00 smithi122 ceph-mon[32546]: pgmap v139: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:31:00.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:00 smithi033 ceph-mon[29274]: pgmap v139: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:31:01.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:00 smithi106 ceph-mon[32716]: pgmap v139: 97 pgs: 97 active+clean; 3.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 12 op/s 2024-04-23T18:31:02.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:02 smithi122 ceph-mon[32546]: pgmap v140: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-23T18:31:02.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:02 smithi033 ceph-mon[29274]: pgmap v140: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-23T18:31:03.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:02 smithi106 ceph-mon[32716]: pgmap v140: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 KiB/s wr, 20 op/s 2024-04-23T18:31:04.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:04 smithi122 ceph-mon[32546]: pgmap v141: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:31:04.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:04 smithi033 ceph-mon[29274]: pgmap v141: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:31:05.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:04 smithi106 ceph-mon[32716]: pgmap v141: 97 pgs: 97 active+clean; 3.1 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-04-23T18:31:06.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:06 smithi122 ceph-mon[32546]: pgmap v142: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 36 op/s 2024-04-23T18:31:06.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:06 smithi033 ceph-mon[29274]: pgmap v142: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 36 op/s 2024-04-23T18:31:07.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:06 smithi106 ceph-mon[32716]: pgmap v142: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 44 KiB/s wr, 36 op/s 2024-04-23T18:31:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/362401492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 msgr2=0x7fe1f010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/362401492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f010ee60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009ca0 tx=0x7fe1e802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/362401492 shutdown_connections 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/362401492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f010ee60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/362401492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f0102b70 0x7fe1f0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/362401492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f01021c0 0x7fe1f01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/362401492 >> 172.21.15.33:0/362401492 conn(0x7fe1f00fda20 msgr2=0x7fe1f00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/362401492 shutdown_connections 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/362401492 wait complete. 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 Processor -- start 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.826+0000 7fe1f7bf9640 1 -- start start 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f01021c0 0x7fe1f0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 0x7fe1f019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0112200 con 0x7fe1f0107290 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0112080 con 0x7fe1f019cfb0 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f0112380 con 0x7fe1f01021c0 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48858/0 (socket says 172.21.15.33:48858) 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 -- 172.21.15.33:0/2001418700 learned_addr learned my addr 172.21.15.33:0/2001418700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5995640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f01021c0 0x7fe1f0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 0x7fe1f019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 -- 172.21.15.33:0/2001418700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f01021c0 msgr2=0x7fe1f0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f01021c0 0x7fe1f0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 -- 172.21.15.33:0/2001418700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 msgr2=0x7fe1f019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 0x7fe1f019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 -- 172.21.15.33:0/2001418700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f019daa0 con 0x7fe1f0107290 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 0x7fe1f019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000d740 tx=0x7fe1e000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1e6ffd640 1 -- 172.21.15.33:0/2001418700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0017070 con 0x7fe1f0107290 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f019dd90 con 0x7fe1f0107290 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1e6ffd640 1 -- 172.21.15.33:0/2001418700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e000de50 con 0x7fe1f0107290 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.827+0000 7fe1e6ffd640 1 -- 172.21.15.33:0/2001418700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0004ce0 con 0x7fe1f0107290 2024-04-23T18:31:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.828+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a5580 con 0x7fe1f0107290 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.830+0000 7fe1e6ffd640 1 -- 172.21.15.33:0/2001418700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1e0004e80 con 0x7fe1f0107290 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.830+0000 7fe1fa9df740 1 Processor -- start 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 -- start start 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1e6ffd640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977db900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa97740970 con 0x55fa977eac10 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fa976dd4a0 con 0x55fa977dbe40 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa9773c930 con 0x55fa977db520 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1e6ffd640 1 -- 172.21.15.33:0/2001418700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe1e00d0110 con 0x7fe1f0107290 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f5995640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977db900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f6196640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f6196640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48872/0 (socket says 172.21.15.33:48872) 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f6196640 1 -- 172.21.15.33:0/203805240 learned_addr learned my addr 172.21.15.33:0/203805240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.831+0000 7fe1f5194640 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 -- 172.21.15.33:0/203805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 msgr2=0x55fa977ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 -- 172.21.15.33:0/203805240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 msgr2=0x55fa977ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 -- 172.21.15.33:0/203805240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa97453210 con 0x55fa977db520 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f6196640 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1e802f500 tx=0x7fe1e802f490 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1f5995640 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977db900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005fd0 tx=0x7fe1ec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9cdea93f4924626 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1dbfff640 1 -- 172.21.15.33:0/203805240 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec01c480 con 0x55fa977db520 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1dbfff640 1 -- 172.21.15.33:0/203805240 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ec01cad0 con 0x55fa977db520 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.832+0000 7fe1dbfff640 1 -- 172.21.15.33:0/203805240 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec0119f0 con 0x55fa977db520 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- 172.21.15.33:0/203805240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 msgr2=0x55fa977db900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977db900 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec005fd0 tx=0x7fe1ec0107a0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- 172.21.15.33:0/203805240 shutdown_connections 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa977ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/203805240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977db900 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- 172.21.15.33:0/203805240 >> 172.21.15.33:0/203805240 conn(0x55fa977d7b30 msgr2=0x55fa977d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- 172.21.15.33:0/203805240 shutdown_connections 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- 172.21.15.33:0/203805240 wait complete. 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.833+0000 7fe1fa9df740 1 -- start start 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 Processor -- start 2024-04-23T18:31:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa977ed840 con 0x55fa977dbe40 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fa976db0b0 con 0x55fa977eac10 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa977ed9c0 con 0x55fa977db520 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1fa9df740 10 client.0 _ll_register_callbacks cb 0x55fa976c8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48888/0 (socket says 172.21.15.33:48888) 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 -- 172.21.15.33:0/456054426 learned_addr learned my addr 172.21.15.33:0/456054426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5995640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f6196640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 -- 172.21.15.33:0/456054426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 msgr2=0x55fa977ea4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977ea4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 -- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 msgr2=0x55fa977e7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f5194640 1 -- 172.21.15.33:0/456054426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9751f620 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.834+0000 7fe1f6196640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1f5194640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cd920 tx=0x7fe1e009c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71046deeef8ad042 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0020040 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe1e00a0120 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1ec01cad0 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe1e00a4d30 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1fa9df740 10 client.16233 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa977f9790 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1ec01cad0 con 0x55fa977dbe40 2024-04-23T18:31:08.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe1e0098730 con 0x55fa977dbe40 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1fa9df740 10 client.16233 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.835+0000 7fe1fa9df740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1fa9df740 20 client.16233 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa977fad40 0x55fa977fd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1fa9df740 4 client.16233 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9774e390 con 0x55fa977fad40 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1f5995640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa977fad40 0x55fa977fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1f5995640 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa977fad40 0x55fa977fd130 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4cc4c0f8dfd55bf server_cookie=97ac29a0f113d2a9 in_seq=0 out_seq=0 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.836+0000 7fe1d97fa640 10 client.16233 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1d97fa640 1 -- 172.21.15.33:0/456054426 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa9774e390 con 0x55fa977fad40 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1d97fa640 10 client.16233 handle_command_reply: tid=0 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 client.16233 shutdown 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 msgr2=0x55fa977e77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00cd920 tx=0x7fe1e009c880 comp rx=0 tx=0).stop 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 shutdown_connections 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa977fad40 0x55fa977fd130 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977dbe40 0x55fa977e77b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/456054426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977ea4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1d97fa640 0 client.16233 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 >> 172.21.15.33:0/456054426 conn(0x55fa977d7b30 msgr2=0x55fa977dd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 shutdown_connections 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 1 -- 172.21.15.33:0/456054426 wait complete. 2024-04-23T18:31:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.837+0000 7fe1fa9df740 20 client.16233 trim_cache size 0 max 16384 2024-04-23T18:31:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.852+0000 7fe1fa9df740 1 Processor -- start 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 -- start start 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ab920 con 0x55fa977eac10 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ad460 con 0x55fa977dbe40 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01abff0 con 0x55fa977db520 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48896/0 (socket says 172.21.15.33:48896) 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 -- 172.21.15.33:0/1445968932 learned_addr learned my addr 172.21.15.33:0/1445968932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f5995640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f5194640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 -- 172.21.15.33:0/1445968932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 msgr2=0x55fa977f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 -- 172.21.15.33:0/1445968932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 msgr2=0x55fa977e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 -- 172.21.15.33:0/1445968932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa977ed720 con 0x55fa977eac10 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f5995640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.853+0000 7fe1f6196640 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009430 tx=0x7fe1e802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be696c0afcc5acaf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1db7fe640 1 -- 172.21.15.33:0/1445968932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8050070 con 0x55fa977eac10 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1db7fe640 1 -- 172.21.15.33:0/1445968932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8036040 con 0x55fa977eac10 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1db7fe640 1 -- 172.21.15.33:0/1445968932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004510 con 0x55fa977eac10 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 -- 172.21.15.33:0/1445968932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 msgr2=0x55fa97802980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe1e8009430 tx=0x7fe1e802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 -- 172.21.15.33:0/1445968932 shutdown_connections 2024-04-23T18:31:08.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977eac10 0x55fa97802980 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977dbe40 0x55fa977e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/1445968932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977db520 0x55fa977f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 -- 172.21.15.33:0/1445968932 >> 172.21.15.33:0/1445968932 conn(0x55fa977d7b30 msgr2=0x55fa977e7770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 -- 172.21.15.33:0/1445968932 shutdown_connections 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.854+0000 7fe1fa9df740 1 -- 172.21.15.33:0/1445968932 wait complete. 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 -- start start 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 Processor -- start 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977dbe40 0x55fa977e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01acd90 con 0x55fa977db520 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01ac6c0 con 0x55fa977eac10 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f01a9900 con 0x55fa977dbe40 2024-04-23T18:31:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1fa9df740 10 client.0 _ll_register_callbacks cb 0x7fe1e009ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f5995640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f5995640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48910/0 (socket says 172.21.15.33:48910) 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f5995640 1 -- 172.21.15.33:0/2559402954 learned_addr learned my addr 172.21.15.33:0/2559402954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977dbe40 0x55fa977e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.855+0000 7fe1f5995640 1 -- 172.21.15.33:0/2559402954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977dbe40 msgr2=0x55fa977e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f5995640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977dbe40 0x55fa977e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f5995640 1 -- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 msgr2=0x55fa977e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f5995640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f5995640 1 -- 172.21.15.33:0/2559402954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa977e4ca0 con 0x55fa977db520 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f6196640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1f5995640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec010ca0 tx=0x7fe1ec005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d77faf18a19d6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:08.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02e060 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fe1ec005450 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1e8036040 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe1ec005650 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 10 client.16239 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fa978610f0 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8036040 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe1ec004910 con 0x55fa977db520 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 10 client.16239 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 10 client.16239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 20 client.16239 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.856+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa978615c0 0x55fa978639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.857+0000 7fe1fa9df740 4 client.16239 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.857+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa977dcbb0 con 0x55fa978615c0 2024-04-23T18:31:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.857+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa978615c0 0x55fa978639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.857+0000 7fe1f5194640 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa978615c0 0x55fa978639e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=910347b698a77e8b server_cookie=d27c9c7c9c1c59e2 in_seq=0 out_seq=0 2024-04-23T18:31:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.857+0000 7fe1d97fa640 10 client.16239 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1d97fa640 1 -- 172.21.15.33:0/2559402954 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa977dcbb0 con 0x55fa978615c0 2024-04-23T18:31:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1d97fa640 10 client.16239 handle_command_reply: tid=0 2024-04-23T18:31:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1fa9df740 1 client.16239 shutdown 2024-04-23T18:31:08.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 msgr2=0x55fa977f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec010ca0 tx=0x7fe1ec005d00 comp rx=0 tx=0).stop 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.865+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 shutdown_connections 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fa978615c0 0x55fa978639e0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fa977eac10 0x55fa977e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa977dbe40 0x55fa977e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1d97fa640 0 client.16239 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 --2- 172.21.15.33:0/2559402954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa977db520 0x55fa977f2f00 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 >> 172.21.15.33:0/2559402954 conn(0x55fa977d7b30 msgr2=0x55fa977e2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 shutdown_connections 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 1 -- 172.21.15.33:0/2559402954 wait complete. 2024-04-23T18:31:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1fa9df740 20 client.16239 trim_cache size 0 max 16384 2024-04-23T18:31:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:31:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:31:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:31:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:31:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 msgr2=0x7fe1dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe1e802f500 tx=0x7fe1e802f490 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 msgr2=0x7fe1f0198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000d740 tx=0x7fe1e000dc00 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 shutdown_connections 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1dc0a3dc0 0x7fe1dc0a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1f019cfb0 0x7fe1f019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1f0107290 0x7fe1f0198c80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 --2- 172.21.15.33:0/2001418700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1f01021c0 0x7fe1f0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 >> 172.21.15.33:0/2001418700 conn(0x7fe1f00fda20 msgr2=0x7fe1f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 shutdown_connections 2024-04-23T18:31:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:08.866+0000 7fe1f7bf9640 1 -- 172.21.15.33:0/2001418700 wait complete. 2024-04-23T18:31:08.922 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:08 smithi106 ceph-mon[32716]: pgmap v143: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:08.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:08 smithi122 ceph-mon[32546]: pgmap v143: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:08.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:08 smithi033 ceph-mon[29274]: pgmap v143: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:10.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:10 smithi122 ceph-mon[32546]: pgmap v144: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:10.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:10 smithi033 ceph-mon[29274]: pgmap v144: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:11.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:10 smithi106 ceph-mon[32716]: pgmap v144: 97 pgs: 97 active+clean; 3.1 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 39 KiB/s wr, 29 op/s 2024-04-23T18:31:12.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:12 smithi122 ceph-mon[32546]: pgmap v145: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 38 op/s 2024-04-23T18:31:12.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:12 smithi033 ceph-mon[29274]: pgmap v145: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 38 op/s 2024-04-23T18:31:13.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:12 smithi106 ceph-mon[32716]: pgmap v145: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 38 op/s 2024-04-23T18:31:14.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:14 smithi122 ceph-mon[32546]: pgmap v146: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:31:14.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:14 smithi033 ceph-mon[29274]: pgmap v146: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:31:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:14 smithi106 ceph-mon[32716]: pgmap v146: 97 pgs: 97 active+clean; 3.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 30 op/s 2024-04-23T18:31:16.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:16 smithi122 ceph-mon[32546]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T18:31:16.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:16 smithi033 ceph-mon[29274]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T18:31:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:16 smithi106 ceph-mon[32716]: pgmap v147: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 36 op/s 2024-04-23T18:31:18.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:18 smithi122 ceph-mon[32546]: pgmap v148: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:18.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:18 smithi033 ceph-mon[29274]: pgmap v148: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:18 smithi106 ceph-mon[32716]: pgmap v148: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:20.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:20 smithi122 ceph-mon[32546]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:20.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:20 smithi033 ceph-mon[29274]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:21.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:20 smithi106 ceph-mon[32716]: pgmap v149: 97 pgs: 97 active+clean; 3.3 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-04-23T18:31:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- 172.21.15.33:0/1479687968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 msgr2=0x7f5648110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 --2- 172.21.15.33:0/1479687968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f5648110d20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5640009ca0 tx=0x7f564002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- 172.21.15.33:0/1479687968 shutdown_connections 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 --2- 172.21.15.33:0/1479687968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f5648110d20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 --2- 172.21.15.33:0/1479687968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56481016a0 0x7f564810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 --2- 172.21.15.33:0/1479687968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5648100d80 0x7f5648101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- 172.21.15.33:0/1479687968 >> 172.21.15.33:0/1479687968 conn(0x7f56480771d0 msgr2=0x7f56480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- 172.21.15.33:0/1479687968 shutdown_connections 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- 172.21.15.33:0/1479687968 wait complete. 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 Processor -- start 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.966+0000 7f564e610640 1 -- start start 2024-04-23T18:31:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5648100d80 0x7f56481a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56481016a0 0x7f56481a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5648114410 con 0x7f564810e930 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5648114290 con 0x7f56481016a0 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564e610640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5648114590 con 0x7f5648100d80 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38340/0 (socket says 172.21.15.33:38340) 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 -- 172.21.15.33:0/638969533 learned_addr learned my addr 172.21.15.33:0/638969533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f56477fe640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56481016a0 0x7f56481a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f5647fff640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5648100d80 0x7f56481a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 -- 172.21.15.33:0/638969533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5648100d80 msgr2=0x7f56481a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5648100d80 0x7f56481a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 -- 172.21.15.33:0/638969533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56481016a0 msgr2=0x7f56481a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56481016a0 0x7f56481a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.967+0000 7f564cbad640 1 -- 172.21.15.33:0/638969533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564819b010 con 0x7f564810e930 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f564cbad640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f564002ade0 tx=0x7f56400319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f56457fa640 1 -- 172.21.15.33:0/638969533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5640003bf0 con 0x7f564810e930 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f56457fa640 1 -- 172.21.15.33:0/638969533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5640003d90 con 0x7f564810e930 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564819b2a0 con 0x7f564810e930 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f56457fa640 1 -- 172.21.15.33:0/638969533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564002fbd0 con 0x7f564810e930 2024-04-23T18:31:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.968+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56481a7610 con 0x7f564810e930 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f56457fa640 1 -- 172.21.15.33:0/638969533 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f564002fd70 con 0x7f564810e930 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 Processor -- start 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 -- start start 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00711a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e0071c800 0x555e0071ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e00676970 con 0x555e00711f40 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e006134a0 con 0x555e00711620 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.971+0000 7f5651404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555e00672930 con 0x555e0071c800 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f56457fa640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 0x7f56340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38350/0 (socket says 172.21.15.33:38350) 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 -- 172.21.15.33:0/3826492014 learned_addr learned my addr 172.21.15.33:0/3826492014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f56477fe640 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e0071c800 0x555e0071ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f56477fe640 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00711a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f564cbad640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 0x7f56340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 -- 172.21.15.33:0/3826492014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e0071c800 msgr2=0x555e0071ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e0071c800 0x555e0071ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 -- 172.21.15.33:0/3826492014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 msgr2=0x555e00711a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00711a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 -- 172.21.15.33:0/3826492014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e00389210 con 0x555e00711f40 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f56457fa640 1 -- 172.21.15.33:0/638969533 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f56400f1c00 con 0x7f564810e930 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f5647fff640 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f563800bcb0 tx=0x7f563800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3fdea66f68f09a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f564cbad640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 0x7f56340a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56400bdc50 tx=0x7f5640003f40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f563e7fc640 1 -- 172.21.15.33:0/3826492014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638015070 con 0x555e00711f40 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f563e7fc640 1 -- 172.21.15.33:0/3826492014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563801a910 con 0x555e00711f40 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.972+0000 7f563e7fc640 1 -- 172.21.15.33:0/3826492014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380118c0 con 0x555e00711f40 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- 172.21.15.33:0/3826492014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 msgr2=0x555e0071c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f563800bcb0 tx=0x7f563800f730 comp rx=0 tx=0).stop 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- 172.21.15.33:0/3826492014 shutdown_connections 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e0071c800 0x555e0071ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e00711f40 0x555e0071c2c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 --2- 172.21.15.33:0/3826492014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00711a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- 172.21.15.33:0/3826492014 >> 172.21.15.33:0/3826492014 conn(0x555e0070db30 msgr2=0x555e0070df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- 172.21.15.33:0/3826492014 shutdown_connections 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- 172.21.15.33:0/3826492014 wait complete. 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.973+0000 7f5651404740 1 -- start start 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 Processor -- start 2024-04-23T18:31:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e0071c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00719460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555e0071f430 con 0x555e0071c800 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555e006110b0 con 0x555e00711620 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555e0071f5b0 con 0x555e00711f40 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5651404740 10 client.0 _ll_register_callbacks cb 0x555e005fed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38356/0 (socket says 172.21.15.33:38356) 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f56477fe640 1 -- 172.21.15.33:0/1677424748 learned_addr learned my addr 172.21.15.33:0/1677424748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f5647fff640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00719460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.974+0000 7f56477fe640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e0071c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 -- 172.21.15.33:0/1677424748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 msgr2=0x555e00719460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00719460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 -- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 msgr2=0x555e0071c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e0071c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 -- 172.21.15.33:0/1677424748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e00455620 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56477fe640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5630012db0 tx=0x7f5630004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a116752d0e76153 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630004050 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f56300041f0 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563801a910 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f56300043f0 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f5651404740 10 client.16251 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555e0072b400 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563801a910 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.975+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5630021440 con 0x555e0071c800 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 10 client.16251 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 20 client.16251 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e0072c8c0 0x555e0072ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 4 client.16251 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e00684390 con 0x555e0072c8c0 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5647fff640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e0072c8c0 0x555e0072ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f5647fff640 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e0072c8c0 0x555e0072ecb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e970f63e48c6900 server_cookie=f465ab84ac2c2ea0 in_seq=0 out_seq=0 2024-04-23T18:31:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.976+0000 7f56277fe640 10 client.16251 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f56277fe640 1 -- 172.21.15.33:0/1677424748 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e00684390 con 0x555e0072c8c0 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f56277fe640 10 client.16251 handle_command_reply: tid=0 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 client.16251 shutdown 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 msgr2=0x555e007199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5630012db0 tx=0x7f5630004730 comp rx=0 tx=0).stop 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 shutdown_connections 2024-04-23T18:31:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e0072c8c0 0x555e0072ecb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e007199c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00719460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f56277fe640 0 client.16251 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 --2- 172.21.15.33:0/1677424748 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e0071c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 >> 172.21.15.33:0/1677424748 conn(0x555e0070db30 msgr2=0x555e007138b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 shutdown_connections 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.977+0000 7f5651404740 1 -- 172.21.15.33:0/1677424748 wait complete. 2024-04-23T18:31:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.978+0000 7f5651404740 20 client.16251 trim_cache size 0 max 16384 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.992+0000 7f5651404740 1 Processor -- start 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 -- start start 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e007282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e0070cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56481afdc0 con 0x555e0071c800 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b1900 con 0x555e00711620 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5651404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b0490 con 0x555e00711f40 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38366/0 (socket says 172.21.15.33:38366) 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 -- 172.21.15.33:0/1271284364 learned_addr learned my addr 172.21.15.33:0/1271284364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e007282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f5647fff640 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e0070cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 -- 172.21.15.33:0/1271284364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 msgr2=0x555e0070cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e0070cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 -- 172.21.15.33:0/1271284364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 msgr2=0x555e007282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e007282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 -- 172.21.15.33:0/1271284364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e00719d30 con 0x555e0071c800 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.993+0000 7f56477fe640 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x555e0072b3a0 tx=0x7f5630014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affb24ac625fa111 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f563dffb640 1 -- 172.21.15.33:0/1271284364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563002e070 con 0x555e0071c800 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f563dffb640 1 -- 172.21.15.33:0/1271284364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563002f040 con 0x555e0071c800 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f563dffb640 1 -- 172.21.15.33:0/1271284364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300120c0 con 0x555e0071c800 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 -- 172.21.15.33:0/1271284364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 msgr2=0x555e00734570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x555e0072b3a0 tx=0x7f5630014040 comp rx=0 tx=0).stop 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 -- 172.21.15.33:0/1271284364 shutdown_connections 2024-04-23T18:31:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00734570 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e0070cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 --2- 172.21.15.33:0/1271284364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e007282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 -- 172.21.15.33:0/1271284364 >> 172.21.15.33:0/1271284364 conn(0x555e0070db30 msgr2=0x555e0070c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 -- 172.21.15.33:0/1271284364 shutdown_connections 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.994+0000 7f5651404740 1 -- 172.21.15.33:0/1271284364 wait complete. 2024-04-23T18:31:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 -- start start 2024-04-23T18:31:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 Processor -- start 2024-04-23T18:31:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00724810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00724d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b1230 con 0x555e0071c800 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56481b0b60 con 0x555e00711620 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56481adda0 con 0x555e00711f40 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5651404740 10 client.0 _ll_register_callbacks cb 0x7f56300128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f56477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38380/0 (socket says 172.21.15.33:38380) 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f56477fe640 1 -- 172.21.15.33:0/1896772213 learned_addr learned my addr 172.21.15.33:0/1896772213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f5647fff640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00724d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.995+0000 7f56477fe640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00724810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 -- 172.21.15.33:0/1896772213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 msgr2=0x555e00724d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00724d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 -- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 msgr2=0x555e00724810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00724810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 -- 172.21.15.33:0/1896772213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e007225a0 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56477fe640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f563000b8c0 tx=0x7f563000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1e5374317c3dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563002e070 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f5630021a80 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f563002f040 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5630021c80 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f5651404740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555e00792a50 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f563002f040 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.996+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f563002a420 con 0x555e0071c800 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 20 client.16257 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e007930b0 0x555e007954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 4 client.16257 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e0067e200 con 0x555e007930b0 2024-04-23T18:31:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5647fff640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e007930b0 0x555e007954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f5647fff640 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e007930b0 0x555e007954d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750c64ac2d1b1fac server_cookie=11c014046bc46aa9 in_seq=0 out_seq=0 2024-04-23T18:31:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:21.997+0000 7f56277fe640 10 client.16257 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f56277fe640 1 -- 172.21.15.33:0/1896772213 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e0067e200 con 0x555e007930b0 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f56277fe640 10 client.16257 handle_command_reply: tid=0 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 client.16257 shutdown 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 msgr2=0x555e00721da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f563000b8c0 tx=0x7f563000b830 comp rx=0 tx=0).stop 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 shutdown_connections 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555e007930b0 0x555e007954d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555e0071c800 0x555e00721da0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f56277fe640 0 client.16257 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:21.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555e00711f40 0x555e00724d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 --2- 172.21.15.33:0/1896772213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555e00711620 0x555e00724810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 >> 172.21.15.33:0/1896772213 conn(0x555e0070db30 msgr2=0x555e0070c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 shutdown_connections 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 1 -- 172.21.15.33:0/1896772213 wait complete. 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f5651404740 20 client.16257 trim_cache size 0 max 16384 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:31:21.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:31:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.005+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 msgr2=0x7f56340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 0x7f56340a6300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56400bdc50 tx=0x7f5640003f40 comp rx=0 tx=0).stop 2024-04-23T18:31:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 msgr2=0x7f564819a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:22.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f564002ade0 tx=0x7f56400319f0 comp rx=0 tx=0).stop 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 shutdown_connections 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56340a3e40 0x7f56340a6300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f564810e930 0x7f564819a750 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56481016a0 0x7f56481a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 --2- 172.21.15.33:0/638969533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5648100d80 0x7f56481a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 >> 172.21.15.33:0/638969533 conn(0x7f56480771d0 msgr2=0x7f564810fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 shutdown_connections 2024-04-23T18:31:22.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:22.006+0000 7f564e610640 1 -- 172.21.15.33:0/638969533 wait complete. 2024-04-23T18:31:22.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:22 smithi122 ceph-mon[32546]: pgmap v150: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:22.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:22 smithi033 ceph-mon[29274]: pgmap v150: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:22 smithi106 ceph-mon[32716]: pgmap v150: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:24.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:24 smithi122 ceph-mon[32546]: pgmap v151: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:31:24.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:24 smithi033 ceph-mon[29274]: pgmap v151: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:31:25.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:24 smithi106 ceph-mon[32716]: pgmap v151: 97 pgs: 97 active+clean; 3.4 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-23T18:31:26.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:26 smithi122 ceph-mon[32546]: pgmap v152: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:26.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:26 smithi033 ceph-mon[29274]: pgmap v152: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:27.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:26 smithi106 ceph-mon[32716]: pgmap v152: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:28.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:28 smithi122 ceph-mon[32546]: pgmap v153: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:28.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:28 smithi033 ceph-mon[29274]: pgmap v153: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:29.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:28 smithi106 ceph-mon[32716]: pgmap v153: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:30.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:30 smithi122 ceph-mon[32546]: pgmap v154: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:30.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:30 smithi033 ceph-mon[29274]: pgmap v154: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:31.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:30 smithi106 ceph-mon[32716]: pgmap v154: 97 pgs: 97 active+clean; 3.5 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:32.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:32 smithi033 ceph-mon[29274]: pgmap v155: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:33.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:32 smithi106 ceph-mon[32716]: pgmap v155: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:33.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:32 smithi122 ceph-mon[32546]: pgmap v155: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:34.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:34 smithi033 ceph-mon[29274]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 -- 172.21.15.33:0/3299112705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 msgr2=0x7f79a410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 --2- 172.21.15.33:0/3299112705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a410ee60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f799c009ca0 tx=0x7f799c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 -- 172.21.15.33:0/3299112705 shutdown_connections 2024-04-23T18:31:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 --2- 172.21.15.33:0/3299112705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a410ee60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 --2- 172.21.15.33:0/3299112705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a4102b70 0x7f79a4106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 --2- 172.21.15.33:0/3299112705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 -- 172.21.15.33:0/3299112705 >> 172.21.15.33:0/3299112705 conn(0x7f79a40fda20 msgr2=0x7f79a40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 -- 172.21.15.33:0/3299112705 shutdown_connections 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.105+0000 7f79aa431640 1 -- 172.21.15.33:0/3299112705 wait complete. 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 Processor -- start 2024-04-23T18:31:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 -- start start 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a419cee0 0x7f79a419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41122a0 con 0x7f79a4107290 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4112120 con 0x7f79a41021c0 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79aa431640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4112420 con 0x7f79a419cee0 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.106+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34740/0 (socket says 172.21.15.33:34740) 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 -- 172.21.15.33:0/2903373719 learned_addr learned my addr 172.21.15.33:0/2903373719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a3fff640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a89ce640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a419cee0 0x7f79a419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 -- 172.21.15.33:0/2903373719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a419cee0 msgr2=0x7f79a419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a419cee0 0x7f79a419d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 -- 172.21.15.33:0/2903373719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 msgr2=0x7f79a4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 -- 172.21.15.33:0/2903373719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a419d9d0 con 0x7f79a4107290 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a3fff640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a37fe640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f799800b450 tx=0x7f799800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a17fa640 1 -- 172.21.15.33:0/2903373719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998015020 con 0x7f79a4107290 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a419dcc0 con 0x7f79a4107290 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a17fa640 1 -- 172.21.15.33:0/2903373719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7998011d40 con 0x7f79a4107290 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79a17fa640 1 -- 172.21.15.33:0/2903373719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799800f340 con 0x7f79a4107290 2024-04-23T18:31:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.107+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41a54b0 con 0x7f79a4107290 2024-04-23T18:31:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79a17fa640 1 -- 172.21.15.33:0/2903373719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7998018020 con 0x7f79a4107290 2024-04-23T18:31:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 Processor -- start 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 -- start start 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79a17fa640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 0x7f798c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cf7712e970 con 0x55cf771c9e20 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cf770cb4a0 con 0x55cf771d8c10 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.110+0000 7f79ad224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cf7712a930 con 0x55cf771c9500 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a3fff640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 0x7f798c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:55906/0 (socket says 172.21.15.33:55906) 2024-04-23T18:31:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34750/0 (socket says 172.21.15.33:34750) 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 -- 172.21.15.33:0/1097598266 learned_addr learned my addr 172.21.15.33:0/1097598266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a17fa640 1 -- 172.21.15.33:0/2903373719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f79980d00f0 con 0x7f79a4107290 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 -- 172.21.15.33:0/1097598266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 msgr2=0x55cf771db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 -- 172.21.15.33:0/1097598266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 msgr2=0x55cf771d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 -- 172.21.15.33:0/1097598266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf76e41210 con 0x55cf771c9500 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a3fff640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 0x7f798c0a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f799400abe0 tx=0x7f799400a5c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f79a89ce640 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f799c005fe0 tx=0x7f799c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fb38aa109d9268b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.111+0000 7f798a7fc640 1 -- 172.21.15.33:0/1097598266 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c032e60 con 0x55cf771c9500 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f798a7fc640 1 -- 172.21.15.33:0/1097598266 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c036070 con 0x55cf771c9500 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f798a7fc640 1 -- 172.21.15.33:0/1097598266 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c004290 con 0x55cf771c9500 2024-04-23T18:31:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 -- 172.21.15.33:0/1097598266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 msgr2=0x55cf771c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f799c005fe0 tx=0x7f799c005ef0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 -- 172.21.15.33:0/1097598266 shutdown_connections 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 --2- 172.21.15.33:0/1097598266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771c98e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 -- 172.21.15.33:0/1097598266 >> 172.21.15.33:0/1097598266 conn(0x55cf771c5b30 msgr2=0x55cf771c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 -- 172.21.15.33:0/1097598266 shutdown_connections 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.112+0000 7f79ad224740 1 -- 172.21.15.33:0/1097598266 wait complete. 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 -- start start 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 Processor -- start 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cf771db7e0 con 0x55cf771c9500 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cf770c90b0 con 0x55cf771d8c10 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cf771db960 con 0x55cf771c9e20 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79ad224740 10 client.0 _ll_register_callbacks cb 0x55cf770b6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34758/0 (socket says 172.21.15.33:34758) 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79a3fff640 1 -- 172.21.15.33:0/188033564 learned_addr learned my addr 172.21.15.33:0/188033564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79a37fe640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.113+0000 7f79a89ce640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 -- 172.21.15.33:0/188033564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 msgr2=0x55cf771d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 -- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 msgr2=0x55cf771d5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 -- 172.21.15.33:0/188033564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf76f0d620 con 0x55cf771c9500 2024-04-23T18:31:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a89ce640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79a3fff640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79940109e0 tx=0x7f79940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0e9e9f2b67e647 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79837fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994021070 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f799400fcd0 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c036070 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f799402d050 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79ad224740 10 client.16269 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf771e77f0 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.114+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f799c036070 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7994025440 con 0x55cf771c9500 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 10 client.16269 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 20 client.16269 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf771e8cb0 0x55cf771eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 4 client.16269 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7713c390 con 0x55cf771e8cb0 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79a37fe640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf771e8cb0 0x55cf771eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79a37fe640 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf771e8cb0 0x55cf771eb0c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1850d070656f9c server_cookie=1640dbd51f3de30c in_seq=0 out_seq=0 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.115+0000 7f79837fe640 10 client.16269 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79837fe640 1 -- 172.21.15.33:0/188033564 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf7713c390 con 0x55cf771e8cb0 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79837fe640 10 client.16269 handle_command_reply: tid=0 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 client.16269 shutdown 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 msgr2=0x55cf771d8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f79940109e0 tx=0x7f79940046c0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 shutdown_connections 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf771e8cb0 0x55cf771eb0c0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 --2- 172.21.15.33:0/188033564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771d8560 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79837fe640 0 client.16269 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.116+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 >> 172.21.15.33:0/188033564 conn(0x55cf771c5b30 msgr2=0x55cf771cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.117+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 shutdown_connections 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.117+0000 7f79ad224740 1 -- 172.21.15.33:0/188033564 wait complete. 2024-04-23T18:31:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.117+0000 7f79ad224740 20 client.16269 trim_cache size 0 max 16384 2024-04-23T18:31:35.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:34 smithi106 ceph-mon[32716]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 Processor -- start 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 -- start start 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ab840 con 0x55cf771c9500 2024-04-23T18:31:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ad380 con 0x55cf771d8c10 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79ad224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41abf10 con 0x55cf771c9e20 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34762/0 (socket says 172.21.15.33:34762) 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 -- 172.21.15.33:0/4196387238 learned_addr learned my addr 172.21.15.33:0/4196387238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a89ce640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a37fe640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 -- 172.21.15.33:0/4196387238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 msgr2=0x55cf771c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 -- 172.21.15.33:0/4196387238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 msgr2=0x55cf771d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 -- 172.21.15.33:0/4196387238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf771d60e0 con 0x55cf771c9500 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a89ce640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a37fe640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.132+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f79940109b0 tx=0x7f799400a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e08628ab8db66b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f7989ffb640 1 -- 172.21.15.33:0/4196387238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994035030 con 0x55cf771c9500 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f7989ffb640 1 -- 172.21.15.33:0/4196387238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994025a50 con 0x55cf771c9500 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f7989ffb640 1 -- 172.21.15.33:0/4196387238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994021050 con 0x55cf771c9500 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 -- 172.21.15.33:0/4196387238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 msgr2=0x55cf771e46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f79940109b0 tx=0x7f799400a8e0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 -- 172.21.15.33:0/4196387238 shutdown_connections 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9e20 0x55cf771c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 --2- 172.21.15.33:0/4196387238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9500 0x55cf771e46e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 -- 172.21.15.33:0/4196387238 >> 172.21.15.33:0/4196387238 conn(0x55cf771c5b30 msgr2=0x55cf771d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 -- 172.21.15.33:0/4196387238 shutdown_connections 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.133+0000 7f79ad224740 1 -- 172.21.15.33:0/4196387238 wait complete. 2024-04-23T18:31:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 -- start start 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 Processor -- start 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41accb0 con 0x55cf771c9e20 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41ac5e0 con 0x55cf771d8c10 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79a41a9820 con 0x55cf771c9500 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79ad224740 10 client.0 _ll_register_callbacks cb 0x7f7994010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34778/0 (socket says 172.21.15.33:34778) 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79a37fe640 1 -- 172.21.15.33:0/4202307073 learned_addr learned my addr 172.21.15.33:0/4202307073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.134+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a89ce640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 -- 172.21.15.33:0/4202307073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 msgr2=0x55cf771d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 -- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 msgr2=0x55cf771d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 -- 172.21.15.33:0/4202307073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7724e240 con 0x55cf771c9e20 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a89ce640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79a37fe640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f799809c1e0 tx=0x7f79980cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9386357252276d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799809c250 con 0x55cf771c9e20 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f79980988c0 con 0x55cf771c9e20 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7994025a50 con 0x55cf771c9e20 2024-04-23T18:31:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f79980a0600 con 0x55cf771c9e20 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 10 client.16275 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cf7724f4c0 con 0x55cf771c9e20 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7994025a50 con 0x55cf771c9e20 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f79980998a0 con 0x55cf771c9e20 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 10 client.16275 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 20 client.16275 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf7724fbf0 0x55cf77252010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 4 client.16275 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.135+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf77136200 con 0x55cf7724fbf0 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.136+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf7724fbf0 0x55cf77252010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.136+0000 7f79837fe640 10 client.16275 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.136+0000 7f79a3fff640 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf7724fbf0 0x55cf77252010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59b440072adfbb48 server_cookie=371cdd9f33c16a70 in_seq=0 out_seq=0 2024-04-23T18:31:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79837fe640 1 -- 172.21.15.33:0/4202307073 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf77136200 con 0x55cf7724fbf0 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79837fe640 10 client.16275 handle_command_reply: tid=0 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 client.16275 shutdown 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 msgr2=0x55cf771d2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f799809c1e0 tx=0x7f79980cf040 comp rx=0 tx=0).stop 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 shutdown_connections 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cf7724fbf0 0x55cf77252010 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf771d8c10 0x55cf771d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cf771c9e20 0x55cf771d2a80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79837fe640 0 client.16275 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 --2- 172.21.15.33:0/4202307073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cf771c9500 0x55cf771d2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 >> 172.21.15.33:0/4202307073 conn(0x55cf771c5b30 msgr2=0x55cf771d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 shutdown_connections 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 1 -- 172.21.15.33:0/4202307073 wait complete. 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79ad224740 20 client.16275 trim_cache size 0 max 16384 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:31:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 msgr2=0x7f798c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 0x7f798c0a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f799400abe0 tx=0x7f799400a5c0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 msgr2=0x7f79a4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.144+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f799800b450 tx=0x7f799800b910 comp rx=0 tx=0).stop 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 shutdown_connections 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f798c0a3dc0 0x7f798c0a6280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79a419cee0 0x7f79a419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a4107290 0x7f79a4198b60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 --2- 172.21.15.33:0/2903373719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a41021c0 0x7f79a4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 >> 172.21.15.33:0/2903373719 conn(0x7f79a40fda20 msgr2=0x7f79a40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 shutdown_connections 2024-04-23T18:31:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:35.145+0000 7f79aa431640 1 -- 172.21.15.33:0/2903373719 wait complete. 2024-04-23T18:31:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:34 smithi122 ceph-mon[32546]: pgmap v156: 97 pgs: 97 active+clean; 3.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:36.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:36 smithi033 ceph-mon[29274]: pgmap v157: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T18:31:37.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:36 smithi106 ceph-mon[32716]: pgmap v157: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T18:31:37.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:36 smithi122 ceph-mon[32546]: pgmap v157: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-23T18:31:38.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:38 smithi033 ceph-mon[29274]: pgmap v158: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:38.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:38 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:39.000 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:38 smithi122 ceph-mon[32546]: pgmap v158: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:39.000 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:38 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:39.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:38 smithi106 ceph-mon[32716]: pgmap v158: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:39.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:38 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:31:39.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:39 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:39.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:39 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:39.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:39 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:31:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:39 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:39 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:39 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:31:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:39 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:31:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:39 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:31:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:39 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:31:40.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:40 smithi033 ceph-mon[29274]: pgmap v159: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:40 smithi106 ceph-mon[32716]: pgmap v159: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:40 smithi122 ceph-mon[32546]: pgmap v159: 97 pgs: 97 active+clean; 3.6 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 14 op/s 2024-04-23T18:31:42.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:42 smithi033 ceph-mon[29274]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:42 smithi106 ceph-mon[32716]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:42 smithi122 ceph-mon[32546]: pgmap v160: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-04-23T18:31:44.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:44 smithi033 ceph-mon[29274]: pgmap v161: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:45.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:44 smithi106 ceph-mon[32716]: pgmap v161: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:45.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:44 smithi122 ceph-mon[32546]: pgmap v161: 97 pgs: 97 active+clean; 3.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:46.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:46 smithi033 ceph-mon[29274]: pgmap v162: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:47.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:46 smithi106 ceph-mon[32716]: pgmap v162: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:47.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:46 smithi122 ceph-mon[32546]: pgmap v162: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 -- 172.21.15.33:0/2020337367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 msgr2=0x7fa54c101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 --2- 172.21.15.33:0/2020337367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c101350 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa548009930 tx=0x7fa54802a4b0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 -- 172.21.15.33:0/2020337367 shutdown_connections 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 --2- 172.21.15.33:0/2020337367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 --2- 172.21.15.33:0/2020337367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 0x7fa54c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 --2- 172.21.15.33:0/2020337367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c101350 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.244+0000 7fa553fc6640 1 -- 172.21.15.33:0/2020337367 >> 172.21.15.33:0/2020337367 conn(0x7fa54c077fd0 msgr2=0x7fa54c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- 172.21.15.33:0/2020337367 shutdown_connections 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- 172.21.15.33:0/2020337367 wait complete. 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 Processor -- start 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- start start 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 0x7fa54c1a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1187c0 con 0x7fa54c100f70 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c118640 con 0x7fa54c106370 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa553fc6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c118940 con 0x7fa54c101890 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.245+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38688/0 (socket says 172.21.15.33:38688) 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 -- 172.21.15.33:0/3866802569 learned_addr learned my addr 172.21.15.33:0/3866802569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:48.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551561640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 0x7fa54c1a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa552563640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 -- 172.21.15.33:0/3866802569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 msgr2=0x7fa54c1a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 0x7fa54c1a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 -- 172.21.15.33:0/3866802569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 msgr2=0x7fa54c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 -- 172.21.15.33:0/3866802569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c19f280 con 0x7fa54c100f70 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa552563640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa551d62640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa548005940 tx=0x7fa5480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa542ffd640 1 -- 172.21.15.33:0/3866802569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54803f030 con 0x7fa54c100f70 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa542ffd640 1 -- 172.21.15.33:0/3866802569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa548003cb0 con 0x7fa54c100f70 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c19f510 con 0x7fa54c100f70 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa542ffd640 1 -- 172.21.15.33:0/3866802569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548035070 con 0x7fa54c100f70 2024-04-23T18:31:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.246+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c1ab7c0 con 0x7fa54c100f70 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.249+0000 7fa542ffd640 1 -- 172.21.15.33:0/3866802569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa548042030 con 0x7fa54c100f70 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.249+0000 7fa556db8740 1 Processor -- start 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.249+0000 7fa556db8740 1 -- start start 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399957a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 0x565399964bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653998bc970 con 0x565399957f40 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5653998594a0 con 0x565399962800 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa556db8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653998b8930 con 0x565399957620 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa542ffd640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 0x7fa5380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa552563640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 0x7fa5380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38698/0 (socket says 172.21.15.33:38698) 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551d62640 1 -- 172.21.15.33:0/1671910297 learned_addr learned my addr 172.21.15.33:0/1671910297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 0x565399964bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa552563640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399957a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa542ffd640 1 -- 172.21.15.33:0/3866802569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa5480f1050 con 0x7fa54c100f70 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 -- 172.21.15.33:0/1671910297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 msgr2=0x565399957a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399957a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 -- 172.21.15.33:0/1671910297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 msgr2=0x5653999622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551561640 1 -- 172.21.15.33:0/1671910297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653995cf210 con 0x565399962800 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa551d62640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:31:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.250+0000 7fa552563640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 0x7fa5380a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa544005fd0 tx=0x7fa544009db0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.251+0000 7fa551561640 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 0x565399964bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa54c1a02a0 tx=0x7fa53c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=589fc94b623333d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.251+0000 7fa537fff640 1 -- 172.21.15.33:0/1671910297 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c004030 con 0x565399962800 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.251+0000 7fa537fff640 1 -- 172.21.15.33:0/1671910297 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53c0035f0 con 0x565399962800 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.251+0000 7fa537fff640 1 -- 172.21.15.33:0/1671910297 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c0116a0 con 0x565399962800 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- 172.21.15.33:0/1671910297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 msgr2=0x565399964bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 0x565399964bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa54c1a02a0 tx=0x7fa53c0107a0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- 172.21.15.33:0/1671910297 shutdown_connections 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399962800 0x565399964bf0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x5653999622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 --2- 172.21.15.33:0/1671910297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399957a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- 172.21.15.33:0/1671910297 >> 172.21.15.33:0/1671910297 conn(0x565399953b30 msgr2=0x565399953f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- 172.21.15.33:0/1671910297 shutdown_connections 2024-04-23T18:31:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- 172.21.15.33:0/1671910297 wait complete. 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 -- start start 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.252+0000 7fa556db8740 1 Processor -- start 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957620 0x5653999620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 0x56539995f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565399965430 con 0x565399957f40 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5653998570b0 con 0x565399957620 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653999655b0 con 0x565399962800 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa556db8740 10 client.0 _ll_register_callbacks cb 0x565399844d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38714/0 (socket says 172.21.15.33:38714) 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 -- 172.21.15.33:0/4011181820 learned_addr learned my addr 172.21.15.33:0/4011181820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551d62640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957620 0x5653999620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa552563640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 0x56539995f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 -- 172.21.15.33:0/4011181820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 msgr2=0x56539995f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 0x56539995f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 -- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957620 msgr2=0x5653999620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957620 0x5653999620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 -- 172.21.15.33:0/4011181820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539969b620 con 0x565399957f40 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa552563640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 0x56539995f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa551561640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00beb0 tx=0x7fa53c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1379f68c8a6b7fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.253+0000 7fa5357fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c02a060 con 0x565399957f40 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa53c004310 con 0x565399957f40 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c0035f0 con 0x565399957f40 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa53c0037a0 con 0x565399957f40 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 10 client.16287 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565399971310 con 0x565399957f40 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53c0035f0 con 0x565399957f40 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa53c004750 con 0x565399957f40 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 10 client.16287 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 10 client.16287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 20 client.16287 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653999726c0 0x565399974ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 4 client.16287 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.254+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653998ca390 con 0x5653999726c0 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.255+0000 7fa551d62640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653999726c0 0x565399974ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.255+0000 7fa551d62640 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653999726c0 0x565399974ab0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab61d0e1982c82ab server_cookie=4a2b84cc358a19a5 in_seq=0 out_seq=0 2024-04-23T18:31:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.255+0000 7fa5357fa640 10 client.16287 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa5357fa640 1 -- 172.21.15.33:0/4011181820 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653998ca390 con 0x5653999726c0 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa5357fa640 10 client.16287 handle_command_reply: tid=0 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 client.16287 shutdown 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 msgr2=0x56539995f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00beb0 tx=0x7fa53c00bee0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 shutdown_connections 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653999726c0 0x565399974ab0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399962800 0x56539995f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa5357fa640 0 client.16287 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957f40 0x56539995f3d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 --2- 172.21.15.33:0/4011181820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957620 0x5653999620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 >> 172.21.15.33:0/4011181820 conn(0x565399953b30 msgr2=0x5653999598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 shutdown_connections 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 1 -- 172.21.15.33:0/4011181820 wait complete. 2024-04-23T18:31:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.256+0000 7fa556db8740 20 client.16287 trim_cache size 0 max 16384 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 Processor -- start 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 -- start start 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399953b30 0x56539996e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x56539997a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b40e0 con 0x565399957620 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b5c20 con 0x565399957f40 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.271+0000 7fa556db8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b47b0 con 0x565399953b30 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38726/0 (socket says 172.21.15.33:38726) 2024-04-23T18:31:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 -- 172.21.15.33:0/16768556 learned_addr learned my addr 172.21.15.33:0/16768556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551d62640 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399953b30 0x56539996e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 -- 172.21.15.33:0/16768556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399953b30 msgr2=0x56539996e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399953b30 0x56539996e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 -- 172.21.15.33:0/16768556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 msgr2=0x56539997a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x56539997a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 -- 172.21.15.33:0/16768556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565399957d70 con 0x565399957620 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa551561640 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00b3d0 tx=0x7fa53c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace69183f71f5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa5377fe640 1 -- 172.21.15.33:0/16768556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c02a030 con 0x565399957620 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa5377fe640 1 -- 172.21.15.33:0/16768556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53c0035f0 con 0x565399957620 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.272+0000 7fa5377fe640 1 -- 172.21.15.33:0/16768556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53c011800 con 0x565399957620 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- 172.21.15.33:0/16768556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 msgr2=0x565399952b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa53c00b3d0 tx=0x7fa53c00b7b0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- 172.21.15.33:0/16768556 shutdown_connections 2024-04-23T18:31:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x56539997a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399957620 0x565399952b30 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 --2- 172.21.15.33:0/16768556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399953b30 0x56539996e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- 172.21.15.33:0/16768556 >> 172.21.15.33:0/16768556 conn(0x56539995fa70 msgr2=0x565399952420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- 172.21.15.33:0/16768556 shutdown_connections 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- 172.21.15.33:0/16768556 wait complete. 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 -- start start 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.273+0000 7fa556db8740 1 Processor -- start 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399968070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x565399968680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b5550 con 0x565399953b30 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b4e80 con 0x565399957f40 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b20c0 con 0x565399957620 2024-04-23T18:31:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa556db8740 10 client.0 _ll_register_callbacks cb 0x7fa53c00ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38730/0 (socket says 172.21.15.33:38730) 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 -- 172.21.15.33:0/989144115 learned_addr learned my addr 172.21.15.33:0/989144115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa552563640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x565399968680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551561640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399968070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 -- 172.21.15.33:0/989144115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 msgr2=0x565399968070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399968070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 -- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 msgr2=0x565399968680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x565399968680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 -- 172.21.15.33:0/989144115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565399968eb0 con 0x565399953b30 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551561640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399968070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa551d62640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa548003f80 tx=0x7fa54808c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a737a1e0cded651b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa5357fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.274+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548049040 con 0x565399953b30 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa5480b9610 con 0x565399953b30 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53c0035f0 con 0x565399953b30 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5480bcfa0 con 0x565399953b30 2024-04-23T18:31:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 10 client.16293 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5653999d8b00 con 0x565399953b30 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa53c0035f0 con 0x565399953b30 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa5480f30b0 con 0x565399953b30 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 10 client.16293 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 20 client.16293 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x565399962800 0x5653999db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 4 client.16293 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653998c4200 con 0x565399962800 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.275+0000 7fa551561640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x565399962800 0x5653999db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.276+0000 7fa551561640 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x565399962800 0x5653999db1e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70415c45651df7fb server_cookie=836b0c94c463f2c5 in_seq=0 out_seq=0 2024-04-23T18:31:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.276+0000 7fa5357fa640 10 client.16293 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa5357fa640 1 -- 172.21.15.33:0/989144115 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653998c4200 con 0x565399962800 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa5357fa640 10 client.16293 handle_command_reply: tid=0 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 client.16293 shutdown 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 msgr2=0x56539996ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa548003f80 tx=0x7fa54808c880 comp rx=0 tx=0).stop 2024-04-23T18:31:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 shutdown_connections 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x565399962800 0x5653999db1e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x565399957f40 0x565399968680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565399957620 0x565399968070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 --2- 172.21.15.33:0/989144115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565399953b30 0x56539996ace0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa5357fa640 0 client.16293 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 >> 172.21.15.33:0/989144115 conn(0x56539995fa70 msgr2=0x565399952cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 shutdown_connections 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.283+0000 7fa556db8740 1 -- 172.21.15.33:0/989144115 wait complete. 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa556db8740 20 client.16293 trim_cache size 0 max 16384 2024-04-23T18:31:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 msgr2=0x7fa5380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 0x7fa5380a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa544005fd0 tx=0x7fa544009db0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 msgr2=0x7fa54c1a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa548005940 tx=0x7fa5480042f0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 shutdown_connections 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa5380a3dc0 0x7fa5380a6280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa54c106370 0x7fa54c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa54c101890 0x7fa54c1a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 --2- 172.21.15.33:0/3866802569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa54c100f70 0x7fa54c1a4900 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 >> 172.21.15.33:0/3866802569 conn(0x7fa54c077fd0 msgr2=0x7fa54c103d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 shutdown_connections 2024-04-23T18:31:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:31:48.284+0000 7fa553fc6640 1 -- 172.21.15.33:0/3866802569 wait complete. 2024-04-23T18:31:48.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:48 smithi033 ceph-mon[29274]: pgmap v163: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:49.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:48 smithi106 ceph-mon[32716]: pgmap v163: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:49.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:48 smithi122 ceph-mon[32546]: pgmap v163: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:127/600: open fd = 3 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:128/600: open fd = 3 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:129/600: open fd = 3 2024-04-23T18:31:49.953 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:130/600: open fd = 3 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:131/600: open fd = 3 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:132/600: open fd = 3 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.954 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:133/600: open fd = 3 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:134/600: open fd = 3 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:135/600: open fd = 3 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:136/600: open fd = 3 2024-04-23T18:31:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:137/600: open fd = 3 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:138/600: open fd = 3 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:139/600: open fd = 3 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:140/600: open fd = 3 2024-04-23T18:31:49.956 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:141/600: open fd = 3 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:142/600: open fd = 3 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:143/600: open fd = 3 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.957 INFO:tasks.workunit.client.0.smithi033.stdout:144/600: open fd = 3 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:145/600: open fd = 3 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:146/600: open fd = 3 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:147/600: open fd = 3 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:148/600: open fd = 3 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:149/600: open fd = 3 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:150/600: open fd = 3 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.959 INFO:tasks.workunit.client.0.smithi033.stdout:151/600: open fd = 3 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:152/600: open fd = 3 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:153/600: open fd = 3 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:154/600: open fd = 3 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:155/600: open fd = 3 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:156/600: open fd = 3 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:157/600: open fd = 3 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:158/600: open fd = 3 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:159/600: open fd = 3 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:160/600: open fd = 3 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:161/600: open fd = 3 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:162/600: open fd = 3 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:163/600: open fd = 3 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:164/600: open fd = 3 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:165/600: open fd = 3 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.963 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:166/600: open fd = 3 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:167/600: open fd = 3 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:168/600: open fd = 3 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:169/600: open fd = 3 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.964 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:170/600: open fd = 3 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:171/600: open fd = 3 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:172/600: open fd = 3 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:173/600: open fd = 3 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.965 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:174/600: open fd = 3 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:175/600: open fd = 3 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:176/600: open fd = 3 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:177/600: open fd = 3 2024-04-23T18:31:49.966 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:178/600: open fd = 3 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:179/600: open fd = 3 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:180/600: open fd = 3 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.967 INFO:tasks.workunit.client.0.smithi033.stdout:181/600: open fd = 3 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:182/600: open fd = 3 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:183/600: open fd = 3 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:184/600: open fd = 3 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.968 INFO:tasks.workunit.client.0.smithi033.stdout:185/600: open fd = 3 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:186/600: open fd = 3 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:187/600: open fd = 3 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:188/600: open fd = 3 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:31:51.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:50 smithi106 ceph-mon[32716]: pgmap v164: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:51.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:50 smithi122 ceph-mon[32546]: pgmap v164: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:51.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:50 smithi033 ceph-mon[29274]: pgmap v164: 97 pgs: 97 active+clean; 3.8 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-23T18:31:53.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:52 smithi106 ceph-mon[32716]: pgmap v165: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:53.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:52 smithi122 ceph-mon[32546]: pgmap v165: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:52 smithi033 ceph-mon[29274]: pgmap v165: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T18:31:54.626 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:31:54.626 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.8 2024-04-23T18:31:54.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:31:54.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 -- 172.21.15.33:0/922985352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c103b00 msgr2=0x7f4f8c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 --2- 172.21.15.33:0/922985352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c103b00 0x7f4f8c10bfe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c009900 tx=0x7f4f7c02a510 comp rx=0 tx=0).stop 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 -- 172.21.15.33:0/922985352 shutdown_connections 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 --2- 172.21.15.33:0/922985352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c10c520 0x7f4f8c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 --2- 172.21.15.33:0/922985352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c103b00 0x7f4f8c10bfe0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 --2- 172.21.15.33:0/922985352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 -- 172.21.15.33:0/922985352 >> 172.21.15.33:0/922985352 conn(0x7f4f8c077f50 msgr2=0x7f4f8c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 -- 172.21.15.33:0/922985352 shutdown_connections 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.750+0000 7f4f93908640 1 -- 172.21.15.33:0/922985352 wait complete. 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 Processor -- start 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 -- start start 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c103b00 0x7f4f8c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 0x7f4f8c19cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c1121b0 con 0x7f4f8c1031e0 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c112030 con 0x7f4f8c10c520 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f93908640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f8c112330 con 0x7f4f8c103b00 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38740/0 (socket says 172.21.15.33:38740) 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 -- 172.21.15.33:0/4273533054 learned_addr learned my addr 172.21.15.33:0/4273533054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f90ea3640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c103b00 0x7f4f8c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f91ea5640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 0x7f4f8c19cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 -- 172.21.15.33:0/4273533054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c103b00 msgr2=0x7f4f8c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c103b00 0x7f4f8c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 -- 172.21.15.33:0/4273533054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 msgr2=0x7f4f8c19cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 0x7f4f8c19cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f916a4640 1 -- 172.21.15.33:0/4273533054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f8c19d720 con 0x7f4f8c1031e0 2024-04-23T18:31:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.751+0000 7f4f91ea5640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 0x7f4f8c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f916a4640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9f0 tx=0x7f4f8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8800cd80 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f8c19da10 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f880108e0 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88018980 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.752+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8c1a52a0 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.753+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f8c068d70 con 0x7f4f8c1031e0 2024-04-23T18:31:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.754+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f88010430 con 0x7f4f8c1031e0 2024-04-23T18:31:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.755+0000 7f4f827fc640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 0x7f4f780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.755+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f4f880d1090 con 0x7f4f8c1031e0 2024-04-23T18:31:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.755+0000 7f4f90ea3640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 0x7f4f780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.755+0000 7f4f90ea3640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 0x7f4f780a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c031440 tx=0x7f4f7c005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.758+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f8809d6e0 con 0x7f4f8c1031e0 2024-04-23T18:31:54.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:54 smithi033 ceph-mon[29274]: pgmap v166: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:54.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.990+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f8c199a80 con 0x7f4f8c1031e0 2024-04-23T18:31:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.990+0000 7f4f827fc640 1 -- 172.21.15.33:0/4273533054 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11926 (secure 0 0 0) 0x7f4f8809cfe0 con 0x7f4f8c1031e0 2024-04-23T18:31:54.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:31:54.984 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:30:16.151605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25696":{"gid":25696,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/111505116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":111505116},{"type":"v1","addr":"172.21.15.122:6835","nonce":111505116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:31:54.984 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-04-23T18:31:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 msgr2=0x7f4f780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 0x7f4f780a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c031440 tx=0x7f4f7c005b50 comp rx=0 tx=0).stop 2024-04-23T18:31:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 msgr2=0x7f4f8c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:54.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e9f0 tx=0x7f4f8800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 shutdown_connections 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4f780a3d20 0x7f4f780a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4f8c10c520 0x7f4f8c19cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f8c103b00 0x7f4f8c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 --2- 172.21.15.33:0/4273533054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f8c1031e0 0x7f4f8c198730 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 >> 172.21.15.33:0/4273533054 conn(0x7f4f8c077f50 msgr2=0x7f4f8c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 shutdown_connections 2024-04-23T18:31:54.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:54.993+0000 7f4f93908640 1 -- 172.21.15.33:0/4273533054 wait complete. 2024-04-23T18:31:54.998 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0' 2024-04-23T18:31:54.998 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0 2024-04-23T18:31:55.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- 172.21.15.33:0/451027237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a41010a0 msgr2=0x7f68a4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 --2- 172.21.15.33:0/451027237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a41010a0 0x7f68a4101480 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009900 tx=0x7f68a002a410 comp rx=0 tx=0).stop 2024-04-23T18:31:55.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- 172.21.15.33:0/451027237 shutdown_connections 2024-04-23T18:31:55.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 --2- 172.21.15.33:0/451027237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a4106170 0x7f68a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 --2- 172.21.15.33:0/451027237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a4101a50 0x7f68a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 --2- 172.21.15.33:0/451027237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a41010a0 0x7f68a4101480 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- 172.21.15.33:0/451027237 >> 172.21.15.33:0/451027237 conn(0x7f68a4077f50 msgr2=0x7f68a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- 172.21.15.33:0/451027237 shutdown_connections 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- 172.21.15.33:0/451027237 wait complete. 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 Processor -- start 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.117+0000 7f68aba4e640 1 -- start start 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 0x7f68a41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 0x7f68a419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41110d0 con 0x7f68a4106170 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4110f50 con 0x7f68a41010a0 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a4111250 con 0x7f68a419a720 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38760/0 (socket says 172.21.15.33:38760) 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 -- 172.21.15.33:0/2525585501 learned_addr learned my addr 172.21.15.33:0/2525585501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a97ea640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 0x7f68a41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 0x7f68a419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 -- 172.21.15.33:0/2525585501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 msgr2=0x7f68a419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 0x7f68a419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 -- 172.21.15.33:0/2525585501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 msgr2=0x7f68a41a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 0x7f68a41a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 -- 172.21.15.33:0/2525585501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a419b320 con 0x7f68a4106170 2024-04-23T18:31:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a97ea640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 0x7f68a41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 0x7f68a419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9f0 tx=0x7f689400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f689a7fc640 1 -- 172.21.15.33:0/2525585501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400ccc0 con 0x7f68a4106170 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a41a7400 con 0x7f68a4106170 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f689a7fc640 1 -- 172.21.15.33:0/2525585501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68940054d0 con 0x7f68a4106170 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f689a7fc640 1 -- 172.21.15.33:0/2525585501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894010430 con 0x7f68a4106170 2024-04-23T18:31:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.118+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a41a7800 con 0x7f68a4106170 2024-04-23T18:31:55.121 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:54 smithi106 ceph-mon[32716]: pgmap v166: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.120+0000 7f689a7fc640 1 -- 172.21.15.33:0/2525585501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68940046c0 con 0x7f68a4106170 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 Processor -- start 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 -- start start 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4479c0 0x55fcdb451b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb454480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb3ac370 con 0x55fcdb4470a0 2024-04-23T18:31:55.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb3492e0 con 0x55fcdb452090 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68ae765740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb3a8330 con 0x55fcdb4479c0 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f689a7fc640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68a9feb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68a9feb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38764/0 (socket says 172.21.15.33:38764) 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.121+0000 7f68a9feb640 1 -- 172.21.15.33:0/2810705294 learned_addr learned my addr 172.21.15.33:0/2810705294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a97ea640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4479c0 0x55fcdb451b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f689a7fc640 1 -- 172.21.15.33:0/2525585501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f68940d0cf0 con 0x7f68a4106170 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb454480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 -- 172.21.15.33:0/2810705294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4479c0 msgr2=0x55fcdb451b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4479c0 0x55fcdb451b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 -- 172.21.15.33:0/2810705294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 msgr2=0x55fcdb454480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb454480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 -- 172.21.15.33:0/2810705294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcdb18ad80 con 0x55fcdb4470a0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb454480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f689c0149a0 tx=0x7f689c014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d04554e3ae84405 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f688f7fe640 1 -- 172.21.15.33:0/2810705294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c021070 con 0x55fcdb4470a0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f688f7fe640 1 -- 172.21.15.33:0/2810705294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c013460 con 0x55fcdb4470a0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f688f7fe640 1 -- 172.21.15.33:0/2810705294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c00f5d0 con 0x55fcdb4470a0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.122+0000 7f68a9feb640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f689c01e000 tx=0x7f689c019000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- 172.21.15.33:0/2810705294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 msgr2=0x55fcdb447480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f689c0149a0 tx=0x7f689c014e60 comp rx=0 tx=0).stop 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- 172.21.15.33:0/2810705294 shutdown_connections 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb454480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4479c0 0x55fcdb451b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 --2- 172.21.15.33:0/2810705294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4470a0 0x55fcdb447480 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- 172.21.15.33:0/2810705294 >> 172.21.15.33:0/2810705294 conn(0x55fcdb4433c0 msgr2=0x55fcdb4437a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- 172.21.15.33:0/2810705294 shutdown_connections 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- 172.21.15.33:0/2810705294 wait complete. 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 -- start start 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.123+0000 7f68ae765740 1 Processor -- start 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb44f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb44fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb454e80 con 0x55fcdb4479c0 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb346fb0 con 0x55fcdb452090 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fcdb455000 con 0x55fcdb4470a0 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68ae765740 10 client.0 _ll_register_callbacks cb 0x55fcdb3469a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38774/0 (socket says 172.21.15.33:38774) 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 -- 172.21.15.33:0/527710399 learned_addr learned my addr 172.21.15.33:0/527710399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a9feb640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb44fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a97ea640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb44f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 -- 172.21.15.33:0/527710399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 msgr2=0x55fcdb44f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb44f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 -- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 msgr2=0x55fcdb44fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb44fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 -- 172.21.15.33:0/527710399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcdb1eb510 con 0x55fcdb4479c0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a97ea640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb44f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f68940990e0 tx=0x7f6894005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1023a2d2669f1a8e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f688cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.124+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68940b5020 con 0x55fcdb4479c0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689c013460 con 0x55fcdb4479c0 2024-04-23T18:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f689409cee0 con 0x55fcdb4479c0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f68940ce8b0 con 0x55fcdb4479c0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 10 client.16311 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcdb07e670 con 0x55fcdb4479c0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689c013460 con 0x55fcdb4479c0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f68940a5460 con 0x55fcdb4479c0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 10 client.16311 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 10 client.16311 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 20 client.16311 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb462360 0x55fcdb464750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 4 client.16311 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcdb3b9d90 con 0x55fcdb462360 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.125+0000 7f68a97ea640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb462360 0x55fcdb464750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.126+0000 7f68a97ea640 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb462360 0x55fcdb464750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25fae6e45ee5fb9 server_cookie=7628aac9f7287dd6 in_seq=0 out_seq=0 2024-04-23T18:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.126+0000 7f688cff9640 10 client.16311 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f688cff9640 1 -- 172.21.15.33:0/527710399 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fcdb3b9d90 con 0x55fcdb462360 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f688cff9640 10 client.16311 handle_command_reply: tid=0 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 client.16311 shutdown 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 msgr2=0x55fcdb44f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f68940990e0 tx=0x7f6894005680 comp rx=0 tx=0).stop 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 shutdown_connections 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb462360 0x55fcdb464750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb44fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f688cff9640 0 client.16311 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f550 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 --2- 172.21.15.33:0/527710399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb44f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 >> 172.21.15.33:0/527710399 conn(0x55fcdb4433c0 msgr2=0x55fcdb4491c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 shutdown_connections 2024-04-23T18:31:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 1 -- 172.21.15.33:0/527710399 wait complete. 2024-04-23T18:31:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.127+0000 7f68ae765740 20 client.16311 trim_cache size 0 max 16384 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 Processor -- start 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 -- start start 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb442d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb469f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41afdb0 con 0x55fcdb4479c0 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b18f0 con 0x55fcdb452090 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.144+0000 7f68ae765740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b0480 con 0x55fcdb4470a0 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38784/0 (socket says 172.21.15.33:38784) 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 -- 172.21.15.33:0/4071921732 learned_addr learned my addr 172.21.15.33:0/4071921732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a97ea640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb442d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 -- 172.21.15.33:0/4071921732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 msgr2=0x55fcdb442d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a9feb640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb469f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb442d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 -- 172.21.15.33:0/4071921732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 msgr2=0x55fcdb469f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb469f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 -- 172.21.15.33:0/4071921732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcdb447f10 con 0x55fcdb4479c0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a9feb640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb469f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f689400c6a0 tx=0x7f68940ceef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf98bb092a16752 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f688effd640 1 -- 172.21.15.33:0/4071921732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894098f20 con 0x55fcdb4479c0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f688effd640 1 -- 172.21.15.33:0/4071921732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689409cc80 con 0x55fcdb4479c0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.145+0000 7f688effd640 1 -- 172.21.15.33:0/4071921732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689409cee0 con 0x55fcdb4479c0 2024-04-23T18:31:55.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- 172.21.15.33:0/4071921732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 msgr2=0x55fcdb44f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f689400c6a0 tx=0x7f68940ceef0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- 172.21.15.33:0/4071921732 shutdown_connections 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb452090 0x55fcdb469f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb4479c0 0x55fcdb44f6e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- 172.21.15.33:0/4071921732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb442d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- 172.21.15.33:0/4071921732 >> 172.21.15.33:0/4071921732 conn(0x55fcdb4433c0 msgr2=0x55fcdb44efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- 172.21.15.33:0/4071921732 shutdown_connections 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- 172.21.15.33:0/4071921732 wait complete. 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- start start 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 Processor -- start 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb45a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb4479c0 0x55fcdb45a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.146+0000 7f68ae765740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b1220 con 0x55fcdb452090 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41b0b50 con 0x55fcdb4479c0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68a41add90 con 0x55fcdb4470a0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 10 client.0 _ll_register_callbacks cb 0x7f689409cb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38786/0 (socket says 172.21.15.33:38786) 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 -- 172.21.15.33:0/267805663 learned_addr learned my addr 172.21.15.33:0/267805663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a97ea640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb45a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a8fe9640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb4479c0 0x55fcdb45a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 -- 172.21.15.33:0/267805663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 msgr2=0x55fcdb45a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb45a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 -- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb4479c0 msgr2=0x55fcdb45a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:31:55.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb4479c0 0x55fcdb45a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 -- 172.21.15.33:0/267805663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcdb457a30 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68a9feb640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f689c0149a0 tx=0x7f689c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365d44d5f13022ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f688cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c034060 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f689c004c00 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689409cc80 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f689c021040 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 10 client.16317 fetch_fsmap learned FSMap version 32 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.147+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fcdb4c9200 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f689409cc80 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f689c0050c0 con 0x55fcdb452090 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 10 client.16317 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 10 client.16317 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 20 client.16317 populate_metadata read hostname 'smithi033' 2024-04-23T18:31:55.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb4c9850 0x55fcdb4cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:31:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 4 client.16317 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}] 2024-04-23T18:31:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}) -- 0x55fcdb3b3c00 con 0x55fcdb4c9850 2024-04-23T18:31:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68a97ea640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb4c9850 0x55fcdb4cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:31:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f688cff9640 10 client.16317 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:55.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.148+0000 7f68a97ea640 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb4c9850 0x55fcdb4cbc70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd69d9a46b27021 server_cookie=5cb1bb22be2466cc in_seq=0 out_seq=0 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f688cff9640 1 -- 172.21.15.33:0/267805663 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fcdb3b3c00 con 0x55fcdb4c9850 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f688cff9640 10 client.16317 handle_command_reply: tid=0 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 client.16317 shutdown 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 msgr2=0x55fcdb457220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f689c0149a0 tx=0x7f689c00e000 comp rx=0 tx=0).stop 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 shutdown_connections 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fcdb4c9850 0x55fcdb4cbc70 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fcdb452090 0x55fcdb457220 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fcdb4479c0 0x55fcdb45a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f688cff9640 0 client.16317 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 --2- 172.21.15.33:0/267805663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fcdb4470a0 0x55fcdb45a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 >> 172.21.15.33:0/267805663 conn(0x55fcdb4433c0 msgr2=0x55fcdb44f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 shutdown_connections 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 1 -- 172.21.15.33:0/267805663 wait complete. 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68ae765740 20 client.16317 trim_cache size 0 max 16384 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 32, 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 18, 2024-04-23T18:31:55.159 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "7da0f1de": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 18, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 78.8, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:31:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:31:55.161 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 msgr2=0x7f68900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 0x7f68900a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f689c01e000 tx=0x7f689c019000 comp rx=0 tx=0).stop 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 msgr2=0x7f68a41a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f689400e9f0 tx=0x7f689400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 shutdown_connections 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f68900a3d20 0x7f68900a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68a419a720 0x7f68a419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68a4106170 0x7f68a41a0ba0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 --2- 172.21.15.33:0/2525585501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f68a41010a0 0x7f68a41a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:31:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 >> 172.21.15.33:0/2525585501 conn(0x7f68a4077f50 msgr2=0x7f68a4100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:31:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 shutdown_connections 2024-04-23T18:31:55.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:31:55.165+0000 7f68aba4e640 1 -- 172.21.15.33:0/2525585501 wait complete. 2024-04-23T18:31:55.169 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7da0f1de', quiesce took 0.5 seconds. Will release after: 18.8 2024-04-23T18:31:55.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:54 smithi122 ceph-mon[32546]: pgmap v166: 97 pgs: 97 active+clean; 3.9 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:56.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:55 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4273533054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:56.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:55 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4273533054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:56.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:55 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4273533054' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:31:57.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:56 smithi106 ceph-mon[32716]: pgmap v167: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:57.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:56 smithi122 ceph-mon[32546]: pgmap v167: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:57.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:56 smithi033 ceph-mon[29274]: pgmap v167: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-23T18:31:59.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:31:58 smithi106 ceph-mon[32716]: pgmap v168: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:59.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:31:58 smithi122 ceph-mon[32546]: pgmap v168: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:31:59.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:31:58 smithi033 ceph-mon[29274]: pgmap v168: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:32:01.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:00 smithi106 ceph-mon[32716]: pgmap v169: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:32:01.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:00 smithi122 ceph-mon[32546]: pgmap v169: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:32:01.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:00 smithi033 ceph-mon[29274]: pgmap v169: 97 pgs: 97 active+clean; 3.9 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 14 op/s 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 -- 172.21.15.33:0/3459459134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 msgr2=0x7fc7d810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/3459459134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d810dd80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009ca0 tx=0x7fc7d002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 -- 172.21.15.33:0/3459459134 shutdown_connections 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/3459459134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d810dd80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/3459459134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d8101a90 0x7fc7d8105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/3459459134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 -- 172.21.15.33:0/3459459134 >> 172.21.15.33:0/3459459134 conn(0x7fc7d80771d0 msgr2=0x7fc7d80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 -- 172.21.15.33:0/3459459134 shutdown_connections 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 -- 172.21.15.33:0/3459459134 wait complete. 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.384+0000 7fc7de6cb640 1 Processor -- start 2024-04-23T18:32:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 -- start start 2024-04-23T18:32:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d8102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d81035f0 0x7fc7d81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8111100 con 0x7fc7d81061b0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8110f80 con 0x7fc7d81010e0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7de6cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d8111280 con 0x7fc7d81035f0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49460/0 (socket says 172.21.15.33:49460) 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2695010566 learned_addr learned my addr 172.21.15.33:0/2695010566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d8102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.385+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d81035f0 0x7fc7d81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2695010566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d81035f0 msgr2=0x7fc7d81a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d81035f0 0x7fc7d81a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2695010566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 msgr2=0x7fc7d8102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d8102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2695010566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d81a6ef0 con 0x7fc7d81061b0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d770 tx=0x7fc7c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d57fa640 1 -- 172.21.15.33:0/2695010566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0017070 con 0x7fc7d81061b0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d57fa640 1 -- 172.21.15.33:0/2695010566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c0009c30 con 0x7fc7d81061b0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7d57fa640 1 -- 172.21.15.33:0/2695010566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000f620 con 0x7fc7d81061b0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d81a71e0 con 0x7fc7d81061b0 2024-04-23T18:32:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.386+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d81a7730 con 0x7fc7d81061b0 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.388+0000 7fc7d57fa640 1 -- 172.21.15.33:0/2695010566 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc7c001a070 con 0x7fc7d81061b0 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 Processor -- start 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 -- start start 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562209fe5970 con 0x56220a080620 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562209f824a0 con 0x56220a080f40 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562209fe1930 con 0x56220a08b800 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7dcc68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7dcc68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49474/0 (socket says 172.21.15.33:49474) 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7dcc68640 1 -- 172.21.15.33:0/669567941 learned_addr learned my addr 172.21.15.33:0/669567941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.389+0000 7fc7d57fa640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 0x7fc7c40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 -- 172.21.15.33:0/669567941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 msgr2=0x56220a08dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 -- 172.21.15.33:0/669567941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 msgr2=0x56220a08b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 -- 172.21.15.33:0/669567941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562209cf8210 con 0x56220a080620 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 0x7fc7c40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7d57fa640 1 -- 172.21.15.33:0/2695010566 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7c00d0320 con 0x7fc7d81061b0 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005cc0 tx=0x7fc7d0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7731f4b904af79e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7ce7fc640 1 -- 172.21.15.33:0/669567941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0003970 con 0x56220a080620 2024-04-23T18:32:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7ce7fc640 1 -- 172.21.15.33:0/669567941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d0036040 con 0x56220a080620 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7ce7fc640 1 -- 172.21.15.33:0/669567941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004840 con 0x56220a080620 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.390+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 0x7fc7c40a62b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7d004c000 tx=0x7fc7d0049000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- 172.21.15.33:0/669567941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 msgr2=0x56220a080a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0005cc0 tx=0x7fc7d0009880 comp rx=0 tx=0).stop 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- 172.21.15.33:0/669567941 shutdown_connections 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/669567941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a080a00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- 172.21.15.33:0/669567941 >> 172.21.15.33:0/669567941 conn(0x56220a07cb30 msgr2=0x56220a07cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- 172.21.15.33:0/669567941 shutdown_connections 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- 172.21.15.33:0/669567941 wait complete. 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.391+0000 7fc7e13d6740 1 -- start start 2024-04-23T18:32:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 Processor -- start 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080620 0x56220a08b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a088960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56220a08e2b0 con 0x56220a080f40 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562209f800b0 con 0x56220a080620 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56220a08e430 con 0x56220a08b800 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7e13d6740 10 client.0 _ll_register_callbacks cb 0x562209f6dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49484/0 (socket says 172.21.15.33:49484) 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2222999220 learned_addr learned my addr 172.21.15.33:0/2222999220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080620 0x56220a08b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a088960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2222999220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 msgr2=0x56220a088960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a088960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080620 msgr2=0x56220a08b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080620 0x56220a08b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 -- 172.21.15.33:0/2222999220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562209dc4620 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d740 tx=0x7fc7c00d3a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c189ec1985b367e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.392+0000 7fc7b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0021040 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7c00ccbc0 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d0036040 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7c009be10 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 10 client.16329 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56220a09a210 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d0036040 con 0x56220a080f40 2024-04-23T18:32:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7c00cd640 con 0x56220a080f40 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 10 client.16329 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 20 client.16329 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a09b680 0x56220a09da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 4 client.16329 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.393+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562209ff3390 con 0x56220a09b680 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.394+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a09b680 0x56220a09da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.394+0000 7fc7b77fe640 10 client.16329 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.394+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a09b680 0x56220a09da70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=359e28a86e86b481 server_cookie=69e1cdfba7aef20d in_seq=0 out_seq=0 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7b77fe640 1 -- 172.21.15.33:0/2222999220 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562209ff3390 con 0x56220a09b680 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7b77fe640 10 client.16329 handle_command_reply: tid=0 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 client.16329 shutdown 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 msgr2=0x56220a088420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d740 tx=0x7fc7c00d3a60 comp rx=0 tx=0).stop 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 shutdown_connections 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a09b680 0x56220a09da70 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a088960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080f40 0x56220a088420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7b77fe640 0 client.16329 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/2222999220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080620 0x56220a08b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 >> 172.21.15.33:0/2222999220 conn(0x56220a07cb30 msgr2=0x56220a0828b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 shutdown_connections 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 1 -- 172.21.15.33:0/2222999220 wait complete. 2024-04-23T18:32:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.395+0000 7fc7e13d6740 20 client.16329 trim_cache size 0 max 16384 2024-04-23T18:32:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.410+0000 7fc7e13d6740 1 Processor -- start 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.410+0000 7fc7e13d6740 1 -- start start 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 0x56220a083850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 0x56220a08ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81afcd0 con 0x56220a080620 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b1810 con 0x56220a08b800 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b03a0 con 0x56220a080f40 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49494/0 (socket says 172.21.15.33:49494) 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 -- 172.21.15.33:0/1863445255 learned_addr learned my addr 172.21.15.33:0/1863445255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 0x56220a083850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 0x56220a08ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 -- 172.21.15.33:0/1863445255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 msgr2=0x56220a083850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 0x56220a083850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 -- 172.21.15.33:0/1863445255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 msgr2=0x56220a08ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 0x56220a08ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 -- 172.21.15.33:0/1863445255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56220a086970 con 0x56220a080620 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 0x56220a083850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 0x56220a08ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800be40 tx=0x7fc7c8013b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22689f397bb919b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7cdffb640 1 -- 172.21.15.33:0/1863445255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c80040f0 con 0x56220a080620 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7cdffb640 1 -- 172.21.15.33:0/1863445255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7c801d920 con 0x56220a080620 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.411+0000 7fc7cdffb640 1 -- 172.21.15.33:0/1863445255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8003bf0 con 0x56220a080620 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- 172.21.15.33:0/1863445255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 msgr2=0x56220a097170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800be40 tx=0x7fc7c8013b00 comp rx=0 tx=0).stop 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- 172.21.15.33:0/1863445255 shutdown_connections 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a08b800 0x56220a08ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a080f40 0x56220a083850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/1863445255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a097170 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- 172.21.15.33:0/1863445255 >> 172.21.15.33:0/1863445255 conn(0x56220a07cb30 msgr2=0x56220a083140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- 172.21.15.33:0/1863445255 shutdown_connections 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- 172.21.15.33:0/1863445255 wait complete. 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.412+0000 7fc7e13d6740 1 -- start start 2024-04-23T18:32:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 Processor -- start 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b1140 con 0x56220a080620 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81b0a70 con 0x56220a080f40 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d81adcb0 con 0x56220a08b800 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7e13d6740 10 client.0 _ll_register_callbacks cb 0x7fc7c0010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49498/0 (socket says 172.21.15.33:49498) 2024-04-23T18:32:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 -- 172.21.15.33:0/16844001 learned_addr learned my addr 172.21.15.33:0/16844001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7dcc68640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 -- 172.21.15.33:0/16844001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 msgr2=0x56220a08f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 -- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 msgr2=0x56220a08ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d7fff640 1 -- 172.21.15.33:0/16844001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56220a08fc90 con 0x56220a080620 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.413+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7d7fff640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8015010 tx=0x7fc7c801d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623bf13d11c79fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8013360 con 0x56220a080620 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fc7c80040f0 con 0x56220a080620 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56220a088cf0 con 0x56220a080620 2024-04-23T18:32:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7c8037050 con 0x56220a080620 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 10 client.16335 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56220a101c10 con 0x56220a080620 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56220a088cf0 con 0x56220a080620 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7c8003a50 con 0x56220a080620 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 10 client.16335 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 20 client.16335 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a102320 0x56220a104740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 4 client.16335 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.414+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562209fed200 con 0x56220a102320 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.415+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a102320 0x56220a104740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.415+0000 7fc7d77fe640 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a102320 0x56220a104740 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=515abd0972f7db4b server_cookie=4cf242a636238b82 in_seq=0 out_seq=0 2024-04-23T18:32:01.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:01.415+0000 7fc7b77fe640 10 client.16335 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:03.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:02 smithi106 ceph-mon[32716]: pgmap v170: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7b77fe640 1 -- 172.21.15.33:0/16844001 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562209fed200 con 0x56220a102320 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7b77fe640 10 client.16335 handle_command_reply: tid=0 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 client.16335 shutdown 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 msgr2=0x56220a093b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8015010 tx=0x7fc7c801d710 comp rx=0 tx=0).stop 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 shutdown_connections 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56220a102320 0x56220a104740 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56220a08b800 0x56220a08f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56220a080f40 0x56220a08ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 --2- 172.21.15.33:0/16844001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56220a080620 0x56220a093b10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7b77fe640 0 client.16335 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 >> 172.21.15.33:0/16844001 conn(0x56220a07cb30 msgr2=0x56220a0839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 shutdown_connections 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.168+0000 7fc7e13d6740 1 -- 172.21.15.33:0/16844001 wait complete. 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7e13d6740 20 client.16335 trim_cache size 0 max 16384 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:32:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:32:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 msgr2=0x7fc7c40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 0x7fc7c40a62b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7d004c000 tx=0x7fc7d0049000 comp rx=0 tx=0).stop 2024-04-23T18:32:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 msgr2=0x7fc7d81030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000d770 tx=0x7fc7c000dc30 comp rx=0 tx=0).stop 2024-04-23T18:32:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 shutdown_connections 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc7c40a3df0 0x7fc7c40a62b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d81035f0 0x7fc7d81a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7d81061b0 0x7fc7d81030b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 --2- 172.21.15.33:0/2695010566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc7d81010e0 0x7fc7d8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 >> 172.21.15.33:0/2695010566 conn(0x7fc7d80771d0 msgr2=0x7fc7d806aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.169+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 shutdown_connections 2024-04-23T18:32:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:03.170+0000 7fc7de6cb640 1 -- 172.21.15.33:0/2695010566 wait complete. 2024-04-23T18:32:03.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:02 smithi122 ceph-mon[32546]: pgmap v170: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-04-23T18:32:03.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:02 smithi033 ceph-mon[29274]: pgmap v170: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 19 op/s 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: in thread 7f15223c4700 thread_name:md_log_replay 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1533785d20] 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55a48fc6a38f] 2024-04-23T18:32:03.699 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a48fc6db23] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 4: (MDLog::_replay_thread()+0x617) [0x55a48fcc8647] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a48fcd1d31] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f153377b1ca] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 7: clone() 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.357+0000 7f15223c4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: in thread 7f15223c4700 thread_name:md_log_replay 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1533785d20] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55a48fc6a38f] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a48fc6db23] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 4: (MDLog::_replay_thread()+0x617) [0x55a48fcc8647] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a48fcd1d31] 2024-04-23T18:32:03.700 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f153377b1ca] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 7: clone() 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 0> 2024-04-23T18:32:03.357+0000 7f15223c4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: in thread 7f15223c4700 thread_name:md_log_replay 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1533785d20] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55a48fc6a38f] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a48fc6db23] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 4: (MDLog::_replay_thread()+0x617) [0x55a48fcc8647] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a48fcd1d31] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f153377b1ca] 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 7: clone() 2024-04-23T18:32:03.701 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 0> 2024-04-23T18:32:03.357+0000 7f15223c4700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: in thread 7f15223c4700 thread_name:md_log_replay 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1533785d20] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55a48fc6a38f] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55a48fc6db23] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 4: (MDLog::_replay_thread()+0x617) [0x55a48fcc8647] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55a48fcd1d31] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f153377b1ca] 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 7: clone() 2024-04-23T18:32:03.702 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:32:03.703 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67297]: 2024-04-23T18:32:03.703 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:03 smithi122 podman[67732]: 2024-04-23 18:32:03.408296532 +0000 UTC m=+0.021510219 container died 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:32:04.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:04 smithi122 podman[67732]: 2024-04-23 18:32:04.013040539 +0000 UTC m=+0.626254215 container remove 9b32d94e083d7ac1019d41e30a0a882acb0369fdf71e0764421c178fa171ba60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:32:04.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:32:04.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:32:05.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:04 smithi106 ceph-mon[32716]: pgmap v171: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-04-23T18:32:05.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:04 smithi122 ceph-mon[32546]: pgmap v171: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-04-23T18:32:05.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:04 smithi033 ceph-mon[29274]: pgmap v171: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-04-23T18:32:07.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:06 smithi106 ceph-mon[32716]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-23T18:32:07.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:06 smithi122 ceph-mon[32546]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-23T18:32:07.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:06 smithi033 ceph-mon[29274]: pgmap v172: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-04-23T18:32:09.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:08 smithi033 ceph-mon[29274]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:09.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:08 smithi106 ceph-mon[32716]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:09.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:08 smithi122 ceph-mon[32546]: pgmap v173: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:11.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:10 smithi106 ceph-mon[32716]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:11.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:10 smithi122 ceph-mon[32546]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:11.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:10 smithi033 ceph-mon[29274]: pgmap v174: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 28 KiB/s wr, 25 op/s 2024-04-23T18:32:13.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:12 smithi106 ceph-mon[32716]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:13.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:12 smithi122 ceph-mon[32546]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:13.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:12 smithi033 ceph-mon[29274]: pgmap v175: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:13.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 -- 172.21.15.33:0/460953753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 msgr2=0x7f62d0101190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 --2- 172.21.15.33:0/460953753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d0101190 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009900 tx=0x7f62c002a410 comp rx=0 tx=0).stop 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 -- 172.21.15.33:0/460953753 shutdown_connections 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 --2- 172.21.15.33:0/460953753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d010e960 0x7f62d0110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 --2- 172.21.15.33:0/460953753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d01016d0 0x7f62d010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 --2- 172.21.15.33:0/460953753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d0101190 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 -- 172.21.15.33:0/460953753 >> 172.21.15.33:0/460953753 conn(0x7f62d0077fb0 msgr2=0x7f62d00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.092+0000 7f62d8020640 1 -- 172.21.15.33:0/460953753 shutdown_connections 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 -- 172.21.15.33:0/460953753 wait complete. 2024-04-23T18:32:14.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 Processor -- start 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 -- start start 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d01016d0 0x7f62d01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 0x7f62d019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0114460 con 0x7f62d0100db0 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01142e0 con 0x7f62d010e960 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d8020640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62d01145e0 con 0x7f62d01016d0 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.093+0000 7f62d5dbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52602/0 (socket says 172.21.15.33:52602) 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 -- 172.21.15.33:0/2366183574 learned_addr learned my addr 172.21.15.33:0/2366183574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d65bd640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 0x7f62d019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d55bb640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d01016d0 0x7f62d01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 -- 172.21.15.33:0/2366183574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d01016d0 msgr2=0x7f62d01a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d01016d0 0x7f62d01a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 -- 172.21.15.33:0/2366183574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 msgr2=0x7f62d019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 0x7f62d019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 -- 172.21.15.33:0/2366183574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d019afb0 con 0x7f62d0100db0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d65bd640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 0x7f62d019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d5dbc640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f62c0005ab0 tx=0x7f62c0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003f070 con 0x7f62d0100db0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62c0031d60 con 0x7f62d0100db0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035030 con 0x7f62d0100db0 2024-04-23T18:32:14.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d019b240 con 0x7f62d0100db0 2024-04-23T18:32:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.094+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d01a74c0 con 0x7f62d0100db0 2024-04-23T18:32:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.095+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d006ad80 con 0x7f62d0100db0 2024-04-23T18:32:14.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.097+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f62c0042070 con 0x7f62d0100db0 2024-04-23T18:32:14.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.097+0000 7f62c6ffd640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 0x7f62bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.097+0000 7f62d55bb640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 0x7f62bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.097+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f62c00f1790 con 0x7f62d0100db0 2024-04-23T18:32:14.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.098+0000 7f62d55bb640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 0x7f62bc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005e10 tx=0x7f62cc005da0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.101+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62c00bde40 con 0x7f62d0100db0 2024-04-23T18:32:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.329+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62d0109af0 con 0x7f62d0100db0 2024-04-23T18:32:14.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.330+0000 7f62c6ffd640 1 -- 172.21.15.33:0/2366183574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11926 (secure 0 0 0) 0x7f62c00bd740 con 0x7f62d0100db0 2024-04-23T18:32:14.323 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:32:14.323 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:30:16.151605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25696":{"gid":25696,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/111505116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":111505116},{"type":"v1","addr":"172.21.15.122:6835","nonce":111505116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:32:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.332+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 msgr2=0x7f62bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.332+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 0x7f62bc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f62cc005e10 tx=0x7f62cc005da0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.332+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 msgr2=0x7f62d01a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.332+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f62c0005ab0 tx=0x7f62c0003cd0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 shutdown_connections 2024-04-23T18:32:14.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f62bc0a3d20 0x7f62bc0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62d010e960 0x7f62d019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d01016d0 0x7f62d01a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 --2- 172.21.15.33:0/2366183574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62d0100db0 0x7f62d01a0670 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 >> 172.21.15.33:0/2366183574 conn(0x7f62d0077fb0 msgr2=0x7f62d010fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 shutdown_connections 2024-04-23T18:32:14.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.333+0000 7f62d8020640 1 -- 172.21.15.33:0/2366183574 wait complete. 2024-04-23T18:32:14.336 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 7da0f1de --release --await' 2024-04-23T18:32:14.336 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 7da0f1de --release --await 2024-04-23T18:32:14.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 -- 172.21.15.33:0/3731707436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8102190 msgr2=0x7f35e8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 --2- 172.21.15.33:0/3731707436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8102190 0x7f35e8102570 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009900 tx=0x7f35dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 -- 172.21.15.33:0/3731707436 shutdown_connections 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 --2- 172.21.15.33:0/3731707436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8107260 0x7f35e810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 --2- 172.21.15.33:0/3731707436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e8102b40 0x7f35e8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 --2- 172.21.15.33:0/3731707436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8102190 0x7f35e8102570 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 -- 172.21.15.33:0/3731707436 >> 172.21.15.33:0/3731707436 conn(0x7f35e80793a0 msgr2=0x7f35e80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 -- 172.21.15.33:0/3731707436 shutdown_connections 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.454+0000 7f35eee59640 1 -- 172.21.15.33:0/3731707436 wait complete. 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 Processor -- start 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 -- start start 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8102190 0x7f35e8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e819d010 0x7f35e819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81121d0 con 0x7f35e8107260 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8112050 con 0x7f35e819d010 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35eee59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8112350 con 0x7f35e8102190 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52614/0 (socket says 172.21.15.33:52614) 2024-04-23T18:32:14.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 -- 172.21.15.33:0/2062288932 learned_addr learned my addr 172.21.15.33:0/2062288932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e819d010 0x7f35e819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8102190 0x7f35e8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 -- 172.21.15.33:0/2062288932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8102190 msgr2=0x7f35e8198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8102190 0x7f35e8198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 -- 172.21.15.33:0/2062288932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e819d010 msgr2=0x7f35e819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e819d010 0x7f35e819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 -- 172.21.15.33:0/2062288932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819db00 con 0x7f35e8107260 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e7fff640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f35d400e9c0 tx=0x7f35d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.455+0000 7f35e5ffb640 1 -- 172.21.15.33:0/2062288932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d400cd80 con 0x7f35e8107260 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.456+0000 7f35e5ffb640 1 -- 172.21.15.33:0/2062288932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d40054d0 con 0x7f35e8107260 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.456+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819ddf0 con 0x7f35e8107260 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.456+0000 7f35e5ffb640 1 -- 172.21.15.33:0/2062288932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4010430 con 0x7f35e8107260 2024-04-23T18:32:14.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.456+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a5630 con 0x7f35e8107260 2024-04-23T18:32:14.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e5ffb640 1 -- 172.21.15.33:0/2062288932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f35d4004fd0 con 0x7f35e8107260 2024-04-23T18:32:14.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 Processor -- start 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 -- start start 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee84c6d0 con 0x55d2ee8f2480 2024-04-23T18:32:14.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee7e91a0 con 0x55d2ee8e7490 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee848690 con 0x55d2ee8e7db0 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52626/0 (socket says 172.21.15.33:52626) 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 -- 172.21.15.33:0/3155639007 learned_addr learned my addr 172.21.15.33:0/3155639007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e5ffb640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 0x7f35d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 -- 172.21.15.33:0/3155639007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7db0 msgr2=0x55d2ee8f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 -- 172.21.15.33:0/3155639007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 msgr2=0x55d2ee8e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.459+0000 7f35e7fff640 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35e7fff640 1 -- 172.21.15.33:0/3155639007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ee7f35b0 con 0x55d2ee8f2480 2024-04-23T18:32:14.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 0x7f35d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35e5ffb640 1 -- 172.21.15.33:0/2062288932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f35d40d0cb0 con 0x7f35e8107260 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35e7fff640 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f35d40ce380 tx=0x7f35d4067260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b987f7e0354bf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35d2ffd640 1 -- 172.21.15.33:0/3155639007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d409c7a0 con 0x55d2ee8f2480 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35d2ffd640 1 -- 172.21.15.33:0/3155639007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35d40046c0 con 0x55d2ee8f2480 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.460+0000 7f35d2ffd640 1 -- 172.21.15.33:0/3155639007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40683a0 con 0x55d2ee8f2480 2024-04-23T18:32:14.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- 172.21.15.33:0/3155639007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 msgr2=0x55d2ee8f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 0x7f35d80a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35e0005fd0 tx=0x7f35e0004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f35d40ce380 tx=0x7f35d4067260 comp rx=0 tx=0).stop 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- 172.21.15.33:0/3155639007 shutdown_connections 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f4870 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/3155639007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- 172.21.15.33:0/3155639007 >> 172.21.15.33:0/3155639007 conn(0x55d2ee8e37b0 msgr2=0x55d2ee8e3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- 172.21.15.33:0/3155639007 shutdown_connections 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- 172.21.15.33:0/3155639007 wait complete. 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- start start 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 Processor -- start 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8ef170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.461+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee8f52b0 con 0x55d2ee8f2480 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee7e6b20 con 0x55d2ee8e7db0 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d2ee8f5430 con 0x55d2ee8e7490 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 10 client.0 _ll_register_callbacks cb 0x55d2ee7e6530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52634/0 (socket says 172.21.15.33:52634) 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 -- 172.21.15.33:0/2751720867 learned_addr learned my addr 172.21.15.33:0/2751720867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35e7fff640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8ef170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 -- 172.21.15.33:0/2751720867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 msgr2=0x55d2ee8f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 -- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 msgr2=0x55d2ee8ef170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8ef170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 -- 172.21.15.33:0/2751720867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ee55e920 con 0x55d2ee8f2480 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f35e0012880 tx=0x7f35e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9efcb40d4557da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e001da70 con 0x55d2ee8f2480 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f35e001dc10 con 0x55d2ee8f2480 2024-04-23T18:32:14.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2ee80a500 con 0x55d2ee8f2480 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35e001de10 con 0x55d2ee8f2480 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 10 client.16353 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.462+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2ee62ad30 con 0x55d2ee8f2480 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d2ee80a500 con 0x55d2ee8f2480 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f35e000f040 con 0x55d2ee8f2480 2024-04-23T18:32:14.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 10 client.16353 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 10 client.16353 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 20 client.16353 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9025f0 0x55d2ee904a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 4 client.16353 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2ee85a0f0 con 0x55d2ee9025f0 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.463+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9025f0 0x55d2ee904a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9025f0 0x55d2ee904a00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1bebe495c8615cb server_cookie=5e5f7ca6a464e4ad in_seq=0 out_seq=0 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35c7fff640 10 client.16353 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35c7fff640 1 -- 172.21.15.33:0/2751720867 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2ee85a0f0 con 0x55d2ee9025f0 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35c7fff640 10 client.16353 handle_command_reply: tid=0 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 client.16353 shutdown 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 msgr2=0x55d2ee8ef780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f35e0012880 tx=0x7f35e0004ab0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 shutdown_connections 2024-04-23T18:32:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9025f0 0x55d2ee904a00 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee8ef780 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8ef170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/2751720867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 >> 172.21.15.33:0/2751720867 conn(0x55d2ee8e37b0 msgr2=0x55d2ee8e95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.464+0000 7f35c7fff640 0 client.16353 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.465+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 shutdown_connections 2024-04-23T18:32:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.465+0000 7f35f1b6a740 1 -- 172.21.15.33:0/2751720867 wait complete. 2024-04-23T18:32:14.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.465+0000 7f35f1b6a740 20 client.16353 trim_cache size 0 max 16384 2024-04-23T18:32:14.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 Processor -- start 2024-04-23T18:32:14.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 -- start start 2024-04-23T18:32:14.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81adc30 con 0x55d2ee8f2480 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81af770 con 0x55d2ee8e7db0 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ae300 con 0x55d2ee8e7490 2024-04-23T18:32:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35ed3f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35ed3f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52642/0 (socket says 172.21.15.33:52642) 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35ed3f6640 1 -- 172.21.15.33:0/71973372 learned_addr learned my addr 172.21.15.33:0/71973372 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.482+0000 7f35e7fff640 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 -- 172.21.15.33:0/71973372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 msgr2=0x55d2ee8fe170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8fe170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 -- 172.21.15.33:0/71973372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 msgr2=0x55d2ee8f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 -- 172.21.15.33:0/71973372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ee8e7be0 con 0x55d2ee8f2480 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f35e001e730 tx=0x7f35e0005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3fb0a78bd27c00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35d27fc640 1 -- 172.21.15.33:0/71973372 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0034040 con 0x55d2ee8f2480 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35d27fc640 1 -- 172.21.15.33:0/71973372 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e0022070 con 0x55d2ee8f2480 2024-04-23T18:32:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35d27fc640 1 -- 172.21.15.33:0/71973372 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0027b80 con 0x55d2ee8f2480 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.483+0000 7f35f1b6a740 1 -- 172.21.15.33:0/71973372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 msgr2=0x55d2ee90a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f35e001e730 tx=0x7f35e0005c30 comp rx=0 tx=0).stop 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- 172.21.15.33:0/71973372 shutdown_connections 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8f2480 0x55d2ee90a2d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/71973372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8e7490 0x55d2ee8fe170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- 172.21.15.33:0/71973372 >> 172.21.15.33:0/71973372 conn(0x55d2ee8e37b0 msgr2=0x55d2ee8ea520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- 172.21.15.33:0/71973372 shutdown_connections 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- 172.21.15.33:0/71973372 wait complete. 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- start start 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 Processor -- start 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81af0a0 con 0x55d2ee8e7db0 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81ae9d0 con 0x55d2ee8e7490 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81abc10 con 0x55d2ee8f2480 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35f1b6a740 10 client.0 _ll_register_callbacks cb 0x7f35e0012550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.484+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52652/0 (socket says 172.21.15.33:52652) 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 -- 172.21.15.33:0/823495042 learned_addr learned my addr 172.21.15.33:0/823495042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 -- 172.21.15.33:0/823495042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 msgr2=0x55d2ee8f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 -- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 msgr2=0x55d2ee8f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 -- 172.21.15.33:0/823495042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ee8f9190 con 0x55d2ee8e7db0 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35ed3f6640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35e7fff640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f35d409cba0 tx=0x7f35d400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88d716299eae80e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d406c5b0 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f35d409af60 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e0022070 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f35d409c5c0 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35f1b6a740 10 client.16359 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2ee969300 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.485+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e0022070 con 0x55d2ee8e7db0 2024-04-23T18:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f35d40992d0 con 0x55d2ee8e7db0 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 10 client.16359 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 10 client.16359 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 20 client.16359 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9696f0 0x55d2ee96bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 4 client.16359 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7da0f1de", "release": true, "await": true}] 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7da0f1de", "release": true, "await": true}) -- 0x55d2ee853f60 con 0x55d2ee9696f0 2024-04-23T18:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9696f0 0x55d2ee96bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35ecbf5640 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9696f0 0x55d2ee96bae0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a46a2d1bdbc8d249 server_cookie=2ad040b3714f3d9a in_seq=0 out_seq=0 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.486+0000 7f35c7fff640 10 client.16359 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35c7fff640 1 -- 172.21.15.33:0/823495042 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d2ee853f60 con 0x55d2ee9696f0 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35c7fff640 10 client.16359 handle_command_reply: tid=0 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 client.16359 shutdown 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 msgr2=0x55d2ee8f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f35d409cba0 tx=0x7f35d400efb0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 shutdown_connections 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d2ee9696f0 0x55d2ee96bae0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d2ee8f2480 0x55d2ee8f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2ee8e7db0 0x55d2ee8f3bb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 --2- 172.21.15.33:0/823495042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d2ee8e7490 0x55d2ee8f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35c7fff640 0 client.16359 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:14.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 >> 172.21.15.33:0/823495042 conn(0x55d2ee8e37b0 msgr2=0x55d2ee8e2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 shutdown_connections 2024-04-23T18:32:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 1 -- 172.21.15.33:0/823495042 wait complete. 2024-04-23T18:32:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.489+0000 7f35f1b6a740 20 client.16359 trim_cache size 0 max 16384 2024-04-23T18:32:14.505 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 32, 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 20, 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "7da0f1de": { 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 20, 2024-04-23T18:32:14.506 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 78.8, 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:32:14.507 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:14.508 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:32:14.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 msgr2=0x7f35d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 0x7f35d80a6260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35e0005fd0 tx=0x7f35e0004a40 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 msgr2=0x7f35e8198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f35d400e9c0 tx=0x7f35d400ee80 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 shutdown_connections 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35d80a3da0 0x7f35d80a6260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35e819d010 0x7f35e819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35e8107260 0x7f35e8198c90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 --2- 172.21.15.33:0/2062288932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35e8102190 0x7f35e8198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 >> 172.21.15.33:0/2062288932 conn(0x7f35e80793a0 msgr2=0x7f35e80fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 shutdown_connections 2024-04-23T18:32:14.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.490+0000 7f35eee59640 1 -- 172.21.15.33:0/2062288932 wait complete. 2024-04-23T18:32:14.511 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7da0f1de', seconds elapsed: 19.9 2024-04-23T18:32:14.511 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.8 2024-04-23T18:32:14.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:32:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 -- 172.21.15.33:0/978029362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103230 msgr2=0x7f1f70103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 --2- 172.21.15.33:0/978029362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103230 0x7f1f70103610 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c02a480 comp rx=0 tx=0).stop 2024-04-23T18:32:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 -- 172.21.15.33:0/978029362 shutdown_connections 2024-04-23T18:32:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 --2- 172.21.15.33:0/978029362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f7010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 --2- 172.21.15.33:0/978029362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103b50 0x7f1f7010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 --2- 172.21.15.33:0/978029362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103230 0x7f1f70103610 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 -- 172.21.15.33:0/978029362 >> 172.21.15.33:0/978029362 conn(0x7f1f70077fb0 msgr2=0x7f1f700783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 -- 172.21.15.33:0/978029362 shutdown_connections 2024-04-23T18:32:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 -- 172.21.15.33:0/978029362 wait complete. 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.633+0000 7f1f784cb640 1 Processor -- start 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 -- start start 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103230 0x7f1f7019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f701985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70112280 con 0x7f1f70103b50 2024-04-23T18:32:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70112100 con 0x7f1f7010c570 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70112400 con 0x7f1f70103230 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52674/0 (socket says 172.21.15.33:52674) 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 -- 172.21.15.33:0/3605629651 learned_addr learned my addr 172.21.15.33:0/3605629651 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f76a68640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f701985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f76267640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103230 0x7f1f7019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 -- 172.21.15.33:0/3605629651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103230 msgr2=0x7f1f7019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103230 0x7f1f7019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 -- 172.21.15.33:0/3605629651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 msgr2=0x7f1f701985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f701985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 -- 172.21.15.33:0/3605629651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70198d60 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f76a68640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f701985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f75a66640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b4a0 tx=0x7f1f6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60014070 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f60010ac0 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.634+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f70199050 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.635+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60017c30 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.635+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f701a52f0 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.635+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f700691c0 con 0x7f1f70103b50 2024-04-23T18:32:14.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.637+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f60017070 con 0x7f1f70103b50 2024-04-23T18:32:14.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.637+0000 7f1f677fe640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 0x7f1f5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.637+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1f600cfe70 con 0x7f1f70103b50 2024-04-23T18:32:14.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.637+0000 7f1f76267640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 0x7f1f5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.638+0000 7f1f76267640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 0x7f1f5c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c031400 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.641+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f6009c630 con 0x7f1f70103b50 2024-04-23T18:32:14.722 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 2. 2024-04-23T18:32:14.722 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:32:14.722 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:32:14.722 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 podman[67826]: 2024-04-23 18:32:14.59541203 +0000 UTC m=+0.018279532 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:32:14.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.869+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f70107750 con 0x7f1f70103b50 2024-04-23T18:32:14.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.870+0000 7f1f677fe640 1 -- 172.21.15.33:0/3605629651 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11926 (secure 0 0 0) 0x7f1f6009bf30 con 0x7f1f70103b50 2024-04-23T18:32:14.862 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:32:14.862 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:30:16.151605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25696":{"gid":25696,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/111505116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":111505116},{"type":"v1","addr":"172.21.15.122:6835","nonce":111505116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24466,24515,24482,24490,24551]},"id":1}]} 2024-04-23T18:32:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-04-23T18:32:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 msgr2=0x7f1f5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 0x7f1f5c0a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c031400 comp rx=0 tx=0).stop 2024-04-23T18:32:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 msgr2=0x7f1f7019e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000b4a0 tx=0x7f1f6000b960 comp rx=0 tx=0).stop 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 shutdown_connections 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1f5c0a3d20 0x7f1f5c0a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7010c570 0x7f1f701985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f70103b50 0x7f1f7019e970 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 --2- 172.21.15.33:0/3605629651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f70103230 0x7f1f7019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 >> 172.21.15.33:0/3605629651 conn(0x7f1f70077fb0 msgr2=0x7f1f700fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 shutdown_connections 2024-04-23T18:32:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.872+0000 7f1f784cb640 1 -- 172.21.15.33:0/3605629651 wait complete. 2024-04-23T18:32:14.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0' 2024-04-23T18:32:14.875 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 78.8 --await-for 42.0 2024-04-23T18:32:14.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:14 smithi122 ceph-mon[32546]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-23T18:32:14.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:14 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2366183574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:14.975 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 podman[67826]: 2024-04-23 18:32:14.730935734 +0000 UTC m=+0.153803213 container create 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T18:32:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 -- 172.21.15.33:0/859854514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a60102240 msgr2=0x7f1a60102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 --2- 172.21.15.33:0/859854514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a60102240 0x7f1a60102620 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 -- 172.21.15.33:0/859854514 shutdown_connections 2024-04-23T18:32:14.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 --2- 172.21.15.33:0/859854514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 0x7f1a6010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 --2- 172.21.15.33:0/859854514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102bf0 0x7f1a60106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 --2- 172.21.15.33:0/859854514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a60102240 0x7f1a60102620 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 -- 172.21.15.33:0/859854514 >> 172.21.15.33:0/859854514 conn(0x7f1a600790e0 msgr2=0x7f1a600794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 -- 172.21.15.33:0/859854514 shutdown_connections 2024-04-23T18:32:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.995+0000 7f1a67613640 1 -- 172.21.15.33:0/859854514 wait complete. 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 Processor -- start 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 -- start start 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102240 0x7f1a60198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 0x7f1a60198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60112210 con 0x7f1a6019d010 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60112090 con 0x7f1a60102240 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a67613640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a60112390 con 0x7f1a60107280 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52696/0 (socket says 172.21.15.33:52696) 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 -- 172.21.15.33:0/577575443 learned_addr learned my addr 172.21.15.33:0/577575443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a653af640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102240 0x7f1a60198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a64bae640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 0x7f1a60198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 -- 172.21.15.33:0/577575443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 msgr2=0x7f1a60198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 0x7f1a60198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 -- 172.21.15.33:0/577575443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102240 msgr2=0x7f1a60198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102240 0x7f1a60198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 -- 172.21.15.33:0/577575443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6019db00 con 0x7f1a6019d010 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.996+0000 7f1a65bb0640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800eb10 tx=0x7f1a5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.997+0000 7f1a567fc640 1 -- 172.21.15.33:0/577575443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004b20 con 0x7f1a6019d010 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.997+0000 7f1a567fc640 1 -- 172.21.15.33:0/577575443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a58004cc0 con 0x7f1a6019d010 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.997+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6019ddf0 con 0x7f1a6019d010 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.997+0000 7f1a567fc640 1 -- 172.21.15.33:0/577575443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58010670 con 0x7f1a6019d010 2024-04-23T18:32:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.997+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a601a55e0 con 0x7f1a6019d010 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a567fc640 1 -- 172.21.15.33:0/577575443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a58005020 con 0x7f1a6019d010 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 Processor -- start 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 -- start start 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 0x559fad2ba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fad21f370 con 0x559fad2ba9c0 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559fad1bc2e0 con 0x559fad2ba0a0 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:14.999+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559fad21b330 con 0x559fad2c5090 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a567fc640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 0x7f1a4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52710/0 (socket says 172.21.15.33:52710) 2024-04-23T18:32:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 -- 172.21.15.33:0/132640338 learned_addr learned my addr 172.21.15.33:0/132640338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a64bae640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a65bb0640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 0x7f1a4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a64bae640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 0x559fad2ba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a567fc640 1 -- 172.21.15.33:0/577575443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a580d0cd0 con 0x7f1a6019d010 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 -- 172.21.15.33:0/132640338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 msgr2=0x559fad2c7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 -- 172.21.15.33:0/132640338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 msgr2=0x559fad2ba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 0x559fad2ba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 -- 172.21.15.33:0/132640338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559facffdd80 con 0x559fad2ba9c0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a64bae640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 0x559fad2ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a653af640 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6079f7484d083ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a65bb0640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 0x7f1a4c0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a5809c980 tx=0x7f1a58000f40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a4b7fe640 1 -- 172.21.15.33:0/132640338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c040070 con 0x559fad2ba9c0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a4b7fe640 1 -- 172.21.15.33:0/132640338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c004030 con 0x559fad2ba9c0 2024-04-23T18:32:14.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.000+0000 7f1a4b7fe640 1 -- 172.21.15.33:0/132640338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c035040 con 0x559fad2ba9c0 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- 172.21.15.33:0/132640338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 msgr2=0x559fad2c4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c032bd0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- 172.21.15.33:0/132640338 shutdown_connections 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2c4b50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/132640338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba0a0 0x559fad2ba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- 172.21.15.33:0/132640338 >> 172.21.15.33:0/132640338 conn(0x559fad2b63c0 msgr2=0x559fad2b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- 172.21.15.33:0/132640338 shutdown_connections 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- 172.21.15.33:0/132640338 wait complete. 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.001+0000 7f1a6a40a740 1 -- start start 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 Processor -- start 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba9c0 0x559fad2c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fad2c7e20 con 0x559fad2ba0a0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559fad1b9fb0 con 0x559fad2ba9c0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559fad2c7fa0 con 0x559fad2c5090 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a6a40a740 10 client.0 _ll_register_callbacks cb 0x559fad1b99a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52714/0 (socket says 172.21.15.33:52714) 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 -- 172.21.15.33:0/2139899330 learned_addr learned my addr 172.21.15.33:0/2139899330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 -- 172.21.15.33:0/2139899330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 msgr2=0x559fad2c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 -- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba9c0 msgr2=0x559fad2c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a653af640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba9c0 0x559fad2c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba9c0 0x559fad2c2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 -- 172.21.15.33:0/2139899330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fad05e510 con 0x559fad2ba0a0 2024-04-23T18:32:15.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a64bae640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a60103160 tx=0x7f1a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9d0078baa3e64f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.002+0000 7f1a48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50005220 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a500053c0 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a5c004030 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a50027050 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 10 client.16377 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559facef1670 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a5c004030 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1a500107d0 con 0x559fad2ba0a0 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 10 client.16377 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 10 client.16377 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 20 client.16377 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad2d7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 4 client.16377 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.003+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fad22cd90 con 0x559fad2d53c0 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.004+0000 7f1a653af640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad2d7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.004+0000 7f1a653af640 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad2d7820 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b8e92106c30450 server_cookie=87613f10656402ae in_seq=0 out_seq=0 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.004+0000 7f1a48ff9640 10 client.16377 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a48ff9640 1 -- 172.21.15.33:0/2139899330 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fad22cd90 con 0x559fad2d53c0 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a48ff9640 10 client.16377 handle_command_reply: tid=0 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 client.16377 shutdown 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 msgr2=0x559fad2c2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a60103160 tx=0x7f1a500046c0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 shutdown_connections 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad2d7820 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2c5090 0x559fad2c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2ba9c0 0x559fad2c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a48ff9640 0 client.16377 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:15.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/2139899330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2010 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 >> 172.21.15.33:0/2139899330 conn(0x559fad2b63c0 msgr2=0x559fad2b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 shutdown_connections 2024-04-23T18:32:15.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 1 -- 172.21.15.33:0/2139899330 wait complete. 2024-04-23T18:32:15.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.005+0000 7f1a6a40a740 20 client.16377 trim_cache size 0 max 16384 2024-04-23T18:32:15.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 Processor -- start 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 -- start start 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba9c0 0x559fad2d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601adc60 con 0x559fad2ba0a0 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601af7a0 con 0x559fad2c5090 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601ae330 con 0x559fad2ba9c0 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a64bae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a64bae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52722/0 (socket says 172.21.15.33:52722) 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.023+0000 7f1a64bae640 1 -- 172.21.15.33:0/684440350 learned_addr learned my addr 172.21.15.33:0/684440350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a653af640 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba9c0 0x559fad2d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 -- 172.21.15.33:0/684440350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba9c0 msgr2=0x559fad2d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba9c0 0x559fad2d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 -- 172.21.15.33:0/684440350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 msgr2=0x559fad2c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 -- 172.21.15.33:0/684440350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fad2baec0 con 0x559fad2ba0a0 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a64bae640 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000bcd0 tx=0x7f1a5000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36032431386e8583 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a4affd640 1 -- 172.21.15.33:0/684440350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50030070 con 0x559fad2ba0a0 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a4affd640 1 -- 172.21.15.33:0/684440350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50010840 con 0x559fad2ba0a0 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.024+0000 7f1a4affd640 1 -- 172.21.15.33:0/684440350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50004b40 con 0x559fad2ba0a0 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- 172.21.15.33:0/684440350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 msgr2=0x559fad2c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000bcd0 tx=0x7f1a5000b410 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- 172.21.15.33:0/684440350 shutdown_connections 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba9c0 0x559fad2d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/684440350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba0a0 0x559fad2c2640 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- 172.21.15.33:0/684440350 >> 172.21.15.33:0/684440350 conn(0x559fad2b63c0 msgr2=0x559fad2bc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- 172.21.15.33:0/684440350 shutdown_connections 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- 172.21.15.33:0/684440350 wait complete. 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 -- start start 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.025+0000 7f1a6a40a740 1 Processor -- start 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba0a0 0x559fad2cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601af0d0 con 0x559fad2ba9c0 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601aea00 con 0x559fad2c5090 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a601abc40 con 0x559fad2ba0a0 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 10 client.0 _ll_register_callbacks cb 0x7f1a50004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52730/0 (socket says 172.21.15.33:52730) 2024-04-23T18:32:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a64bae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba0a0 0x559fad2cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 -- 172.21.15.33:0/1160530111 learned_addr learned my addr 172.21.15.33:0/1160530111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a64bae640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 -- 172.21.15.33:0/1160530111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba0a0 msgr2=0x559fad2cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba0a0 0x559fad2cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 -- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 msgr2=0x559fad2ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 -- 172.21.15.33:0/1160530111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fad2ca890 con 0x559fad2ba9c0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a653af640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0051c0 tx=0x7f1a5c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce34af5f10fbb277 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c040050 con 0x559fad2ba9c0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f1a5c0314a0 con 0x559fad2ba9c0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.026+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a50010840 con 0x559fad2ba9c0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a5c035040 con 0x559fad2ba9c0 2024-04-23T18:32:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 10 client.16389 fetch_fsmap learned FSMap version 32 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559fad33c320 con 0x559fad2ba9c0 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50010840 con 0x559fad2ba9c0 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1a5c04e450 con 0x559fad2ba9c0 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 10 client.16389 fetch_fsmap finished waiting for FSMap version 32 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 10 client.16389 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 20 client.16389 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad33e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 4 client.16389 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}] 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 78.8, "await_for": 42.0}) -- 0x559fad226c00 con 0x559fad2d53c0 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.027+0000 7f1a64bae640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad33e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.028+0000 7f1a48ff9640 10 client.16389 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.028+0000 7f1a64bae640 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad33e7a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee17f8ea9751b9a5 server_cookie=15671b1b72ad400e in_seq=0 out_seq=0 2024-04-23T18:32:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.041+0000 7f1a48ff9640 1 -- 172.21.15.33:0/1160530111 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559fad226c00 con 0x559fad2d53c0 2024-04-23T18:32:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.041+0000 7f1a48ff9640 10 client.16389 handle_command_reply: tid=0 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.041+0000 7f1a6a40a740 1 client.16389 shutdown 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 msgr2=0x559fad2cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0051c0 tx=0x7f1a5c031400 comp rx=0 tx=0).stop 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 shutdown_connections 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fad2d53c0 0x559fad33e7a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fad2c5090 0x559fad2ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fad2ba9c0 0x559fad2cd760 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 --2- 172.21.15.33:0/1160530111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fad2ba0a0 0x559fad2cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a48ff9640 0 client.16389 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 >> 172.21.15.33:0/1160530111 conn(0x559fad2b63c0 msgr2=0x559fad2c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 shutdown_connections 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 1 -- 172.21.15.33:0/1160530111 wait complete. 2024-04-23T18:32:15.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a6a40a740 20 client.16389 trim_cache size 0 max 16384 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 32, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 22, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "70580dcf": { 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 22, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:32:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 78.8, 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:15.037 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 msgr2=0x7f1a4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 0x7f1a4c0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1a5809c980 tx=0x7f1a58000f40 comp rx=0 tx=0).stop 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 msgr2=0x7f1a6019d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.042+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800eb10 tx=0x7f1a5800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 shutdown_connections 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1a4c0a3d20 0x7f1a4c0a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a6019d010 0x7f1a6019d4c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a60107280 0x7f1a60198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 --2- 172.21.15.33:0/577575443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a60102240 0x7f1a60198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 >> 172.21.15.33:0/577575443 conn(0x7f1a600790e0 msgr2=0x7f1a601003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 shutdown_connections 2024-04-23T18:32:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:15.043+0000 7f1a67613640 1 -- 172.21.15.33:0/577575443 wait complete. 2024-04-23T18:32:15.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:14 smithi033 ceph-mon[29274]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-23T18:32:15.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:14 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2366183574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:15.045 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70580dcf', quiesce took 0.5 seconds. Will release after: 18.8 2024-04-23T18:32:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:14 smithi106 ceph-mon[32716]: pgmap v176: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 26 KiB/s wr, 24 op/s 2024-04-23T18:32:15.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:14 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2366183574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 podman[67826]: 2024-04-23 18:32:14.984057121 +0000 UTC m=+0.406924601 container init 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:14 smithi122 podman[67826]: 2024-04-23 18:32:14.986600883 +0000 UTC m=+0.409468362 container start 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mds[67845]: -- 172.21.15.122:0/2722737749 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557cba37da40 con 0x557cba350800 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: starting mds.f at 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:15 smithi122 bash[67826]: 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 2024-04-23T18:32:15.448 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:32:15 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3605629651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: daemon mds.f restarted 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/1758592712,v1:172.21.15.122:6835/1758592712] up:boot 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:32:16.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:32:16.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3605629651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: daemon mds.f restarted 2024-04-23T18:32:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:32:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/1758592712,v1:172.21.15.122:6835/1758592712] up:boot 2024-04-23T18:32:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:32:16.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:32:16.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3605629651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:16.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: daemon mds.f restarted 2024-04-23T18:32:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: osdmap e90: 12 total, 12 up, 12 in 2024-04-23T18:32:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/1758592712,v1:172.21.15.122:6835/1758592712] up:boot 2024-04-23T18:32:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:32:16.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:32:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- 172.21.15.33:0/164225790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c102b50 msgr2=0x7f5d2c106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 --2- 172.21.15.33:0/164225790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c102b50 0x7f5d2c106b90 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009900 tx=0x7f5d1c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- 172.21.15.33:0/164225790 shutdown_connections 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 --2- 172.21.15.33:0/164225790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c107270 0x7f5d2c10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 --2- 172.21.15.33:0/164225790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c102b50 0x7f5d2c106b90 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 --2- 172.21.15.33:0/164225790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c1021a0 0x7f5d2c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- 172.21.15.33:0/164225790 >> 172.21.15.33:0/164225790 conn(0x7f5d2c0793e0 msgr2=0x7f5d2c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- 172.21.15.33:0/164225790 shutdown_connections 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- 172.21.15.33:0/164225790 wait complete. 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 Processor -- start 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.273+0000 7f5d343f0640 1 -- start start 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c1021a0 0x7f5d2c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 0x7f5d2c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1121e0 con 0x7f5d2c107270 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c112060 con 0x7f5d2c19cf50 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d343f0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c112360 con 0x7f5d2c1021a0 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52748/0 (socket says 172.21.15.33:52748) 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 -- 172.21.15.33:0/451140250 learned_addr learned my addr 172.21.15.33:0/451140250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3298d640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 0x7f5d2c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3218c640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c1021a0 0x7f5d2c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 -- 172.21.15.33:0/451140250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c1021a0 msgr2=0x7f5d2c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c1021a0 0x7f5d2c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 -- 172.21.15.33:0/451140250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 msgr2=0x7f5d2c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 0x7f5d2c19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 -- 172.21.15.33:0/451140250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2c19da40 con 0x7f5d2c107270 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3298d640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 0x7f5d2c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:16.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.274+0000 7f5d3198b640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.275+0000 7f5d237fe640 1 -- 172.21.15.33:0/451140250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c03f070 con 0x7f5d2c107270 2024-04-23T18:32:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.275+0000 7f5d237fe640 1 -- 172.21.15.33:0/451140250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c031d60 con 0x7f5d2c107270 2024-04-23T18:32:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.275+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2c19dcd0 con 0x7f5d2c107270 2024-04-23T18:32:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.275+0000 7f5d237fe640 1 -- 172.21.15.33:0/451140250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c035030 con 0x7f5d2c107270 2024-04-23T18:32:16.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.275+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2c1a5490 con 0x7f5d2c107270 2024-04-23T18:32:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.277+0000 7f5d237fe640 1 -- 172.21.15.33:0/451140250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5d1c042070 con 0x7f5d2c107270 2024-04-23T18:32:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.277+0000 7f5d371dd740 1 Processor -- start 2024-04-23T18:32:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 -- start start 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827b500 0x5620c827b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c828ac10 0x5620c828d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620c81e0970 con 0x5620c827be20 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5620c817d4a0 con 0x5620c828ac10 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d371dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620c81dc930 con 0x5620c827b500 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d237fe640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 0x7f5d180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3218c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3218c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52754/0 (socket says 172.21.15.33:52754) 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3218c640 1 -- 172.21.15.33:0/193587810 learned_addr learned my addr 172.21.15.33:0/193587810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3298d640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 0x7f5d180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d237fe640 1 -- 172.21.15.33:0/451140250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5d1c0f10c0 con 0x7f5d2c107270 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3298d640 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827b500 0x5620c827b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.278+0000 7f5d3198b640 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c828ac10 0x5620c828d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 -- 172.21.15.33:0/193587810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827b500 msgr2=0x5620c827b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827b500 0x5620c827b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 -- 172.21.15.33:0/193587810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c828ac10 msgr2=0x5620c828d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c828ac10 0x5620c828d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 -- 172.21.15.33:0/193587810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c7ef3210 con 0x5620c827be20 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3218c640 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800bce0 tx=0x7f5d2800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c941d767de45dafe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d3298d640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 0x7f5d180a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005c40 tx=0x7f5d24009b10 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d20bfe640 1 -- 172.21.15.33:0/193587810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28015070 con 0x5620c827be20 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d20bfe640 1 -- 172.21.15.33:0/193587810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2801a910 con 0x5620c827be20 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.279+0000 7f5d20bfe640 1 -- 172.21.15.33:0/193587810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d280118e0 con 0x5620c827be20 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- 172.21.15.33:0/193587810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 msgr2=0x5620c828a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800bce0 tx=0x7f5d2800f730 comp rx=0 tx=0).stop 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- 172.21.15.33:0/193587810 shutdown_connections 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c828ac10 0x5620c828d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a6d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 --2- 172.21.15.33:0/193587810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827b500 0x5620c827b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- 172.21.15.33:0/193587810 >> 172.21.15.33:0/193587810 conn(0x5620c8277b30 msgr2=0x5620c8277f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- 172.21.15.33:0/193587810 shutdown_connections 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- 172.21.15.33:0/193587810 wait complete. 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 -- start start 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.280+0000 7f5d371dd740 1 Processor -- start 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c828a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827be20 0x5620c8287860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620c828d840 con 0x5620c828ac10 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5620c817b0b0 con 0x5620c827b500 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620c828d9c0 con 0x5620c827be20 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d371dd740 10 client.0 _ll_register_callbacks cb 0x5620c8168d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52756/0 (socket says 172.21.15.33:52756) 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 -- 172.21.15.33:0/1146259100 learned_addr learned my addr 172.21.15.33:0/1146259100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3218c640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c828a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3198b640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827be20 0x5620c8287860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 -- 172.21.15.33:0/1146259100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827be20 msgr2=0x5620c8287860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827be20 0x5620c8287860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 -- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 msgr2=0x5620c828a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c828a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 -- 172.21.15.33:0/1146259100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c7fbf620 con 0x5620c828ac10 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d3298d640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010bf0 tx=0x7f5d24005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e21a7778e6d41a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.281+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2401cdf0 con 0x5620c828ac10 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5d24004b20 con 0x5620c828ac10 2024-04-23T18:32:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2801a910 con 0x5620c828ac10 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d24004d60 con 0x5620c828ac10 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 10 client.16401 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620c82997f0 con 0x5620c828ac10 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2801a910 con 0x5620c828ac10 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5d24033470 con 0x5620c828ac10 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 10 client.16401 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 10 client.16401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 20 client.16401 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c829acb0 0x5620c829d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 4 client.16401 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.282+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c81ee390 con 0x5620c829acb0 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.283+0000 7f5d3218c640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c829acb0 0x5620c829d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.283+0000 7f5d3218c640 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c829acb0 0x5620c829d0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1824d577d40e510 server_cookie=dbccd559da662bf7 in_seq=0 out_seq=0 2024-04-23T18:32:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.283+0000 7f5d15ffb640 10 client.16401 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d15ffb640 1 -- 172.21.15.33:0/1146259100 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620c81ee390 con 0x5620c829acb0 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d15ffb640 10 client.16401 handle_command_reply: tid=0 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 client.16401 shutdown 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 msgr2=0x5620c8287dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5d24010bf0 tx=0x7f5d24005bd0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 shutdown_connections 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c829acb0 0x5620c829d0c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c828ac10 0x5620c8287dc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c827be20 0x5620c8287860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1146259100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c828a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 >> 172.21.15.33:0/1146259100 conn(0x5620c8277b30 msgr2=0x5620c827d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d15ffb640 0 client.16401 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 shutdown_connections 2024-04-23T18:32:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 1 -- 172.21.15.33:0/1146259100 wait complete. 2024-04-23T18:32:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.284+0000 7f5d371dd740 20 client.16401 trim_cache size 0 max 16384 2024-04-23T18:32:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 Processor -- start 2024-04-23T18:32:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 -- start start 2024-04-23T18:32:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c8296740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c82a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1ab8c0 con 0x5620c827be20 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1ad400 con 0x5620c827b500 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d371dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1abf90 con 0x5620c828ac10 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d3198b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d3198b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52770/0 (socket says 172.21.15.33:52770) 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.299+0000 7f5d3198b640 1 -- 172.21.15.33:0/1614090858 learned_addr learned my addr 172.21.15.33:0/1614090858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3218c640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c8296740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3298d640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c82a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 -- 172.21.15.33:0/1614090858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 msgr2=0x5620c82a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c82a29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 -- 172.21.15.33:0/1614090858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 msgr2=0x5620c8296740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c8296740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 -- 172.21.15.33:0/1614090858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c82732e0 con 0x5620c827be20 2024-04-23T18:32:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3218c640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c8296740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d3198b640 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0b8d50 tx=0x7f5d1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c5146ab06e220e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d17fff640 1 -- 172.21.15.33:0/1614090858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c049040 con 0x5620c827be20 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d17fff640 1 -- 172.21.15.33:0/1614090858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c0bd070 con 0x5620c827be20 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.300+0000 7f5d17fff640 1 -- 172.21.15.33:0/1614090858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0b96e0 con 0x5620c827be20 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- 172.21.15.33:0/1614090858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 msgr2=0x5620c828a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0b8d50 tx=0x7f5d1c0045e0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- 172.21.15.33:0/1614090858 shutdown_connections 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c82a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827be20 0x5620c828a0a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- 172.21.15.33:0/1614090858 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827b500 0x5620c8296740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- 172.21.15.33:0/1614090858 >> 172.21.15.33:0/1614090858 conn(0x5620c8277b30 msgr2=0x5620c8287820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- 172.21.15.33:0/1614090858 shutdown_connections 2024-04-23T18:32:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- 172.21.15.33:0/1614090858 wait complete. 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- start start 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 Processor -- start 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827be20 0x5620c8293130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c828e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1acd30 con 0x5620c827b500 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1ac660 con 0x5620c827be20 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d2c1a98e0 con 0x5620c828ac10 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d371dd740 10 client.0 _ll_register_callbacks cb 0x7f5d24012d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d3218c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d3218c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52778/0 (socket says 172.21.15.33:52778) 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d3218c640 1 -- 172.21.15.33:0/2170461139 learned_addr learned my addr 172.21.15.33:0/2170461139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.301+0000 7f5d3198b640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827be20 0x5620c8293130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3298d640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c828e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 -- 172.21.15.33:0/2170461139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 msgr2=0x5620c828e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c828e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 -- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827be20 msgr2=0x5620c8293130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827be20 0x5620c8293130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 -- 172.21.15.33:0/2170461139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c828e920 con 0x5620c827b500 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3298d640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c828e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d3218c640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5d28011550 tx=0x7f5d2800fd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caf5cea873d3a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2802c060 con 0x5620c827b500 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5d28005450 con 0x5620c827b500 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c0bd070 con 0x5620c827b500 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5d28015040 con 0x5620c827b500 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d371dd740 10 client.16407 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5620c8300e20 con 0x5620c827b500 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.302+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1c0bd070 con 0x5620c827b500 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5d280270d0 con 0x5620c827b500 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 10 client.16407 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 20 client.16407 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c83013b0 0x5620c83037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 4 client.16407 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620c81e8200 con 0x5620c83013b0 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d3198b640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c83013b0 0x5620c83037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d3198b640 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c83013b0 0x5620c83037a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dea67dd99264a8 server_cookie=d2d4741aa8d687e8 in_seq=0 out_seq=0 2024-04-23T18:32:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.303+0000 7f5d15ffb640 10 client.16407 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d15ffb640 1 -- 172.21.15.33:0/2170461139 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620c81e8200 con 0x5620c83013b0 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d15ffb640 10 client.16407 handle_command_reply: tid=0 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 client.16407 shutdown 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 msgr2=0x5620c8292bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5d28011550 tx=0x7f5d2800fd70 comp rx=0 tx=0).stop 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 shutdown_connections 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5620c83013b0 0x5620c83037a0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620c828ac10 0x5620c828e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5620c827be20 0x5620c8293130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d15ffb640 0 client.16407 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 --2- 172.21.15.33:0/2170461139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5620c827b500 0x5620c8292bf0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 >> 172.21.15.33:0/2170461139 conn(0x5620c8277b30 msgr2=0x5620c8282670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 shutdown_connections 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 1 -- 172.21.15.33:0/2170461139 wait complete. 2024-04-23T18:32:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.307+0000 7f5d371dd740 20 client.16407 trim_cache size 0 max 16384 2024-04-23T18:32:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:32:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:32:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:32:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 msgr2=0x7f5d180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 0x7f5d180a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d24005c40 tx=0x7f5d24009b10 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 msgr2=0x7f5d2c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c005ab0 tx=0x7f5d1c003cd0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 shutdown_connections 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5d180a3dc0 0x7f5d180a6280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d2c19cf50 0x7f5d2c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d2c107270 0x7f5d2c198c20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 --2- 172.21.15.33:0/451140250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d2c1021a0 0x7f5d2c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 >> 172.21.15.33:0/451140250 conn(0x7f5d2c0793e0 msgr2=0x7f5d2c0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 shutdown_connections 2024-04-23T18:32:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:16.308+0000 7f5d343f0640 1 -- 172.21.15.33:0/451140250 wait complete. 2024-04-23T18:32:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:16 smithi106 ceph-mon[32716]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:17.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:16 smithi122 ceph-mon[32546]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:17.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:16 smithi033 ceph-mon[29274]: pgmap v177: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 28 op/s 2024-04-23T18:32:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:18 smithi106 ceph-mon[32716]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:19.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:18 smithi122 ceph-mon[32546]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:19.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:18 smithi033 ceph-mon[29274]: pgmap v179: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:21.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:20 smithi106 ceph-mon[32716]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:21.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:20 smithi122 ceph-mon[32546]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:21.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:20 smithi033 ceph-mon[29274]: pgmap v180: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 8 op/s 2024-04-23T18:32:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:22 smithi106 ceph-mon[32716]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:22 smithi122 ceph-mon[32546]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:23.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:22 smithi033 ceph-mon[29274]: pgmap v181: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:25.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:24 smithi106 ceph-mon[32716]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:25.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:24 smithi122 ceph-mon[32546]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:25.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:24 smithi033 ceph-mon[29274]: pgmap v182: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-23T18:32:27.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:26 smithi106 ceph-mon[32716]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-04-23T18:32:27.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:26 smithi122 ceph-mon[32546]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-04-23T18:32:27.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:26 smithi033 ceph-mon[29274]: pgmap v183: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-04-23T18:32:29.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:28 smithi106 ceph-mon[32716]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:29.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:28 smithi122 ceph-mon[32546]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:29.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:28 smithi033 ceph-mon[29274]: pgmap v184: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 -- 172.21.15.33:0/2648419786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf4102b80 msgr2=0x7f5cf4106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/2648419786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf4102b80 0x7f5cf4106bc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009900 tx=0x7f5ce402a410 comp rx=0 tx=0).stop 2024-04-23T18:32:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 -- 172.21.15.33:0/2648419786 shutdown_connections 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/2648419786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf41072a0 0x7f5cf410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/2648419786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf4102b80 0x7f5cf4106bc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/2648419786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 0x7f5cf41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 -- 172.21.15.33:0/2648419786 >> 172.21.15.33:0/2648419786 conn(0x7f5cf40793e0 msgr2=0x7f5cf40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 -- 172.21.15.33:0/2648419786 shutdown_connections 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.408+0000 7f5cfbe79640 1 -- 172.21.15.33:0/2648419786 wait complete. 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 Processor -- start 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 -- start start 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 0x7f5cf4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf419cee0 0x7f5cf419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4112210 con 0x7f5cf41072a0 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4112090 con 0x7f5cf419cee0 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfbe79640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4112390 con 0x7f5cf41021d0 2024-04-23T18:32:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55188/0 (socket says 172.21.15.33:55188) 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 -- 172.21.15.33:0/355575488 learned_addr learned my addr 172.21.15.33:0/355575488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cfa416640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf419cee0 0x7f5cf419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 0x7f5cf4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 -- 172.21.15.33:0/355575488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 msgr2=0x7f5cf4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 0x7f5cf4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 -- 172.21.15.33:0/355575488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf419cee0 msgr2=0x7f5cf419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf419cee0 0x7f5cf419d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.409+0000 7f5cf9414640 1 -- 172.21.15.33:0/355575488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf419d9d0 con 0x7f5cf41072a0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5cf9414640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4005ab0 tx=0x7f5ce4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5ceaffd640 1 -- 172.21.15.33:0/355575488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce403f070 con 0x7f5cf41072a0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf419dc60 con 0x7f5cf41072a0 2024-04-23T18:32:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5ceaffd640 1 -- 172.21.15.33:0/355575488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce4031b50 con 0x7f5cf41072a0 2024-04-23T18:32:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5ceaffd640 1 -- 172.21.15.33:0/355575488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4035030 con 0x7f5cf41072a0 2024-04-23T18:32:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.410+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf41a5480 con 0x7f5cf41072a0 2024-04-23T18:32:29.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.412+0000 7f5ceaffd640 1 -- 172.21.15.33:0/355575488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ce4042070 con 0x7f5cf41072a0 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.412+0000 7f5cfeb8c740 1 Processor -- start 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 -- start start 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9f520 0x562ebed9f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ebed04970 con 0x562ebed9fe40 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562ebeca14a0 con 0x562ebedaec10 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ebed00930 con 0x562ebed9f520 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5ceaffd640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 0x7f5ce00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55192/0 (socket says 172.21.15.33:55192) 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1432784887 learned_addr learned my addr 172.21.15.33:0/1432784887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfa416640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9f520 0x562ebed9f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5cfa416640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 0x7f5ce00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.413+0000 7f5ceaffd640 1 -- 172.21.15.33:0/355575488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5ce40f0e80 con 0x7f5cf41072a0 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1432784887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9f520 msgr2=0x562ebed9f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9f520 0x562ebed9f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1432784887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 msgr2=0x562ebedb1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1432784887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebea17210 con 0x562ebed9fe40 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000dcb0 tx=0x7f5cf000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34c7e7bf8d8c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cfa416640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 0x7f5ce00a6300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005fd0 tx=0x7f5cec009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cdffff640 1 -- 172.21.15.33:0/1432784887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0011440 con 0x562ebed9fe40 2024-04-23T18:32:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cdffff640 1 -- 172.21.15.33:0/1432784887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0011a90 con 0x562ebed9fe40 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.414+0000 7f5cdffff640 1 -- 172.21.15.33:0/1432784887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0010a00 con 0x562ebed9fe40 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1432784887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 msgr2=0x562ebedae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000dcb0 tx=0x7f5cf000f730 comp rx=0 tx=0).stop 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1432784887 shutdown_connections 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedae6d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1432784887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9f520 0x562ebed9f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1432784887 >> 172.21.15.33:0/1432784887 conn(0x562ebed9bb30 msgr2=0x562ebed9bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1432784887 shutdown_connections 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1432784887 wait complete. 2024-04-23T18:32:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 -- start start 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.415+0000 7f5cfeb8c740 1 Processor -- start 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebed9f520 0x562ebedae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebedaec10 0x562ebedabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ebedb1840 con 0x562ebed9fe40 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562ebec9f0b0 con 0x562ebed9f520 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ebedb19c0 con 0x562ebedaec10 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfeb8c740 10 client.0 _ll_register_callbacks cb 0x562ebec8cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55194/0 (socket says 172.21.15.33:55194) 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 -- 172.21.15.33:0/1838876570 learned_addr learned my addr 172.21.15.33:0/1838876570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebed9f520 0x562ebedae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cfa416640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebedaec10 0x562ebedabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 -- 172.21.15.33:0/1838876570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebedaec10 msgr2=0x562ebedabd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebedaec10 0x562ebedabd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 -- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebed9f520 msgr2=0x562ebedae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebed9f520 0x562ebedae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.416+0000 7f5cf9414640 1 -- 172.21.15.33:0/1838876570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebeae3620 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40ee5e0 tx=0x7f5ce4042f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1aaab0630dd4bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4049040 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5ce40b9440 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf0011a90 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5ce40bcdd0 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 10 client.16419 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ebedbd790 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf0011a90 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce40d6140 con 0x562ebed9fe40 2024-04-23T18:32:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 10 client.16419 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 20 client.16419 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebedbeba0 0x562ebedc0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 4 client.16419 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.417+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ebed12390 con 0x562ebedbeba0 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.418+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebedbeba0 0x562ebedc0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.418+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebedbeba0 0x562ebedc0fe0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=455be299118518f8 server_cookie=25edf83c3cd11f8c in_seq=0 out_seq=0 2024-04-23T18:32:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.418+0000 7f5cdd7fa640 10 client.16419 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/1838876570 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ebed12390 con 0x562ebedbeba0 2024-04-23T18:32:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cdd7fa640 10 client.16419 handle_command_reply: tid=0 2024-04-23T18:32:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 client.16419 shutdown 2024-04-23T18:32:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 msgr2=0x562ebedab800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40ee5e0 tx=0x7f5ce4042f90 comp rx=0 tx=0).stop 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 shutdown_connections 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebedbeba0 0x562ebedc0fe0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebedaec10 0x562ebedabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9fe40 0x562ebedab800 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cdd7fa640 0 client.16419 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1838876570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebed9f520 0x562ebedae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 >> 172.21.15.33:0/1838876570 conn(0x562ebed9bb30 msgr2=0x562ebeda19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 shutdown_connections 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1838876570 wait complete. 2024-04-23T18:32:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.419+0000 7f5cfeb8c740 20 client.16419 trim_cache size 0 max 16384 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.434+0000 7f5cfeb8c740 1 Processor -- start 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 -- start start 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebeda6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41ab920 con 0x562ebed9f520 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41ad460 con 0x562ebedaec10 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41abff0 con 0x562ebed9fe40 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55196/0 (socket says 172.21.15.33:55196) 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1231094525 learned_addr learned my addr 172.21.15.33:0/1231094525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9414640 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebeda6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cfa416640 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1231094525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 msgr2=0x562ebeda6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebeda6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1231094525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 msgr2=0x562ebedae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 -- 172.21.15.33:0/1231094525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ebed9fc70 con 0x562ebed9f520 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0006010 tx=0x7f5cf000d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4192d78318bb9eba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.435+0000 7f5cdf7fe640 1 -- 172.21.15.33:0/1231094525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0030070 con 0x562ebed9f520 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cdf7fe640 1 -- 172.21.15.33:0/1231094525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf00054f0 con 0x562ebed9f520 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cdf7fe640 1 -- 172.21.15.33:0/1231094525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0010970 con 0x562ebed9f520 2024-04-23T18:32:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1231094525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 msgr2=0x562ebedba700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0006010 tx=0x7f5cf000d5a0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1231094525 shutdown_connections 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebeda6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/1231094525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedba700 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1231094525 >> 172.21.15.33:0/1231094525 conn(0x562ebed9bb30 msgr2=0x562ebeda6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1231094525 shutdown_connections 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/1231094525 wait complete. 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.436+0000 7f5cfeb8c740 1 -- start start 2024-04-23T18:32:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 Processor -- start 2024-04-23T18:32:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebedb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41acd90 con 0x562ebed9f520 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41ac6c0 con 0x562ebedaec10 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf41a9900 con 0x562ebed9fe40 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfeb8c740 10 client.0 _ll_register_callbacks cb 0x7f5ce40bcc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55206/0 (socket says 172.21.15.33:55206) 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 -- 172.21.15.33:0/3134383532 learned_addr learned my addr 172.21.15.33:0/3134383532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9414640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebedb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfa416640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 -- 172.21.15.33:0/3134383532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 msgr2=0x562ebedb21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebedb21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 -- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 msgr2=0x562ebedb2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cf9c15640 1 -- 172.21.15.33:0/3134383532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ebedb2d00 con 0x562ebed9f520 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.437+0000 7f5cfa416640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cf9c15640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000f770 tx=0x7f5cf0005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc28192a28a1adc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf003a060 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5cf0005240 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf00054f0 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0030070 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 10 client.16425 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562ebee24f70 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf00054f0 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5cf0004890 con 0x562ebed9f520 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 10 client.16425 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 20 client.16425 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebee25430 0x562ebee27820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 4 client.16425 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.438+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ebedac0d0 con 0x562ebee25430 2024-04-23T18:32:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.439+0000 7f5cf9414640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebee25430 0x562ebee27820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.439+0000 7f5cdd7fa640 10 client.16425 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.439+0000 7f5cf9414640 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebee25430 0x562ebee27820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185ae7f50ef9c674 server_cookie=2b2dd152ed5caa95 in_seq=0 out_seq=0 2024-04-23T18:32:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.446+0000 7f5cdd7fa640 1 -- 172.21.15.33:0/3134383532 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ebedac0d0 con 0x562ebee25430 2024-04-23T18:32:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.446+0000 7f5cdd7fa640 10 client.16425 handle_command_reply: tid=0 2024-04-23T18:32:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.446+0000 7f5cfeb8c740 1 client.16425 shutdown 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 msgr2=0x562ebedb6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000f770 tx=0x7f5cf0005f50 comp rx=0 tx=0).stop 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 shutdown_connections 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ebee25430 0x562ebee27820 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ebedaec10 0x562ebedb2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ebed9fe40 0x562ebedb21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 --2- 172.21.15.33:0/3134383532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ebed9f520 0x562ebedb6de0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cdd7fa640 0 client.16425 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 >> 172.21.15.33:0/3134383532 conn(0x562ebed9bb30 msgr2=0x562ebeda10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 shutdown_connections 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 1 -- 172.21.15.33:0/3134383532 wait complete. 2024-04-23T18:32:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfeb8c740 20 client.16425 trim_cache size 0 max 16384 2024-04-23T18:32:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:32:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:32:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:32:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 msgr2=0x7f5ce00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 0x7f5ce00a6300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cec005fd0 tx=0x7f5cec009500 comp rx=0 tx=0).stop 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 msgr2=0x7f5cf4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4005ab0 tx=0x7f5ce4003b90 comp rx=0 tx=0).stop 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.447+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 shutdown_connections 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5ce00a3e40 0x7f5ce00a6300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cf419cee0 0x7f5cf419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cf41072a0 0x7f5cf4198bb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 --2- 172.21.15.33:0/355575488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf41021d0 0x7f5cf4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 >> 172.21.15.33:0/355575488 conn(0x7f5cf40793e0 msgr2=0x7f5cf40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 shutdown_connections 2024-04-23T18:32:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:29.448+0000 7f5cfbe79640 1 -- 172.21.15.33:0/355575488 wait complete. 2024-04-23T18:32:31.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:30 smithi106 ceph-mon[32716]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:30 smithi122 ceph-mon[32546]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:31.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:30 smithi033 ceph-mon[29274]: pgmap v185: 97 pgs: 97 active+clean; 4.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 3.6 KiB/s wr, 9 op/s 2024-04-23T18:32:33.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:32 smithi106 ceph-mon[32716]: pgmap v186: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-04-23T18:32:33.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:32 smithi122 ceph-mon[32546]: pgmap v186: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-04-23T18:32:33.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:32 smithi033 ceph-mon[29274]: pgmap v186: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-04-23T18:32:33.847 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 -- 172.21.15.33:0/1224918068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181031e0 msgr2=0x7fe1181035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 --2- 172.21.15.33:0/1224918068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181031e0 0x7fe1181035c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe114009900 tx=0x7fe11402a510 comp rx=0 tx=0).stop 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 -- 172.21.15.33:0/1224918068 shutdown_connections 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 --2- 172.21.15.33:0/1224918068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe11810c520 0x7fe11810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 --2- 172.21.15.33:0/1224918068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe118103b00 0x7fe11810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 --2- 172.21.15.33:0/1224918068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1181031e0 0x7fe1181035c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 -- 172.21.15.33:0/1224918068 >> 172.21.15.33:0/1224918068 conn(0x7fe118077f50 msgr2=0x7fe118078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 -- 172.21.15.33:0/1224918068 shutdown_connections 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.969+0000 7fe1201e5640 1 -- 172.21.15.33:0/1224918068 wait complete. 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 Processor -- start 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 -- start start 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1181031e0 0x7fe11819e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe118103b00 0x7fe11819e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe118112200 con 0x7fe11810c520 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe118112080 con 0x7fe118103b00 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe1201e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe118112380 con 0x7fe1181031e0 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe11e782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.970+0000 7fe11e782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55226/0 (socket says 172.21.15.33:55226) 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 -- 172.21.15.33:0/2317300580 learned_addr learned my addr 172.21.15.33:0/2317300580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11df81640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1181031e0 0x7fe11819e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11d780640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe118103b00 0x7fe11819e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 -- 172.21.15.33:0/2317300580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1181031e0 msgr2=0x7fe11819e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1181031e0 0x7fe11819e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 -- 172.21.15.33:0/2317300580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe118103b00 msgr2=0x7fe11819e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe118103b00 0x7fe11819e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 -- 172.21.15.33:0/2317300580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe118198d80 con 0x7fe11810c520 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe11e782640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe11000eaa0 tx=0x7fe11000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11000cdb0 con 0x7fe11810c520 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118199070 con 0x7fe11810c520 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe110005020 con 0x7fe11810c520 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100116d0 con 0x7fe11810c520 2024-04-23T18:32:33.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.971+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1181a5310 con 0x7fe11810c520 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.972+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe118068d70 con 0x7fe11810c520 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.973+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe1100051d0 con 0x7fe11810c520 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.974+0000 7fe10effd640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 0x7fe1040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.974+0000 7fe11df81640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 0x7fe1040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.974+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7fe1100d08d0 con 0x7fe11810c520 2024-04-23T18:32:33.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.975+0000 7fe11df81640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 0x7fe1040a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe114005ab0 tx=0x7fe114005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:33.979+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe110016030 con 0x7fe11810c520 2024-04-23T18:32:34.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.204+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1181076a0 con 0x7fe11810c520 2024-04-23T18:32:34.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.205+0000 7fe10effd640 1 -- 172.21.15.33:0/2317300580 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11929 (secure 0 0 0) 0x7fe11009c930 con 0x7fe11810c520 2024-04-23T18:32:34.197 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:32:34.197 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:32:15.719012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25900":{"gid":25900,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/1758592712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1758592712},{"type":"v1","addr":"172.21.15.122:6835","nonce":1758592712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:32:34.198 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 msgr2=0x7fe1040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 0x7fe1040a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe114005ab0 tx=0x7fe114005a20 comp rx=0 tx=0).stop 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 msgr2=0x7fe118198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe11000eaa0 tx=0x7fe11000ef60 comp rx=0 tx=0).stop 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 shutdown_connections 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe1040a3d20 0x7fe1040a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11810c520 0x7fe118198600 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe118103b00 0x7fe11819e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 --2- 172.21.15.33:0/2317300580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1181031e0 0x7fe11819e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 >> 172.21.15.33:0/2317300580 conn(0x7fe118077f50 msgr2=0x7fe1180ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 shutdown_connections 2024-04-23T18:32:34.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.207+0000 7fe1201e5640 1 -- 172.21.15.33:0/2317300580 wait complete. 2024-04-23T18:32:34.210 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 70580dcf --release --await' 2024-04-23T18:32:34.210 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 70580dcf --release --await 2024-04-23T18:32:34.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 -- 172.21.15.33:0/152419176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50102190 msgr2=0x7f0d50102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 --2- 172.21.15.33:0/152419176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50102190 0x7f0d50102570 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009900 tx=0x7f0d4002a410 comp rx=0 tx=0).stop 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 -- 172.21.15.33:0/152419176 shutdown_connections 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 --2- 172.21.15.33:0/152419176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d50107260 0x7f0d5010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 --2- 172.21.15.33:0/152419176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102b40 0x7f0d50106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 --2- 172.21.15.33:0/152419176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50102190 0x7f0d50102570 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 -- 172.21.15.33:0/152419176 >> 172.21.15.33:0/152419176 conn(0x7f0d500fd990 msgr2=0x7f0d500ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 -- 172.21.15.33:0/152419176 shutdown_connections 2024-04-23T18:32:34.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.328+0000 7f0d54baf640 1 -- 172.21.15.33:0/152419176 wait complete. 2024-04-23T18:32:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 Processor -- start 2024-04-23T18:32:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 -- start start 2024-04-23T18:32:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102190 0x7f0d501986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 0x7f0d5019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501121a0 con 0x7f0d50107260 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50112020 con 0x7f0d50102190 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d54baf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50112320 con 0x7f0d5019cf30 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55252/0 (socket says 172.21.15.33:55252) 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 -- 172.21.15.33:0/1695939954 learned_addr learned my addr 172.21.15.33:0/1695939954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102190 0x7f0d501986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 0x7f0d5019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 -- 172.21.15.33:0/1695939954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 msgr2=0x7f0d5019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 0x7f0d5019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 -- 172.21.15.33:0/1695939954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102190 msgr2=0x7f0d501986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102190 0x7f0d501986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d47fff640 1 -- 172.21.15.33:0/1695939954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5019da20 con 0x7f0d50107260 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.329+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 0x7f0d5019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d47fff640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c970 tx=0x7f0d3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d477fe640 1 -- 172.21.15.33:0/1695939954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38014070 con 0x7f0d50107260 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5019dd10 con 0x7f0d50107260 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d477fe640 1 -- 172.21.15.33:0/1695939954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38004440 con 0x7f0d50107260 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d477fe640 1 -- 172.21.15.33:0/1695939954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38003e50 con 0x7f0d50107260 2024-04-23T18:32:34.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.330+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a5550 con 0x7f0d50107260 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.332+0000 7f0d477fe640 1 -- 172.21.15.33:0/1695939954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d38017070 con 0x7f0d50107260 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 Processor -- start 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 -- start start 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d477fe640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 0x7f0d3c0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e0d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e1a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562621d726d0 con 0x562621e0ddb0 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562621d0f1a0 con 0x562621e0d490 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d5799c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562621d6e690 con 0x562621e18480 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 0x7f0d3c0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55268/0 (socket says 172.21.15.33:55268) 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d47fff640 1 -- 172.21.15.33:0/3582040690 learned_addr learned my addr 172.21.15.33:0/3582040690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.333+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e0d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d477fe640 1 -- 172.21.15.33:0/1695939954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f0d380d0300 con 0x7f0d50107260 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 -- 172.21.15.33:0/3582040690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 msgr2=0x562621e1a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e1a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 -- 172.21.15.33:0/3582040690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 msgr2=0x562621e0d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e0d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 -- 172.21.15.33:0/3582040690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562621d195b0 con 0x562621e0ddb0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 0x7f0d3c0a6400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005cc0 tx=0x7f0d480087f0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d47fff640 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d38097f90 tx=0x7f0d38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a635ccdd93654835 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d44ff9640 1 -- 172.21.15.33:0/3582040690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d380b7020 con 0x562621e0ddb0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d44ff9640 1 -- 172.21.15.33:0/3582040690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3809c9b0 con 0x562621e0ddb0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.334+0000 7f0d44ff9640 1 -- 172.21.15.33:0/3582040690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38098920 con 0x562621e0ddb0 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- 172.21.15.33:0/3582040690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 msgr2=0x562621e17f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0d38097f90 tx=0x7f0d38000bc0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- 172.21.15.33:0/3582040690 shutdown_connections 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e1a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e17f40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 --2- 172.21.15.33:0/3582040690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e0d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- 172.21.15.33:0/3582040690 >> 172.21.15.33:0/3582040690 conn(0x562621e097b0 msgr2=0x562621e09b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- 172.21.15.33:0/3582040690 shutdown_connections 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- 172.21.15.33:0/3582040690 wait complete. 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 -- start start 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.335+0000 7f0d5799c740 1 Processor -- start 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e17f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e15960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562621e1b250 con 0x562621e0ddb0 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562621d0cb20 con 0x562621e0d490 2024-04-23T18:32:34.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562621e1b3d0 con 0x562621e18480 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 10 client.0 _ll_register_callbacks cb 0x562621d0c530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55272/0 (socket says 172.21.15.33:55272) 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 -- 172.21.15.33:0/845962281 learned_addr learned my addr 172.21.15.33:0/845962281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e17f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e15960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 -- 172.21.15.33:0/845962281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 msgr2=0x562621e15960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e15960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 -- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 msgr2=0x562621e17f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e17f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 -- 172.21.15.33:0/845962281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562621a84920 con 0x562621e0ddb0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d47fff640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d380038f0 tx=0x7f0d380a4f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462dc29833cf5685 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3807d020 con 0x562621e0ddb0 2024-04-23T18:32:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f0d380cdce0 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3809c9b0 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d380a0660 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 10 client.16443 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.336+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562621b50d30 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3809c9b0 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f0d38098000 con 0x562621e0ddb0 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 10 client.16443 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:34.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 10 client.16443 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 20 client.16443 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e28480 0x562621e2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 4 client.16443 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562621d30500 con 0x562621e28480 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e28480 0x562621e2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e28480 0x562621e2a870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b901535aa80c7444 server_cookie=26273302166f6f03 in_seq=0 out_seq=0 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.337+0000 7f0d35ffb640 10 client.16443 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d35ffb640 1 -- 172.21.15.33:0/845962281 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562621d30500 con 0x562621e28480 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d35ffb640 10 client.16443 handle_command_reply: tid=0 2024-04-23T18:32:34.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 client.16443 shutdown 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 msgr2=0x562621e15220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0d380038f0 tx=0x7f0d380a4f20 comp rx=0 tx=0).stop 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 shutdown_connections 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e28480 0x562621e2a870 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e18480 0x562621e15960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d35ffb640 0 client.16443 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0ddb0 0x562621e15220 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 --2- 172.21.15.33:0/845962281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e17f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 >> 172.21.15.33:0/845962281 conn(0x562621e097b0 msgr2=0x562621e0f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 shutdown_connections 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.338+0000 7f0d5799c740 1 -- 172.21.15.33:0/845962281 wait complete. 2024-04-23T18:32:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.339+0000 7f0d5799c740 20 client.16443 trim_cache size 0 max 16384 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 Processor -- start 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 -- start start 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 0x562621e2ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ada90 con 0x562621e0d490 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501af5d0 con 0x562621e18480 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d5799c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ae160 con 0x562621e0ddb0 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55284/0 (socket says 172.21.15.33:55284) 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.356+0000 7f0d4e59c640 1 -- 172.21.15.33:0/738298466 learned_addr learned my addr 172.21.15.33:0/738298466 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d47fff640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 0x562621e2ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 -- 172.21.15.33:0/738298466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 msgr2=0x562621e2d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 -- 172.21.15.33:0/738298466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 msgr2=0x562621e2ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 0x562621e2ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 -- 172.21.15.33:0/738298466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562621e0dbe0 con 0x562621e0d490 2024-04-23T18:32:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 0x562621e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002abe0 tx=0x7f0d40005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d173203874d0179 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d37fff640 1 -- 172.21.15.33:0/738298466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4004a030 con 0x562621e0d490 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d37fff640 1 -- 172.21.15.33:0/738298466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40031980 con 0x562621e0d490 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.357+0000 7f0d37fff640 1 -- 172.21.15.33:0/738298466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40035040 con 0x562621e0d490 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- 172.21.15.33:0/738298466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 msgr2=0x562621e17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f0d4002abe0 tx=0x7f0d40005e80 comp rx=0 tx=0).stop 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- 172.21.15.33:0/738298466 shutdown_connections 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e18480 0x562621e2ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- 172.21.15.33:0/738298466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e0d490 0x562621e17a60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- 172.21.15.33:0/738298466 >> 172.21.15.33:0/738298466 conn(0x562621e097b0 msgr2=0x562621e23eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- 172.21.15.33:0/738298466 shutdown_connections 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- 172.21.15.33:0/738298466 wait complete. 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- start start 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 Processor -- start 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501aef00 con 0x562621e18480 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501ae830 con 0x562621e0d490 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d501aba70 con 0x562621e0ddb0 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d5799c740 10 client.0 _ll_register_callbacks cb 0x7f0d3809c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d4ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d4ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55294/0 (socket says 172.21.15.33:55294) 2024-04-23T18:32:34.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d4ed9d640 1 -- 172.21.15.33:0/1253475701 learned_addr learned my addr 172.21.15.33:0/1253475701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.358+0000 7f0d47fff640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 -- 172.21.15.33:0/1253475701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 msgr2=0x562621e2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 -- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 msgr2=0x562621e2d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e2d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 -- 172.21.15.33:0/1253475701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562621e2df10 con 0x562621e18480 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d4ed9d640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0d48008fd0 tx=0x7f0d48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990aef66d5ad514d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48024070 con 0x562621e18480 2024-04-23T18:32:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f0d48004b20 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40031980 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0d4802f050 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d5799c740 10 client.16449 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562621e8eff0 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.359+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d40031980 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f0d480205d0 con 0x562621e18480 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 10 client.16449 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 10 client.16449 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 20 client.16449 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e8f770 0x562621e91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 4 client.16449 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70580dcf", "release": true, "await": true}] 2024-04-23T18:32:34.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70580dcf", "release": true, "await": true}) -- 0x562621d800f0 con 0x562621e8f770 2024-04-23T18:32:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e8f770 0x562621e91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d4e59c640 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e8f770 0x562621e91b90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda4faf6be1fc01e server_cookie=a1796758ccfcdbe7 in_seq=0 out_seq=0 2024-04-23T18:32:34.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.360+0000 7f0d35ffb640 10 client.16449 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d35ffb640 1 -- 172.21.15.33:0/1253475701 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562621d800f0 con 0x562621e8f770 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d35ffb640 10 client.16449 handle_command_reply: tid=0 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 client.16449 shutdown 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 msgr2=0x562621e2adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f0d48008fd0 tx=0x7f0d48004a40 comp rx=0 tx=0).stop 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 shutdown_connections 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562621e8f770 0x562621e91b90 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562621e18480 0x562621e2adb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562621e0ddb0 0x562621e2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 --2- 172.21.15.33:0/1253475701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562621e0d490 0x562621e2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d35ffb640 0 client.16449 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 >> 172.21.15.33:0/1253475701 conn(0x562621e097b0 msgr2=0x562621e08140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 shutdown_connections 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 1 -- 172.21.15.33:0/1253475701 wait complete. 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d5799c740 20 client.16449 trim_cache size 0 max 16384 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 33, 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 24, 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:32:34.361 INFO:teuthology.orchestra.run.smithi033.stdout: "70580dcf": { 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 24, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 78.8, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:32:34.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:32:34.363 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 msgr2=0x7f0d3c0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 0x7f0d3c0a6400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005cc0 tx=0x7f0d480087f0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 msgr2=0x7f0d50198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c970 tx=0x7f0d3800ce30 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 shutdown_connections 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0d3c0a3f40 0x7f0d3c0a6400 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d5019cf30 0x7f0d5019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d50107260 0x7f0d50198c00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 --2- 172.21.15.33:0/1695939954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0d50102190 0x7f0d501986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.363+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 >> 172.21.15.33:0/1695939954 conn(0x7f0d500fd990 msgr2=0x7f0d500ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.364+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 shutdown_connections 2024-04-23T18:32:34.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:32:34.364+0000 7f0d54baf640 1 -- 172.21.15.33:0/1695939954 wait complete. 2024-04-23T18:32:34.367 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70580dcf', seconds elapsed: 19.9 2024-04-23T18:32:35.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:34 smithi106 ceph-mon[32716]: pgmap v187: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-23T18:32:35.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:34 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2317300580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:34 smithi122 ceph-mon[32546]: pgmap v187: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-23T18:32:35.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:34 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2317300580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:34 smithi033 ceph-mon[29274]: pgmap v187: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-23T18:32:35.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:34 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2317300580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:32:37.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:36 smithi106 ceph-mon[32716]: pgmap v188: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-23T18:32:37.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:36 smithi122 ceph-mon[32546]: pgmap v188: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-23T18:32:37.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:36 smithi033 ceph-mon[29274]: pgmap v188: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 15 op/s 2024-04-23T18:32:39.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:38 smithi106 ceph-mon[32716]: pgmap v189: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:39.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:38 smithi122 ceph-mon[32546]: pgmap v189: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:39.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:38 smithi033 ceph-mon[29274]: pgmap v189: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:39 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:40.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:39 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:40.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:39 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:32:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:40 smithi106 ceph-mon[32716]: pgmap v190: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:40 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:40 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:40 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:32:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:40 smithi122 ceph-mon[32546]: pgmap v190: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:40 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:40 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:40 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:32:41.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:40 smithi033 ceph-mon[29274]: pgmap v190: 97 pgs: 97 active+clean; 4.1 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:32:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:40 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:32:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:40 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:32:41.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:40 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 -- 172.21.15.33:0/4118559352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 msgr2=0x7fe67810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 --2- 172.21.15.33:0/4118559352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe67810bfe0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe660009900 tx=0x7fe66002a510 comp rx=0 tx=0).stop 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 -- 172.21.15.33:0/4118559352 shutdown_connections 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 --2- 172.21.15.33:0/4118559352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 0x7fe67810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 --2- 172.21.15.33:0/4118559352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe67810bfe0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 --2- 172.21.15.33:0/4118559352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 0x7fe6781035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 -- 172.21.15.33:0/4118559352 >> 172.21.15.33:0/4118559352 conn(0x7fe678077fd0 msgr2=0x7fe6780783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 -- 172.21.15.33:0/4118559352 shutdown_connections 2024-04-23T18:32:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.547+0000 7fe67d88a640 1 -- 172.21.15.33:0/4118559352 wait complete. 2024-04-23T18:32:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 Processor -- start 2024-04-23T18:32:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 -- start start 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 0x7fe678198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 0x7fe67819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781121b0 con 0x7fe678103b00 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112030 con 0x7fe6781031e0 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe67d88a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112330 con 0x7fe67810c520 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47254/0 (socket says 172.21.15.33:47254) 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe6767fc640 1 -- 172.21.15.33:0/3942068568 learned_addr learned my addr 172.21.15.33:0/3942068568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.548+0000 7fe676ffd640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 0x7fe678198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6777fe640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 0x7fe67819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 -- 172.21.15.33:0/3942068568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 msgr2=0x7fe67819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 0x7fe67819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 -- 172.21.15.33:0/3942068568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 msgr2=0x7fe678198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 0x7fe678198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 -- 172.21.15.33:0/3942068568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67819d6b0 con 0x7fe678103b00 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe6767fc640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe660005940 tx=0x7fe660004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe67c888640 1 -- 172.21.15.33:0/3942068568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66003f030 con 0x7fe678103b00 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe67c888640 1 -- 172.21.15.33:0/3942068568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe660003aa0 con 0x7fe678103b00 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67819d940 con 0x7fe678103b00 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe67c888640 1 -- 172.21.15.33:0/3942068568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660035070 con 0x7fe678103b00 2024-04-23T18:32:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.549+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a5230 con 0x7fe678103b00 2024-04-23T18:32:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe67c888640 1 -- 172.21.15.33:0/3942068568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe660042030 con 0x7fe678103b00 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 Processor -- start 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 -- start start 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe67c888640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56029b640970 con 0x56029b6e6800 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56029b5dd4a0 con 0x56029b6dbf40 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.552+0000 7fe6805a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56029b63c930 con 0x56029b6db620 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe676ffd640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47268/0 (socket says 172.21.15.33:47268) 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 -- 172.21.15.33:0/3346694258 learned_addr learned my addr 172.21.15.33:0/3346694258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe67c888640 1 -- 172.21.15.33:0/3942068568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7fe6600f0e40 con 0x7fe678103b00 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6767fc640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 -- 172.21.15.33:0/3346694258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 msgr2=0x56029b6dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 -- 172.21.15.33:0/3346694258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 msgr2=0x56029b6e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 -- 172.21.15.33:0/3346694258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029b353210 con 0x56029b6e6800 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6767fc640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe676ffd640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe670009bd0 tx=0x7fe670005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe6777fe640 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe668012950 tx=0x7fe668009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2004adafaa3700 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe66dffb640 1 -- 172.21.15.33:0/3346694258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668010770 con 0x56029b6e6800 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe66dffb640 1 -- 172.21.15.33:0/3346694258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668010dc0 con 0x56029b6e6800 2024-04-23T18:32:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.553+0000 7fe66dffb640 1 -- 172.21.15.33:0/3346694258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66801ec70 con 0x56029b6e6800 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- 172.21.15.33:0/3346694258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 msgr2=0x56029b6e8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe668012950 tx=0x7fe668009710 comp rx=0 tx=0).stop 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- 172.21.15.33:0/3346694258 shutdown_connections 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e8bf0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 --2- 172.21.15.33:0/3346694258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- 172.21.15.33:0/3346694258 >> 172.21.15.33:0/3346694258 conn(0x56029b6d7b30 msgr2=0x56029b6d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- 172.21.15.33:0/3346694258 shutdown_connections 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- 172.21.15.33:0/3346694258 wait complete. 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.554+0000 7fe6805a2740 1 -- start start 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 Processor -- start 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6e6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56029b6e9430 con 0x56029b6e6800 2024-04-23T18:32:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56029b5db0b0 con 0x56029b6dbf40 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56029b6e95b0 con 0x56029b6db620 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6805a2740 10 client.0 _ll_register_callbacks cb 0x56029b5c8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47278/0 (socket says 172.21.15.33:47278) 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 -- 172.21.15.33:0/2178992621 learned_addr learned my addr 172.21.15.33:0/2178992621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe676ffd640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6e6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6767fc640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 -- 172.21.15.33:0/2178992621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 msgr2=0x56029b6e6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6e6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 -- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 msgr2=0x56029b6e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6777fe640 1 -- 172.21.15.33:0/2178992621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029b41f620 con 0x56029b6e6800 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.555+0000 7fe6767fc640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe6777fe640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe668002230 tx=0x7fe66801e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fded4843d314fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668004c50 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7fe668017040 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe668010dc0 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe668004df0 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe6805a2740 10 client.16461 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56029b6f53a0 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe668010dc0 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.556+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe66800c070 con 0x56029b6e6800 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 10 client.16461 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 20 client.16461 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b6f67a0 0x56029b6f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 4 client.16461 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56029b64e390 con 0x56029b6f67a0 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe676ffd640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b6f67a0 0x56029b6f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe676ffd640 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b6f67a0 0x56029b6f8b90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91606010570696ef server_cookie=3e7d068b8a7e4bde in_seq=0 out_seq=0 2024-04-23T18:32:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.557+0000 7fe656ffd640 10 client.16461 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe656ffd640 1 -- 172.21.15.33:0/2178992621 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56029b64e390 con 0x56029b6f67a0 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe656ffd640 10 client.16461 handle_command_reply: tid=0 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 client.16461 shutdown 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 msgr2=0x56029b6e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe668002230 tx=0x7fe66801e600 comp rx=0 tx=0).stop 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 shutdown_connections 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b6f67a0 0x56029b6f8b90 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6e6800 0x56029b6e39c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6dbf40 0x56029b6e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe656ffd640 0 client.16461 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2178992621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6e6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 >> 172.21.15.33:0/2178992621 conn(0x56029b6d7b30 msgr2=0x56029b6dd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 shutdown_connections 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 1 -- 172.21.15.33:0/2178992621 wait complete. 2024-04-23T18:32:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.558+0000 7fe6805a2740 20 client.16461 trim_cache size 0 max 16384 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 Processor -- start 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 -- start start 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 0x56029b6dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781adb70 con 0x56029b6dbf40 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781af6b0 con 0x56029b6e6800 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6805a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ae240 con 0x56029b6db620 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47288/0 (socket says 172.21.15.33:47288) 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 -- 172.21.15.33:0/2399222617 learned_addr learned my addr 172.21.15.33:0/2399222617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe676ffd640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6777fe640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 0x56029b6dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 -- 172.21.15.33:0/2399222617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 msgr2=0x56029b6d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 -- 172.21.15.33:0/2399222617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 msgr2=0x56029b6dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 0x56029b6dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 -- 172.21.15.33:0/2399222617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56029b6d8d60 con 0x56029b6dbf40 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6777fe640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 0x56029b6dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.574+0000 7fe6767fc640 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b97c0 tx=0x7fe6600bc930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200607c957babf3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe66d7fa640 1 -- 172.21.15.33:0/2399222617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660049070 con 0x56029b6dbf40 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe66d7fa640 1 -- 172.21.15.33:0/2399222617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6600bcdf0 con 0x56029b6dbf40 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe66d7fa640 1 -- 172.21.15.33:0/2399222617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6600c11a0 con 0x56029b6dbf40 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 -- 172.21.15.33:0/2399222617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 msgr2=0x56029b6e25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe6600b97c0 tx=0x7fe6600bc930 comp rx=0 tx=0).stop 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 -- 172.21.15.33:0/2399222617 shutdown_connections 2024-04-23T18:32:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6e6800 0x56029b6dd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e25c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 --2- 172.21.15.33:0/2399222617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6db620 0x56029b6d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 -- 172.21.15.33:0/2399222617 >> 172.21.15.33:0/2399222617 conn(0x56029b6d7b30 msgr2=0x56029b6dcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 -- 172.21.15.33:0/2399222617 shutdown_connections 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.575+0000 7fe6805a2740 1 -- 172.21.15.33:0/2399222617 wait complete. 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 -- start start 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 Processor -- start 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6db620 0x56029b6e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6e6800 0x56029b75aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781aefe0 con 0x56029b6dbf40 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781ae910 con 0x56029b6db620 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781abb50 con 0x56029b6e6800 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6805a2740 10 client.0 _ll_register_callbacks cb 0x7fe668010f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47304/0 (socket says 172.21.15.33:47304) 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe6767fc640 1 -- 172.21.15.33:0/53010977 learned_addr learned my addr 172.21.15.33:0/53010977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.576+0000 7fe676ffd640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6db620 0x56029b6e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6777fe640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6e6800 0x56029b75aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 -- 172.21.15.33:0/53010977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6e6800 msgr2=0x56029b75aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6e6800 0x56029b75aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 -- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6db620 msgr2=0x56029b6e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6db620 0x56029b6e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 -- 172.21.15.33:0/53010977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56029b75aff0 con 0x56029b6dbf40 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6767fc640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe6600eeba0 tx=0x7fe66003ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351722872f5c0b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660049070 con 0x56029b6dbf40 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7fe6600b9460 con 0x56029b6dbf40 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6600bcdf0 con 0x56029b6dbf40 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6600c5a60 con 0x56029b6dbf40 2024-04-23T18:32:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 10 client.16467 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56029b75c210 con 0x56029b6dbf40 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6600bcdf0 con 0x56029b6dbf40 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fe6600c55d0 con 0x56029b6dbf40 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 10 client.16467 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.577+0000 7fe6805a2740 20 client.16467 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b75c850 0x56029b75ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe6805a2740 4 client.16467 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56029b648200 con 0x56029b75c850 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe676ffd640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b75c850 0x56029b75ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe676ffd640 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b75c850 0x56029b75ec70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c2f17b43ebe963d server_cookie=7173cf848eac866b in_seq=0 out_seq=0 2024-04-23T18:32:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.578+0000 7fe656ffd640 10 client.16467 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:42.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.582+0000 7fe656ffd640 1 -- 172.21.15.33:0/53010977 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56029b648200 con 0x56029b75c850 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.582+0000 7fe656ffd640 10 client.16467 handle_command_reply: tid=0 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.582+0000 7fe6805a2740 1 client.16467 shutdown 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 msgr2=0x56029b6e9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe6600eeba0 tx=0x7fe66003ba50 comp rx=0 tx=0).stop 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 shutdown_connections 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56029b75c850 0x56029b75ec70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56029b6e6800 0x56029b75aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56029b6dbf40 0x56029b6e9ef0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe656ffd640 0 client.16467 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 --2- 172.21.15.33:0/53010977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56029b6db620 0x56029b6e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 >> 172.21.15.33:0/53010977 conn(0x56029b6d7b30 msgr2=0x56029b6dd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 shutdown_connections 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 1 -- 172.21.15.33:0/53010977 wait complete. 2024-04-23T18:32:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe6805a2740 20 client.16467 trim_cache size 0 max 16384 2024-04-23T18:32:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:32:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:32:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:32:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 msgr2=0x7fe6640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe670009bd0 tx=0x7fe670005f90 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 msgr2=0x7fe678198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe660005940 tx=0x7fe660004270 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 shutdown_connections 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe67810c520 0x7fe67819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe678103b00 0x7fe678198c50 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.583+0000 7fe67d88a640 1 --2- 172.21.15.33:0/3942068568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781031e0 0x7fe678198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.584+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 >> 172.21.15.33:0/3942068568 conn(0x7fe678077fd0 msgr2=0x7fe6780fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.584+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 shutdown_connections 2024-04-23T18:32:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:42.584+0000 7fe67d88a640 1 -- 172.21.15.33:0/3942068568 wait complete. 2024-04-23T18:32:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:42 smithi106 ceph-mon[32716]: pgmap v191: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-23T18:32:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:42 smithi122 ceph-mon[32546]: pgmap v191: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-23T18:32:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:42 smithi033 ceph-mon[29274]: pgmap v191: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 17 op/s 2024-04-23T18:32:45.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:44 smithi106 ceph-mon[32716]: pgmap v192: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:32:45.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:44 smithi122 ceph-mon[32546]: pgmap v192: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:32:45.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:44 smithi033 ceph-mon[29274]: pgmap v192: 97 pgs: 97 active+clean; 4.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-23T18:32:47.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:46 smithi106 ceph-mon[32716]: pgmap v193: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:32:47.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:46 smithi122 ceph-mon[32546]: pgmap v193: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:32:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:46 smithi033 ceph-mon[29274]: pgmap v193: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 19 KiB/s wr, 18 op/s 2024-04-23T18:32:49.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:48 smithi106 ceph-mon[32716]: pgmap v194: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:49.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:48 smithi122 ceph-mon[32546]: pgmap v194: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:49.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:48 smithi033 ceph-mon[29274]: pgmap v194: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:51.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:50 smithi106 ceph-mon[32716]: pgmap v195: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:51.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:50 smithi122 ceph-mon[32546]: pgmap v195: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:51.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:50 smithi033 ceph-mon[29274]: pgmap v195: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:53.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:52 smithi106 ceph-mon[32716]: pgmap v196: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-23T18:32:53.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:52 smithi122 ceph-mon[32546]: pgmap v196: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-23T18:32:53.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:52 smithi033 ceph-mon[29274]: pgmap v196: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-23T18:32:55.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:54 smithi106 ceph-mon[32716]: pgmap v197: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:55.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:54 smithi122 ceph-mon[32546]: pgmap v197: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:55.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:54 smithi033 ceph-mon[29274]: pgmap v197: 97 pgs: 97 active+clean; 4.3 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 13 op/s 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/768471811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68102b70 msgr2=0x7f5b68106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/768471811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68102b70 0x7f5b68106bb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009900 tx=0x7f5b5c02a410 comp rx=0 tx=0).stop 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/768471811 shutdown_connections 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/768471811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b68107290 0x7f5b6810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/768471811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b68102b70 0x7f5b68106bb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/768471811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b681021c0 0x7f5b681025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/768471811 >> 172.21.15.33:0/768471811 conn(0x7f5b680fda20 msgr2=0x7f5b680ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/768471811 shutdown_connections 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/768471811 wait complete. 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.682+0000 7f5b6e0e1640 1 Processor -- start 2024-04-23T18:32:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 -- start start 2024-04-23T18:32:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68107290 0x7f5b68198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6819cf20 0x7f5b6819d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112220 con 0x7f5b681021c0 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681120a0 con 0x7f5b6819cf20 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b6e0e1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681123a0 con 0x7f5b68107290 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60746/0 (socket says 172.21.15.33:60746) 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 -- 172.21.15.33:0/1224987099 learned_addr learned my addr 172.21.15.33:0/1224987099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68107290 0x7f5b68198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6819cf20 0x7f5b6819d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 -- 172.21.15.33:0/1224987099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68107290 msgr2=0x7f5b68198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68107290 0x7f5b68198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 -- 172.21.15.33:0/1224987099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6819cf20 msgr2=0x7f5b6819d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6819cf20 0x7f5b6819d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.683+0000 7f5b677fe640 1 -- 172.21.15.33:0/1224987099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819da10 con 0x7f5b681021c0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800ea80 tx=0x7f5b5800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b64ff9640 1 -- 172.21.15.33:0/1224987099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800ce40 con 0x7f5b681021c0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b64ff9640 1 -- 172.21.15.33:0/1224987099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b58005020 con 0x7f5b681021c0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6819dd00 con 0x7f5b681021c0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b64ff9640 1 -- 172.21.15.33:0/1224987099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580116d0 con 0x7f5b681021c0 2024-04-23T18:32:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.684+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a54c0 con 0x7f5b681021c0 2024-04-23T18:32:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.686+0000 7f5b64ff9640 1 -- 172.21.15.33:0/1224987099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b580054d0 con 0x7f5b681021c0 2024-04-23T18:32:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.686+0000 7f5b70df7740 1 Processor -- start 2024-04-23T18:32:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 -- start start 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b64ff9640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988eb2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988ebd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b988e17970 con 0x55b988ebd800 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b988db44a0 con 0x55b988eb2f40 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b70df7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b988e13930 con 0x55b988eb2620 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b64ff9640 1 -- 172.21.15.33:0/1224987099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5b580d0a50 con 0x7f5b681021c0 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60754/0 (socket says 172.21.15.33:60754) 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 -- 172.21.15.33:0/2078983554 learned_addr learned my addr 172.21.15.33:0/2078983554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988ebd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b677fe640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988eb2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 -- 172.21.15.33:0/2078983554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 msgr2=0x55b988eb2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988eb2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 -- 172.21.15.33:0/2078983554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 msgr2=0x55b988ebd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988ebd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b67fff640 1 -- 172.21.15.33:0/2078983554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b988b2a210 con 0x55b988ebd800 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b677fe640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988eb2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.687+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988ebd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b600138d0 tx=0x7f5b6000baa0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b67fff640 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010c10 tx=0x7f5b6001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0a03066e74f9af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b4dffb640 1 -- 172.21.15.33:0/2078983554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000bdf0 con 0x55b988ebd800 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b4dffb640 1 -- 172.21.15.33:0/2078983554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b600130d0 con 0x55b988ebd800 2024-04-23T18:32:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b4dffb640 1 -- 172.21.15.33:0/2078983554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60005020 con 0x55b988ebd800 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b70df7740 1 -- 172.21.15.33:0/2078983554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 msgr2=0x55b988ebfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.688+0000 7f5b70df7740 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010c10 tx=0x7f5b6001a000 comp rx=0 tx=0).stop 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- 172.21.15.33:0/2078983554 shutdown_connections 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ebfbf0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988ebd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- 172.21.15.33:0/2078983554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988eb2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- 172.21.15.33:0/2078983554 >> 172.21.15.33:0/2078983554 conn(0x55b988eaeb30 msgr2=0x55b988eaef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- 172.21.15.33:0/2078983554 shutdown_connections 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- 172.21.15.33:0/2078983554 wait complete. 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- start start 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 Processor -- start 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2f40 0x55b988eba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ebaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b988ec03d0 con 0x55b988eb2620 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b988db20b0 con 0x55b988ebd800 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.689+0000 7f5b70df7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b988ec0550 con 0x55b988eb2f40 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b70df7740 10 client.0 _ll_register_callbacks cb 0x55b988d9fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60766/0 (socket says 172.21.15.33:60766) 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 -- 172.21.15.33:0/1866395242 learned_addr learned my addr 172.21.15.33:0/1866395242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2f40 0x55b988eba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ebaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 -- 172.21.15.33:0/1866395242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2f40 msgr2=0x55b988eba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2f40 0x55b988eba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 -- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 msgr2=0x55b988ebaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ebaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 -- 172.21.15.33:0/1866395242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b988bf6620 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ebaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b677fe640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b580ce210 tx=0x7f5b5809c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155f3884e772bd05 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5801f040 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5b580a51e0 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b600130d0 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b5809ca00 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b70df7740 10 client.16479 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.690+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b988ecc3e0 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b600130d0 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b5800f040 con 0x55b988eb2620 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 10 client.16479 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 20 client.16479 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988ecd850 0x55b988ecfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 4 client.16479 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b988e25390 con 0x55b988ecd850 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.691+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988ecd850 0x55b988ecfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.692+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988ecd850 0x55b988ecfc40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51a776b64c54e88 server_cookie=9d445daf71b8e0a in_seq=0 out_seq=0 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.692+0000 7f5b46ffd640 10 client.16479 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.692+0000 7f5b46ffd640 1 -- 172.21.15.33:0/1866395242 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b988e25390 con 0x55b988ecd850 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.692+0000 7f5b46ffd640 10 client.16479 handle_command_reply: tid=0 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.692+0000 7f5b70df7740 1 client.16479 shutdown 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 msgr2=0x55b988ebd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5b580ce210 tx=0x7f5b5809c650 comp rx=0 tx=0).stop 2024-04-23T18:32:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 shutdown_connections 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988ecd850 0x55b988ecfc40 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ebaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2f40 0x55b988eba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b46ffd640 0 client.16479 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1866395242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2620 0x55b988ebd1a0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 >> 172.21.15.33:0/1866395242 conn(0x55b988eaeb30 msgr2=0x55b988eb48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 shutdown_connections 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 1 -- 172.21.15.33:0/1866395242 wait complete. 2024-04-23T18:32:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.693+0000 7f5b70df7740 20 client.16479 trim_cache size 0 max 16384 2024-04-23T18:32:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 Processor -- start 2024-04-23T18:32:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 -- start start 2024-04-23T18:32:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988eadb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681ad990 con 0x55b988ebd800 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681af4d0 con 0x55b988eb2f40 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b70df7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681ae060 con 0x55b988eb2620 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.708+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60774/0 (socket says 172.21.15.33:60774) 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:49660/0 (socket says 172.21.15.33:49660) 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 -- 172.21.15.33:0/1805190180 learned_addr learned my addr 172.21.15.33:0/1805190180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988eadb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 -- 172.21.15.33:0/1805190180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 msgr2=0x55b988ec9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 -- 172.21.15.33:0/1805190180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 msgr2=0x55b988eadb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988eadb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 -- 172.21.15.33:0/1805190180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b988eb9440 con 0x55b988ebd800 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988eadb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b67fff640 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010c10 tx=0x7f5b6000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d1895d3dddeb21 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b4d7fa640 1 -- 172.21.15.33:0/1805190180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000bdf0 con 0x55b988ebd800 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b4d7fa640 1 -- 172.21.15.33:0/1805190180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6000ec20 con 0x55b988ebd800 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.709+0000 7f5b4d7fa640 1 -- 172.21.15.33:0/1805190180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600035f0 con 0x55b988ebd800 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- 172.21.15.33:0/1805190180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 msgr2=0x55b988ed5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5b60010c10 tx=0x7f5b6000b7f0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- 172.21.15.33:0/1805190180 shutdown_connections 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988ebd800 0x55b988ed5540 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988eb2f40 0x55b988eadb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 --2- 172.21.15.33:0/1805190180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- 172.21.15.33:0/1805190180 >> 172.21.15.33:0/1805190180 conn(0x55b988eaeb30 msgr2=0x55b988ead420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- 172.21.15.33:0/1805190180 shutdown_connections 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- 172.21.15.33:0/1805190180 wait complete. 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 -- start start 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.710+0000 7f5b70df7740 1 Processor -- start 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ec35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681aee00 con 0x55b988eb2f40 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681ae730 con 0x55b988ebd800 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681ab970 con 0x55b988eb2620 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b70df7740 10 client.0 _ll_register_callbacks cb 0x7f5b5809d100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60780/0 (socket says 172.21.15.33:60780) 2024-04-23T18:32:55.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 -- 172.21.15.33:0/581004885 learned_addr learned my addr 172.21.15.33:0/581004885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b67fff640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ec35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b677fe640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 -- 172.21.15.33:0/581004885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 msgr2=0x55b988ec5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 -- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 msgr2=0x55b988ec35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ec35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 -- 172.21.15.33:0/581004885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b988eb2d70 con 0x55b988eb2f40 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b677fe640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b66ffd640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806b5f0 tx=0x7f5b5c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640855902e331a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.711+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c049070 con 0x55b988eb2f40 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6000ec20 con 0x55b988eb2f40 2024-04-23T18:32:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f5b5c0053b0 con 0x55b988eb2f40 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b5c055050 con 0x55b988eb2f40 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 10 client.16485 fetch_fsmap learned FSMap version 33 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b988f381a0 con 0x55b988eb2f40 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b6000ec20 con 0x55b988eb2f40 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f5b5c004980 con 0x55b988eb2f40 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 10 client.16485 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 20 client.16485 populate_metadata read hostname 'smithi033' 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988f38920 0x55b988f3ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 4 client.16485 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.712+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b988e1f200 con 0x55b988f38920 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.713+0000 7f5b677fe640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988f38920 0x55b988f3ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.713+0000 7f5b677fe640 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988f38920 0x55b988f3ad40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44647df2b180cc59 server_cookie=4c9a92babdc52e95 in_seq=0 out_seq=0 2024-04-23T18:32:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.713+0000 7f5b46ffd640 10 client.16485 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b46ffd640 1 -- 172.21.15.33:0/581004885 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b988e1f200 con 0x55b988f38920 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b46ffd640 10 client.16485 handle_command_reply: tid=0 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 client.16485 shutdown 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 msgr2=0x55b988ec2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5b6806b5f0 tx=0x7f5b5c005040 comp rx=0 tx=0).stop 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 shutdown_connections 2024-04-23T18:32:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b988f38920 0x55b988f3ad40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b988ebd800 0x55b988ec35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b988eb2f40 0x55b988ec2fe0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b46ffd640 0 client.16485 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 --2- 172.21.15.33:0/581004885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b988eb2620 0x55b988ec5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 >> 172.21.15.33:0/581004885 conn(0x55b988eaeb30 msgr2=0x55b988eadcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 shutdown_connections 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 1 -- 172.21.15.33:0/581004885 wait complete. 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b70df7740 20 client.16485 trim_cache size 0 max 16384 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:32:55.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:32:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:32:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.721+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 msgr2=0x7f5b500a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b600138d0 tx=0x7f5b6000baa0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 msgr2=0x7f5b68198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800ea80 tx=0x7f5b5800ef40 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 shutdown_connections 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f5b500a3e40 0x7f5b500a6300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5b6819cf20 0x7f5b6819d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b68107290 0x7f5b68198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 --2- 172.21.15.33:0/1224987099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b681021c0 0x7f5b68198660 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 >> 172.21.15.33:0/1224987099 conn(0x7f5b680fda20 msgr2=0x7f5b680fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 shutdown_connections 2024-04-23T18:32:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:32:55.722+0000 7f5b6e0e1640 1 -- 172.21.15.33:0/1224987099 wait complete. 2024-04-23T18:32:57.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:56 smithi106 ceph-mon[32716]: pgmap v198: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-23T18:32:57.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:56 smithi122 ceph-mon[32546]: pgmap v198: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-23T18:32:57.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:56 smithi033 ceph-mon[29274]: pgmap v198: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-23T18:32:59.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:32:58 smithi106 ceph-mon[32716]: pgmap v199: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:32:59.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:32:58 smithi122 ceph-mon[32546]: pgmap v199: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:32:59.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:32:58 smithi033 ceph-mon[29274]: pgmap v199: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:33:01.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:00 smithi106 ceph-mon[32716]: pgmap v200: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:33:01.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:00 smithi122 ceph-mon[32546]: pgmap v200: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:33:01.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:00 smithi033 ceph-mon[29274]: pgmap v200: 97 pgs: 97 active+clean; 4.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-23T18:33:03.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:02 smithi106 ceph-mon[32716]: pgmap v201: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T18:33:03.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:02 smithi122 ceph-mon[32546]: pgmap v201: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T18:33:03.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:02 smithi033 ceph-mon[29274]: pgmap v201: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-23T18:33:05.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:04 smithi106 ceph-mon[32716]: pgmap v202: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:33:05.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:04 smithi122 ceph-mon[32546]: pgmap v202: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:33:05.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:04 smithi033 ceph-mon[29274]: pgmap v202: 97 pgs: 97 active+clean; 4.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-23T18:33:07.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:06 smithi106 ceph-mon[32716]: pgmap v203: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T18:33:07.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:06 smithi122 ceph-mon[32546]: pgmap v203: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T18:33:07.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:06 smithi033 ceph-mon[29274]: pgmap v203: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 72 KiB/s wr, 37 op/s 2024-04-23T18:33:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 -- 172.21.15.33:0/1683954279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d041021a0 msgr2=0x7f1d04102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/1683954279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d041021a0 0x7f1d04102580 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009900 tx=0x7f1cf402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 -- 172.21.15.33:0/1683954279 shutdown_connections 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/1683954279 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 0x7f1d0410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/1683954279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d04102b50 0x7f1d04106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/1683954279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d041021a0 0x7f1d04102580 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 -- 172.21.15.33:0/1683954279 >> 172.21.15.33:0/1683954279 conn(0x7f1d040fda00 msgr2=0x7f1d040ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 -- 172.21.15.33:0/1683954279 shutdown_connections 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.822+0000 7f1d09c4e640 1 -- 172.21.15.33:0/1683954279 wait complete. 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 Processor -- start 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 -- start start 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 0x7f1d041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 0x7f1d04198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041121b0 con 0x7f1d0419cf90 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112030 con 0x7f1d04107270 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d09c4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04112330 con 0x7f1d041021a0 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d03fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d03fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47150/0 (socket says 172.21.15.33:47150) 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d03fff640 1 -- 172.21.15.33:0/2076671159 learned_addr learned my addr 172.21.15.33:0/2076671159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 0x7f1d04198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.823+0000 7f1d037fe640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 0x7f1d041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 -- 172.21.15.33:0/2076671159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 msgr2=0x7f1d041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 0x7f1d041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 -- 172.21.15.33:0/2076671159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 msgr2=0x7f1d04198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 0x7f1d04198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 -- 172.21.15.33:0/2076671159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0419da80 con 0x7f1d0419cf90 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d037fe640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 0x7f1d041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d03fff640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00eaa0 tx=0x7f1cfc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d00ff9640 1 -- 172.21.15.33:0/2076671159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00cdb0 con 0x7f1d0419cf90 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0419dd70 con 0x7f1d0419cf90 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d00ff9640 1 -- 172.21.15.33:0/2076671159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cfc005020 con 0x7f1d0419cf90 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d00ff9640 1 -- 172.21.15.33:0/2076671159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0116d0 con 0x7f1d0419cf90 2024-04-23T18:33:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.824+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a5560 con 0x7f1d0419cf90 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.827+0000 7f1d00ff9640 1 -- 172.21.15.33:0/2076671159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1cfc0051d0 con 0x7f1d0419cf90 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.827+0000 7f1d0c966740 1 Processor -- start 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.827+0000 7f1d0c966740 1 -- start start 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb5ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb68bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a05cac0970 con 0x55a05cb5bf40 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a05ca5d4a0 con 0x55a05cb66800 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d0c966740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a05cabc930 con 0x55a05cb5b620 2024-04-23T18:33:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47152/0 (socket says 172.21.15.33:47152) 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 -- 172.21.15.33:0/2560704521 learned_addr learned my addr 172.21.15.33:0/2560704521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb5ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d00ff9640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 0x7f1cec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb68bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 -- 172.21.15.33:0/2560704521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 msgr2=0x55a05cb5ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb5ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 -- 172.21.15.33:0/2560704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 msgr2=0x55a05cb68bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb68bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d03fff640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 0x7f1cec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 -- 172.21.15.33:0/2560704521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05c7d3210 con 0x55a05cb5bf40 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d00ff9640 1 -- 172.21.15.33:0/2076671159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f1cfc0d09f0 con 0x7f1d0419cf90 2024-04-23T18:33:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.828+0000 7f1d037fe640 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009900 tx=0x7f1cf4032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1190f92ec36b6842 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1ce9ffb640 1 -- 172.21.15.33:0/2560704521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf403f070 con 0x55a05cb5bf40 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1ce9ffb640 1 -- 172.21.15.33:0/2560704521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf4004030 con 0x55a05cb5bf40 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d03fff640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 0x7f1cec0a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efd0 tx=0x7f1cfc06e380 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1ce9ffb640 1 -- 172.21.15.33:0/2560704521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4035070 con 0x55a05cb5bf40 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 -- 172.21.15.33:0/2560704521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 msgr2=0x55a05cb662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009900 tx=0x7f1cf4032cd0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 -- 172.21.15.33:0/2560704521 shutdown_connections 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb68bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb662c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 --2- 172.21.15.33:0/2560704521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb5ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 -- 172.21.15.33:0/2560704521 >> 172.21.15.33:0/2560704521 conn(0x55a05cb57b30 msgr2=0x55a05cb57f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 -- 172.21.15.33:0/2560704521 shutdown_connections 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.829+0000 7f1d0c966740 1 -- 172.21.15.33:0/2560704521 wait complete. 2024-04-23T18:33:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 -- start start 2024-04-23T18:33:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 Processor -- start 2024-04-23T18:33:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb63ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a05cb693d0 con 0x55a05cb5bf40 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a05ca5b0b0 con 0x55a05cb66800 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a05cb69550 con 0x55a05cb5b620 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d0c966740 10 client.0 _ll_register_callbacks cb 0x55a05ca48d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47160/0 (socket says 172.21.15.33:47160) 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d037fe640 1 -- 172.21.15.33:0/124572783 learned_addr learned my addr 172.21.15.33:0/124572783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.830+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb63ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 -- 172.21.15.33:0/124572783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 msgr2=0x55a05cb661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 -- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 msgr2=0x55a05cb63ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb63ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 -- 172.21.15.33:0/124572783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05c89f620 con 0x55a05cb5bf40 2024-04-23T18:33:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d037fe640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4000c00 tx=0x7f1cf40041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6ac76906ec325a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4035070 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f1cf40316a0 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf4004030 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1cf4055050 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d0c966740 10 client.16497 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.831+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a05cb75380 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf4004030 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1cf40047e0 con 0x55a05cb5bf40 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 10 client.16497 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 20 client.16497 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cb766d0 0x55a05cb78ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 4 client.16497 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a05cace390 con 0x55a05cb766d0 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cb766d0 0x55a05cb78ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1ce2ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.832+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cb766d0 0x55a05cb78ac0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3237e765215498 server_cookie=983e6240ee52117f in_seq=0 out_seq=0 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/124572783 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a05cace390 con 0x55a05cb766d0 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1ce2ffd640 10 client.16497 handle_command_reply: tid=0 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 client.16497 shutdown 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 msgr2=0x55a05cb634d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4000c00 tx=0x7f1cf40041e0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 shutdown_connections 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cb766d0 0x55a05cb78ac0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb66800 0x55a05cb63ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb634d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 --2- 172.21.15.33:0/124572783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb5b620 0x55a05cb661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1ce2ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.833+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 >> 172.21.15.33:0/124572783 conn(0x55a05cb57b30 msgr2=0x55a05cb56cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.834+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 shutdown_connections 2024-04-23T18:33:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.834+0000 7f1d0c966740 1 -- 172.21.15.33:0/124572783 wait complete. 2024-04-23T18:33:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.834+0000 7f1d0c966740 20 client.16497 trim_cache size 0 max 16384 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 Processor -- start 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 -- start start 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5bf40 0x55a05cb57f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041adad0 con 0x55a05cb5b620 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041af610 con 0x55a05cb5bf40 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d0c966740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041ae1a0 con 0x55a05cb66800 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47168/0 (socket says 172.21.15.33:47168) 2024-04-23T18:33:08.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 -- 172.21.15.33:0/599993492 learned_addr learned my addr 172.21.15.33:0/599993492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d037fe640 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5bf40 0x55a05cb57f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 -- 172.21.15.33:0/599993492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 msgr2=0x55a05cb5d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb5d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 -- 172.21.15.33:0/599993492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5bf40 msgr2=0x55a05cb57f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5bf40 0x55a05cb57f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.849+0000 7f1d02ffd640 1 -- 172.21.15.33:0/599993492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a05cb63e50 con 0x55a05cb5b620 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800bc90 tx=0x7f1cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9959ee1b591657a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1ce97fa640 1 -- 172.21.15.33:0/599993492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8012dd0 con 0x55a05cb5b620 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1ce97fa640 1 -- 172.21.15.33:0/599993492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf8025e50 con 0x55a05cb5b620 2024-04-23T18:33:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1ce97fa640 1 -- 172.21.15.33:0/599993492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf80051a0 con 0x55a05cb5b620 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 -- 172.21.15.33:0/599993492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 msgr2=0x55a05cb72290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1cf800bc90 tx=0x7f1cf80046c0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 -- 172.21.15.33:0/599993492 shutdown_connections 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb5d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5bf40 0x55a05cb57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 --2- 172.21.15.33:0/599993492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5b620 0x55a05cb72290 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 -- 172.21.15.33:0/599993492 >> 172.21.15.33:0/599993492 conn(0x55a05cb57b30 msgr2=0x55a05cb62d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 -- 172.21.15.33:0/599993492 shutdown_connections 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.850+0000 7f1d0c966740 1 -- 172.21.15.33:0/599993492 wait complete. 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 -- start start 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 Processor -- start 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 0x55a05cb6e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041aef40 con 0x55a05cb5bf40 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041ae870 con 0x55a05cb5b620 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041abad0 con 0x55a05cb66800 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d0c966740 10 client.0 _ll_register_callbacks cb 0x7f1cf4032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47174/0 (socket says 172.21.15.33:47174) 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d037fe640 1 -- 172.21.15.33:0/31597871 learned_addr learned my addr 172.21.15.33:0/31597871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 0x55a05cb6e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.851+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 -- 172.21.15.33:0/31597871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 msgr2=0x55a05cb602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 -- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 msgr2=0x55a05cb6e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 0x55a05cb6e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 -- 172.21.15.33:0/31597871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a05cb607e0 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 0x55a05cb6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d037fe640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1cf402a930 tx=0x7f1cf4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aababfa6ea13282 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4035070 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8498+0+0 (secure 0 0 0) 0x7f1cf4032030 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf8025e50 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1cf4066050 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 10 client.16503 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a05cbdc650 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf8025e50 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f1cf4061030 con 0x55a05cb5bf40 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 10 client.16503 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 20 client.16503 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cbdcd30 0x55a05cbdf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.852+0000 7f1d0c966740 4 client.16503 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.853+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a05cac8200 con 0x55a05cbdcd30 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.853+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cbdcd30 0x55a05cbdf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.853+0000 7f1ce2ffd640 10 client.16503 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.853+0000 7f1d02ffd640 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cbdcd30 0x55a05cbdf150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aaa185a7fa21922 server_cookie=fa25a49c12213bd1 in_seq=0 out_seq=0 2024-04-23T18:33:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1ce2ffd640 1 -- 172.21.15.33:0/31597871 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a05cac8200 con 0x55a05cbdcd30 2024-04-23T18:33:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1ce2ffd640 10 client.16503 handle_command_reply: tid=0 2024-04-23T18:33:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 client.16503 shutdown 2024-04-23T18:33:08.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 msgr2=0x55a05cb5fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1cf402a930 tx=0x7f1cf4005080 comp rx=0 tx=0).stop 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 shutdown_connections 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a05cbdcd30 0x55a05cbdf150 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a05cb66800 0x55a05cb602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a05cb5bf40 0x55a05cb5fd60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1ce2ffd640 0 client.16503 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 --2- 172.21.15.33:0/31597871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a05cb5b620 0x55a05cb6e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 >> 172.21.15.33:0/31597871 conn(0x55a05cb57b30 msgr2=0x55a05cb5e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 shutdown_connections 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.861+0000 7f1d0c966740 1 -- 172.21.15.33:0/31597871 wait complete. 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d0c966740 20 client.16503 trim_cache size 0 max 16384 2024-04-23T18:33:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 msgr2=0x7f1cec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 0x7f1cec0a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00efd0 tx=0x7f1cfc06e380 comp rx=0 tx=0).stop 2024-04-23T18:33:08.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 msgr2=0x7f1d0419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00eaa0 tx=0x7f1cfc00ef60 comp rx=0 tx=0).stop 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 shutdown_connections 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1cec0a3dc0 0x7f1cec0a6280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d0419cf90 0x7f1d0419d440 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d04107270 0x7f1d04198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 --2- 172.21.15.33:0/2076671159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d041021a0 0x7f1d041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 >> 172.21.15.33:0/2076671159 conn(0x7f1d040fda00 msgr2=0x7f1d040fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 shutdown_connections 2024-04-23T18:33:08.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:08.862+0000 7f1d09c4e640 1 -- 172.21.15.33:0/2076671159 wait complete. 2024-04-23T18:33:09.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:08 smithi122 ceph-mon[32546]: pgmap v204: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 31 op/s 2024-04-23T18:33:09.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:08 smithi033 ceph-mon[29274]: pgmap v204: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 31 op/s 2024-04-23T18:33:09.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:08 smithi106 ceph-mon[32716]: pgmap v204: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 64 KiB/s wr, 31 op/s 2024-04-23T18:33:10.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:09 smithi122 ceph-mon[32546]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:10.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:09 smithi033 ceph-mon[29274]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:10.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:09 smithi106 ceph-mon[32716]: osdmap e91: 12 total, 12 up, 12 in 2024-04-23T18:33:11.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:10 smithi122 ceph-mon[32546]: pgmap v206: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:33:11.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:10 smithi033 ceph-mon[29274]: pgmap v206: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:33:11.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:10 smithi106 ceph-mon[32716]: pgmap v206: 97 pgs: 97 active+clean; 4.6 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:33:13.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:12 smithi122 ceph-mon[32546]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:13.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:12 smithi033 ceph-mon[29274]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:13.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:12 smithi106 ceph-mon[32716]: pgmap v207: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:15.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:14 smithi122 ceph-mon[32546]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:15.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:14 smithi033 ceph-mon[29274]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:15.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:14 smithi106 ceph-mon[32716]: pgmap v208: 97 pgs: 97 active+clean; 4.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 80 KiB/s wr, 38 op/s 2024-04-23T18:33:16.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:16 smithi033 ceph-mon[29274]: pgmap v209: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:16.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:16 smithi033 ceph-mon[29274]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:33:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:16 smithi106 ceph-mon[32716]: pgmap v209: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:17.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:16 smithi106 ceph-mon[32716]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:33:17.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:16 smithi122 ceph-mon[32546]: pgmap v209: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:17.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:16 smithi122 ceph-mon[32546]: osdmap e92: 12 total, 12 up, 12 in 2024-04-23T18:33:19.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:18 smithi106 ceph-mon[32716]: pgmap v211: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:19.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:18 smithi122 ceph-mon[32546]: pgmap v211: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:19.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:18 smithi033 ceph-mon[29274]: pgmap v211: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:21.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:20 smithi106 ceph-mon[32716]: pgmap v212: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:21.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:20 smithi122 ceph-mon[32546]: pgmap v212: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:21.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:20 smithi033 ceph-mon[29274]: pgmap v212: 97 pgs: 97 active+clean; 4.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-23T18:33:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 -- 172.21.15.33:0/890165670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 msgr2=0x7f38dc10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/890165670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc10ee60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009ca0 tx=0x7f38d402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 -- 172.21.15.33:0/890165670 shutdown_connections 2024-04-23T18:33:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/890165670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc10ee60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/890165670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc102b70 0x7f38dc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/890165670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 -- 172.21.15.33:0/890165670 >> 172.21.15.33:0/890165670 conn(0x7f38dc0fda20 msgr2=0x7f38dc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 -- 172.21.15.33:0/890165670 shutdown_connections 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 -- 172.21.15.33:0/890165670 wait complete. 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.962+0000 7f38e3ea0640 1 Processor -- start 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 -- start start 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 0x7f38dc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112200 con 0x7f38dc107290 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112080 con 0x7f38dc19cf10 2024-04-23T18:33:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e3ea0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc112380 con 0x7f38dc1021c0 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59656/0 (socket says 172.21.15.33:59656) 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 -- 172.21.15.33:0/2906996419 learned_addr learned my addr 172.21.15.33:0/2906996419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e243d640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 0x7f38dc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 -- 172.21.15.33:0/2906996419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 msgr2=0x7f38dc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 -- 172.21.15.33:0/2906996419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 msgr2=0x7f38dc19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 0x7f38dc19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 -- 172.21.15.33:0/2906996419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc19da00 con 0x7f38dc107290 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e243d640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 0x7f38dc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.963+0000 7f38e143b640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e9c0 tx=0x7f38cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.964+0000 7f38d2ffd640 1 -- 172.21.15.33:0/2906996419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00cd80 con 0x7f38dc107290 2024-04-23T18:33:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.964+0000 7f38d2ffd640 1 -- 172.21.15.33:0/2906996419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38cc0054d0 con 0x7f38dc107290 2024-04-23T18:33:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.964+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc19dcf0 con 0x7f38dc107290 2024-04-23T18:33:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.964+0000 7f38d2ffd640 1 -- 172.21.15.33:0/2906996419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc010430 con 0x7f38dc107290 2024-04-23T18:33:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.964+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1a5530 con 0x7f38dc107290 2024-04-23T18:33:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.966+0000 7f38d2ffd640 1 -- 172.21.15.33:0/2906996419 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f38cc004fd0 con 0x7f38dc107290 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.966+0000 7f38e6bb8740 1 Processor -- start 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 -- start start 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 0x55a17c6a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 0x55a17c6a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38d2ffd640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 0x7f38c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a17c601970 con 0x55a17c69c620 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a17c59e4a0 con 0x55a17c69cf40 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a17c5fd930 con 0x55a17c6a7800 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59662/0 (socket says 172.21.15.33:59662) 2024-04-23T18:33:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 -- 172.21.15.33:0/1050597634 learned_addr learned my addr 172.21.15.33:0/1050597634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 0x55a17c6a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 0x7f38c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38d2ffd640 1 -- 172.21.15.33:0/2906996419 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f38cc0d0cc0 con 0x7f38dc107290 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e143b640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 0x55a17c6a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 -- 172.21.15.33:0/1050597634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 msgr2=0x55a17c6a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 0x55a17c6a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 -- 172.21.15.33:0/1050597634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 msgr2=0x55a17c6a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 0x55a17c6a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e243d640 1 -- 172.21.15.33:0/1050597634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a17c314210 con 0x55a17c69c620 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e143b640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 0x55a17c6a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.967+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 0x55a17c6a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.968+0000 7f38e243d640 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fa0 tx=0x7f38d4033000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac481f5eb4f2247 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.968+0000 7f38e243d640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 0x7f38c80a6300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38d40314d0 tx=0x7f38d4043000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.968+0000 7f38c7fff640 1 -- 172.21.15.33:0/1050597634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d404e070 con 0x55a17c69c620 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.968+0000 7f38c7fff640 1 -- 172.21.15.33:0/1050597634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d4036040 con 0x55a17c69c620 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.968+0000 7f38c7fff640 1 -- 172.21.15.33:0/1050597634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4004640 con 0x55a17c69c620 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- 172.21.15.33:0/1050597634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 msgr2=0x55a17c69ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fa0 tx=0x7f38d4033000 comp rx=0 tx=0).stop 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- 172.21.15.33:0/1050597634 shutdown_connections 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c6a7800 0x55a17c6a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69cf40 0x55a17c6a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/1050597634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69c620 0x55a17c69ca00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- 172.21.15.33:0/1050597634 >> 172.21.15.33:0/1050597634 conn(0x55a17c698b30 msgr2=0x55a17c698f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- 172.21.15.33:0/1050597634 shutdown_connections 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- 172.21.15.33:0/1050597634 wait complete. 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 -- start start 2024-04-23T18:33:21.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.969+0000 7f38e6bb8740 1 Processor -- start 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 0x55a17c6a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c6a7800 0x55a17c6a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a17c6aa3d0 con 0x55a17c69cf40 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a17c59c0b0 con 0x55a17c6a7800 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a17c6aa550 con 0x55a17c69c620 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e6bb8740 10 client.0 _ll_register_callbacks cb 0x55a17c589d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59672/0 (socket says 172.21.15.33:59672) 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 -- 172.21.15.33:0/4204521707 learned_addr learned my addr 172.21.15.33:0/4204521707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e243d640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c6a7800 0x55a17c6a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 0x55a17c6a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 -- 172.21.15.33:0/4204521707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 msgr2=0x55a17c6a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 0x55a17c6a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 -- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c6a7800 msgr2=0x55a17c6a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c6a7800 0x55a17c6a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 -- 172.21.15.33:0/4204521707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a17c3e0620 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 0x55a17c6a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38e143b640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f38cc09cb80 tx=0x7f38cc0b49e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80264863a21e2641 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc09cbf0 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.970+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f38cc0bd0b0 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d4036040 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f38cc0a0fa0 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 10 client.16515 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a17c6b62f0 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d4036040 con 0x55a17c69cf40 2024-04-23T18:33:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f38cc0a53d0 con 0x55a17c69cf40 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 10 client.16515 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 20 client.16515 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c6b77b0 0x55a17c6b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 4 client.16515 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.971+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a17c60f390 con 0x55a17c6b77b0 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.972+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c6b77b0 0x55a17c6b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.972+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c6b77b0 0x55a17c6b9ba0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae69398fc2e1d212 server_cookie=90fabdf7b7b2f04c in_seq=0 out_seq=0 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.972+0000 7f38c57fa640 10 client.16515 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38c57fa640 1 -- 172.21.15.33:0/4204521707 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a17c60f390 con 0x55a17c6b77b0 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38c57fa640 10 client.16515 handle_command_reply: tid=0 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 client.16515 shutdown 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 msgr2=0x55a17c6a44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f38cc09cb80 tx=0x7f38cc0b49e0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 shutdown_connections 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c6b77b0 0x55a17c6b9ba0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c6a7800 0x55a17c6a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c69cf40 0x55a17c6a44e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/4204521707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69c620 0x55a17c6a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38c57fa640 0 client.16515 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 >> 172.21.15.33:0/4204521707 conn(0x55a17c698b30 msgr2=0x55a17c69e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 shutdown_connections 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 1 -- 172.21.15.33:0/4204521707 wait complete. 2024-04-23T18:33:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.973+0000 7f38e6bb8740 20 client.16515 trim_cache size 0 max 16384 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.988+0000 7f38e6bb8740 1 Processor -- start 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.988+0000 7f38e6bb8740 1 -- start start 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1adbe0 con 0x55a17c6a7800 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1af720 con 0x55a17c69c620 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1ae2b0 con 0x55a17c69cf40 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59678/0 (socket says 172.21.15.33:59678) 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 -- 172.21.15.33:0/8720803 learned_addr learned my addr 172.21.15.33:0/8720803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e143b640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 -- 172.21.15.33:0/8720803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 msgr2=0x55a17c6a6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6a6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 -- 172.21.15.33:0/8720803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 msgr2=0x55a17c6b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 -- 172.21.15.33:0/8720803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a17c6a4e60 con 0x55a17c6a7800 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e143b640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.989+0000 7f38e243d640 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fa0 tx=0x7f38d4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cca4c8ed7e4f78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38c77fe640 1 -- 172.21.15.33:0/8720803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d404e070 con 0x55a17c6a7800 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38c77fe640 1 -- 172.21.15.33:0/8720803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d4036050 con 0x55a17c6a7800 2024-04-23T18:33:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38c77fe640 1 -- 172.21.15.33:0/8720803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d402f8c0 con 0x55a17c6a7800 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 -- 172.21.15.33:0/8720803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 msgr2=0x55a17c6bf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005fa0 tx=0x7f38d4009360 comp rx=0 tx=0).stop 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 -- 172.21.15.33:0/8720803 shutdown_connections 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6bf440 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6a6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/8720803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 -- 172.21.15.33:0/8720803 >> 172.21.15.33:0/8720803 conn(0x55a17c698b30 msgr2=0x55a17c697cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 -- 172.21.15.33:0/8720803 shutdown_connections 2024-04-23T18:33:21.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.990+0000 7f38e6bb8740 1 -- 172.21.15.33:0/8720803 wait complete. 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 -- start start 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 Processor -- start 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1af050 con 0x55a17c6a7800 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1ae980 con 0x55a17c69c620 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1abbe0 con 0x55a17c69cf40 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e6bb8740 10 client.0 _ll_register_callbacks cb 0x7f38cc09c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e1c3c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e243d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59694/0 (socket says 172.21.15.33:59694) 2024-04-23T18:33:21.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e243d640 1 -- 172.21.15.33:0/2920935345 learned_addr learned my addr 172.21.15.33:0/2920935345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.991+0000 7f38e143b640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 -- 172.21.15.33:0/2920935345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 msgr2=0x55a17c6af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 -- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 msgr2=0x55a17c6af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 -- 172.21.15.33:0/2920935345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a17c6ac080 con 0x55a17c6a7800 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e243d640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f38d402ade0 tx=0x7f38d4004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c1beed6eb617b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d404e070 con 0x55a17c6a7800 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f38d402f660 con 0x55a17c6a7800 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d4036050 con 0x55a17c6a7800 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f38d406a050 con 0x55a17c6a7800 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e6bb8740 10 client.16521 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a17c722b00 con 0x55a17c6a7800 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d4036050 con 0x55a17c6a7800 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f38d4030890 con 0x55a17c6a7800 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e6bb8740 10 client.16521 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.992+0000 7f38e6bb8740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e6bb8740 20 client.16521 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c722dd0 0x55a17c7251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e6bb8740 4 client.16521 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a17c698980 con 0x55a17c722dd0 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c722dd0 0x55a17c7251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38e1c3c640 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c722dd0 0x55a17c7251c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b67a0e5734b5732 server_cookie=b828c67d195e2139 in_seq=0 out_seq=0 2024-04-23T18:33:21.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:21.993+0000 7f38c57fa640 10 client.16521 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38c57fa640 1 -- 172.21.15.33:0/2920935345 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a17c698980 con 0x55a17c722dd0 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38c57fa640 10 client.16521 handle_command_reply: tid=0 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 client.16521 shutdown 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 msgr2=0x55a17c6ab880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f38d402ade0 tx=0x7f38d4004640 comp rx=0 tx=0).stop 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 shutdown_connections 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a17c722dd0 0x55a17c7251c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a17c6a7800 0x55a17c6ab880 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a17c69cf40 0x55a17c6af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 --2- 172.21.15.33:0/2920935345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a17c69c620 0x55a17c6af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38c57fa640 0 client.16521 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 >> 172.21.15.33:0/2920935345 conn(0x55a17c698b30 msgr2=0x55a17c69dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.001+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 shutdown_connections 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e6bb8740 1 -- 172.21.15.33:0/2920935345 wait complete. 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e6bb8740 20 client.16521 trim_cache size 0 max 16384 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:33:21.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 msgr2=0x7f38c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 0x7f38c80a6300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f38d40314d0 tx=0x7f38d4043000 comp rx=0 tx=0).stop 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 msgr2=0x7f38dc198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00e9c0 tx=0x7f38cc00ee80 comp rx=0 tx=0).stop 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 shutdown_connections 2024-04-23T18:33:21.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f38c80a3e40 0x7f38c80a6300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f38dc19cf10 0x7f38dc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38dc107290 0x7f38dc198be0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 --2- 172.21.15.33:0/2906996419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38dc1021c0 0x7f38dc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 >> 172.21.15.33:0/2906996419 conn(0x7f38dc0fda20 msgr2=0x7f38dc0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 shutdown_connections 2024-04-23T18:33:21.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:22.002+0000 7f38e3ea0640 1 -- 172.21.15.33:0/2906996419 wait complete. 2024-04-23T18:33:23.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:22 smithi106 ceph-mon[32716]: pgmap v213: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:23.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:22 smithi122 ceph-mon[32546]: pgmap v213: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:23.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:22 smithi033 ceph-mon[29274]: pgmap v213: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:25.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:24 smithi106 ceph-mon[32716]: pgmap v214: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:25.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:24 smithi122 ceph-mon[32546]: pgmap v214: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:25.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:24 smithi033 ceph-mon[29274]: pgmap v214: 97 pgs: 97 active+clean; 4.9 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-23T18:33:27.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:26 smithi106 ceph-mon[32716]: pgmap v215: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-23T18:33:27.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:26 smithi122 ceph-mon[32546]: pgmap v215: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-23T18:33:27.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:26 smithi033 ceph-mon[29274]: pgmap v215: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-23T18:33:29.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:28 smithi106 ceph-mon[32716]: pgmap v216: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:33:29.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:28 smithi122 ceph-mon[32546]: pgmap v216: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:33:29.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:28 smithi033 ceph-mon[29274]: pgmap v216: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:33:29.754 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:189/600: open fd = 3 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:190/600: open fd = 3 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:191/600: open fd = 3 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.755 INFO:tasks.workunit.client.0.smithi033.stdout:192/600: open fd = 3 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:193/600: open fd = 3 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:194/600: open fd = 3 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:195/600: open fd = 3 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.756 INFO:tasks.workunit.client.0.smithi033.stdout:196/600: open fd = 3 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:197/600: open fd = 3 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:198/600: open fd = 3 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:199/600: open fd = 3 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.757 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:200/600: open fd = 3 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:201/600: open fd = 3 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:202/600: open fd = 3 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:203/600: open fd = 3 2024-04-23T18:33:29.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:204/600: open fd = 3 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:205/600: open fd = 3 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:206/600: open fd = 3 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.759 INFO:tasks.workunit.client.0.smithi033.stdout:207/600: open fd = 3 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:208/600: open fd = 3 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:209/600: open fd = 3 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:210/600: open fd = 3 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.760 INFO:tasks.workunit.client.0.smithi033.stdout:211/600: open fd = 3 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:212/600: open fd = 3 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:213/600: open fd = 3 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:214/600: open fd = 3 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:215/600: open fd = 3 2024-04-23T18:33:29.761 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:216/600: open fd = 3 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:217/600: open fd = 3 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:218/600: open fd = 3 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.762 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:219/600: open fd = 3 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:220/600: open fd = 3 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:221/600: open fd = 3 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:222/600: open fd = 3 2024-04-23T18:33:29.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:223/600: open fd = 3 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:224/600: open fd = 3 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:225/600: open fd = 3 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.764 INFO:tasks.workunit.client.0.smithi033.stdout:226/600: open fd = 3 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:227/600: open fd = 3 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:228/600: open fd = 3 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:229/600: open fd = 3 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.765 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:230/600: open fd = 3 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:231/600: open fd = 3 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:232/600: open fd = 3 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.766 INFO:tasks.workunit.client.0.smithi033.stdout:233/600: open fd = 3 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:234/600: open fd = 3 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:235/600: open fd = 3 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:236/600: open fd = 3 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.767 INFO:tasks.workunit.client.0.smithi033.stdout:237/600: open fd = 3 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:238/600: open fd = 3 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:239/600: open fd = 3 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:240/600: open fd = 3 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.768 INFO:tasks.workunit.client.0.smithi033.stdout:241/600: open fd = 3 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:242/600: open fd = 3 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:243/600: open fd = 3 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:244/600: open fd = 3 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:245/600: open fd = 3 2024-04-23T18:33:29.769 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:246/600: open fd = 3 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:247/600: open fd = 3 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:248/600: open fd = 3 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.770 INFO:tasks.workunit.client.0.smithi033.stdout:249/600: open fd = 3 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:250/600: open fd = 3 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:29.771 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:33:31.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:30 smithi106 ceph-mon[32716]: pgmap v217: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:30 smithi122 ceph-mon[32546]: pgmap v217: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:31.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:30 smithi033 ceph-mon[29274]: pgmap v217: 97 pgs: 97 active+clean; 5.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:33.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:32 smithi106 ceph-mon[32716]: pgmap v218: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:33.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:32 smithi122 ceph-mon[32546]: pgmap v218: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:33.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:32 smithi033 ceph-mon[29274]: pgmap v218: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 -- 172.21.15.33:0/3866927304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44341016a0 msgr2=0x7f443410e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 --2- 172.21.15.33:0/3866927304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44341016a0 0x7f443410e3f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4424009900 tx=0x7f442402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 -- 172.21.15.33:0/3866927304 shutdown_connections 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 --2- 172.21.15.33:0/3866927304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f443410e930 0x7f4434110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 --2- 172.21.15.33:0/3866927304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44341016a0 0x7f443410e3f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 --2- 172.21.15.33:0/3866927304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 0x7f4434101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 -- 172.21.15.33:0/3866927304 >> 172.21.15.33:0/3866927304 conn(0x7f44340771d0 msgr2=0x7f44340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 -- 172.21.15.33:0/3866927304 shutdown_connections 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.101+0000 7f443c010640 1 -- 172.21.15.33:0/3866927304 wait complete. 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 Processor -- start 2024-04-23T18:33:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 -- start start 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 0x7f44341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44341016a0 0x7f44341a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4434114480 con 0x7f443410e930 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4434114300 con 0x7f44341016a0 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443c010640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4434114600 con 0x7f4434100d80 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.102+0000 7f443a5ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f4439dac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 0x7f44341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37274/0 (socket says 172.21.15.33:37274) 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 -- 172.21.15.33:0/2966667328 learned_addr learned my addr 172.21.15.33:0/2966667328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f44395ab640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44341016a0 0x7f44341a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 -- 172.21.15.33:0/2966667328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 msgr2=0x7f44341a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 0x7f44341a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 -- 172.21.15.33:0/2966667328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44341016a0 msgr2=0x7f44341a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44341016a0 0x7f44341a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 -- 172.21.15.33:0/2966667328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443419afc0 con 0x7f443410e930 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443a5ad640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f442affd640 1 -- 172.21.15.33:0/2966667328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c004d20 con 0x7f443410e930 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f442affd640 1 -- 172.21.15.33:0/2966667328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f442c005020 con 0x7f443410e930 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443419b2b0 con 0x7f443410e930 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f442affd640 1 -- 172.21.15.33:0/2966667328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c0116b0 con 0x7f443410e930 2024-04-23T18:33:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.103+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a75c0 con 0x7f443410e930 2024-04-23T18:33:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.106+0000 7f442affd640 1 -- 172.21.15.33:0/2966667328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f442c0051d0 con 0x7f443410e930 2024-04-23T18:33:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.106+0000 7f443ee02740 1 Processor -- start 2024-04-23T18:33:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.106+0000 7f443ee02740 1 -- start start 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba5512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba551800 0x5636ba553bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636ba4ab970 con 0x5636ba546620 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636ba4484a0 con 0x5636ba546f40 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636ba4a7930 con 0x5636ba551800 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f442affd640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 0x7f44200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37278/0 (socket says 172.21.15.33:37278) 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 -- 172.21.15.33:0/2006717151 learned_addr learned my addr 172.21.15.33:0/2006717151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f4439dac640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba5512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f443a5ad640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 0x7f44200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba551800 0x5636ba553bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f442affd640 1 -- 172.21.15.33:0/2966667328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f442c0d0990 con 0x7f443410e930 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 -- 172.21.15.33:0/2006717151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba551800 msgr2=0x5636ba553bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba551800 0x5636ba553bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 -- 172.21.15.33:0/2006717151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 msgr2=0x5636ba5512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba5512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 -- 172.21.15.33:0/2006717151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ba1be210 con 0x5636ba546620 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f4439dac640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba5512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.107+0000 7f44395ab640 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4424000c00 tx=0x7f4424005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6541d1d8f36a63d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443a5ad640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 0x7f44200a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f442c09c900 tx=0x7f442c06e3b0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f441ffff640 1 -- 172.21.15.33:0/2006717151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424044020 con 0x5636ba546620 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f441ffff640 1 -- 172.21.15.33:0/2006717151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4424004770 con 0x5636ba546620 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f441ffff640 1 -- 172.21.15.33:0/2006717151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424035040 con 0x5636ba546620 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 -- 172.21.15.33:0/2006717151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 msgr2=0x5636ba546a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4424000c00 tx=0x7f4424005c00 comp rx=0 tx=0).stop 2024-04-23T18:33:35.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 -- 172.21.15.33:0/2006717151 shutdown_connections 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba551800 0x5636ba553bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba5512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 --2- 172.21.15.33:0/2006717151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba546a00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.108+0000 7f443ee02740 1 -- 172.21.15.33:0/2006717151 >> 172.21.15.33:0/2006717151 conn(0x5636ba542b30 msgr2=0x5636ba542f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- 172.21.15.33:0/2006717151 shutdown_connections 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- 172.21.15.33:0/2006717151 wait complete. 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- start start 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 Processor -- start 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba551110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636ba5543d0 con 0x5636ba551800 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636ba4460b0 con 0x5636ba546f40 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636ba554550 con 0x5636ba546620 2024-04-23T18:33:35.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.109+0000 7f443ee02740 10 client.0 _ll_register_callbacks cb 0x5636ba433d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37288/0 (socket says 172.21.15.33:37288) 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 -- 172.21.15.33:0/2461911532 learned_addr learned my addr 172.21.15.33:0/2461911532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba551110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f4439dac640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 -- 172.21.15.33:0/2461911532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 msgr2=0x5636ba551110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba551110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 -- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 msgr2=0x5636ba54e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 -- 172.21.15.33:0/2461911532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ba28a620 con 0x5636ba551800 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f44395ab640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae452497c1bb292 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f441d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424044070 con 0x5636ba551800 2024-04-23T18:33:35.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f44240054d0 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4424004770 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4424035040 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f443ee02740 10 client.16533 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.110+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5636ba5602b0 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4424004770 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f44240314a0 con 0x5636ba551800 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 10 client.16533 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 20 client.16533 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba561600 0x5636ba5639f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 4 client.16533 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636ba4b9390 con 0x5636ba561600 2024-04-23T18:33:35.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.111+0000 7f4439dac640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba561600 0x5636ba5639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.112+0000 7f4439dac640 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba561600 0x5636ba5639f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724769bc93c1c374 server_cookie=f693597cad69766 in_seq=0 out_seq=0 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.112+0000 7f441d7fa640 10 client.16533 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.112+0000 7f441d7fa640 1 -- 172.21.15.33:0/2461911532 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636ba4b9390 con 0x5636ba561600 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.112+0000 7f441d7fa640 10 client.16533 handle_command_reply: tid=0 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.112+0000 7f443ee02740 1 client.16533 shutdown 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 msgr2=0x5636ba54e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4424006010 tx=0x7f4424004610 comp rx=0 tx=0).stop 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 shutdown_connections 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba561600 0x5636ba5639f0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba551800 0x5636ba54e940 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 --2- 172.21.15.33:0/2461911532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba551110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f441d7fa640 0 client.16533 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 >> 172.21.15.33:0/2461911532 conn(0x5636ba542b30 msgr2=0x5636ba5488b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 shutdown_connections 2024-04-23T18:33:35.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 1 -- 172.21.15.33:0/2461911532 wait complete. 2024-04-23T18:33:35.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.113+0000 7f443ee02740 20 client.16533 trim_cache size 0 max 16384 2024-04-23T18:33:35.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:34 smithi106 ceph-mon[32716]: pgmap v219: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 Processor -- start 2024-04-23T18:33:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 -- start start 2024-04-23T18:33:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba542b30 0x5636ba55d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba550c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44341afc20 con 0x5636ba546620 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b1760 con 0x5636ba546f40 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f443ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b02f0 con 0x5636ba542b30 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f4439dac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f4439dac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37294/0 (socket says 172.21.15.33:37294) 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f4439dac640 1 -- 172.21.15.33:0/2235249831 learned_addr learned my addr 172.21.15.33:0/2235249831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f44395ab640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba550c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f44395ab640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba542b30 0x5636ba55d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f4439dac640 1 -- 172.21.15.33:0/2235249831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba542b30 msgr2=0x5636ba55d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.128+0000 7f4439dac640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba542b30 0x5636ba55d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f4439dac640 1 -- 172.21.15.33:0/2235249831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 msgr2=0x5636ba550c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f4439dac640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba550c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f4439dac640 1 -- 172.21.15.33:0/2235249831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636ba546d70 con 0x5636ba546620 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f44395ab640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba550c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f4439dac640 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x5636ba547bc0 tx=0x7f443000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57010674acf0c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f441f7fe640 1 -- 172.21.15.33:0/2235249831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430004220 con 0x5636ba546620 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f441f7fe640 1 -- 172.21.15.33:0/2235249831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4430024030 con 0x5636ba546620 2024-04-23T18:33:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.129+0000 7f441f7fe640 1 -- 172.21.15.33:0/2235249831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300039e0 con 0x5636ba546620 2024-04-23T18:33:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- 172.21.15.33:0/2235249831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 msgr2=0x5636ba549850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x5636ba547bc0 tx=0x7f443000b410 comp rx=0 tx=0).stop 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- 172.21.15.33:0/2235249831 shutdown_connections 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba550c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba546620 0x5636ba549850 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- 172.21.15.33:0/2235249831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba542b30 0x5636ba55d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- 172.21.15.33:0/2235249831 >> 172.21.15.33:0/2235249831 conn(0x5636ba54eaa0 msgr2=0x5636ba549140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- 172.21.15.33:0/2235249831 shutdown_connections 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- 172.21.15.33:0/2235249831 wait complete. 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 -- start start 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 Processor -- start 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba559ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.130+0000 7f443ee02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b1090 con 0x5636ba542b30 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f44341b09c0 con 0x5636ba546f40 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44341adc20 con 0x5636ba546620 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 10 client.0 _ll_register_callbacks cb 0x7f4424004b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37310/0 (socket says 172.21.15.33:37310) 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f4439dac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba559ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 -- 172.21.15.33:0/966177201 learned_addr learned my addr 172.21.15.33:0/966177201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 -- 172.21.15.33:0/966177201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 msgr2=0x5636ba559ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba559ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 -- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 msgr2=0x5636ba54ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 -- 172.21.15.33:0/966177201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636ba54b370 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f44395ab640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4424031870 tx=0x7f4424004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f891d51a7a37fa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f441d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424044040 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f44240046c0 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4430024030 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4424031d10 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 10 client.16539 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.131+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5636ba5c7410 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4430024030 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f4424003ca0 con 0x5636ba542b30 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 10 client.16539 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 20 client.16539 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba551800 0x5636ba5c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 4 client.16539 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636ba4b3200 con 0x5636ba551800 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f4439dac640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba551800 0x5636ba5c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f441d7fa640 10 client.16539 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.132+0000 7f4439dac640 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba551800 0x5636ba5c9aa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97092b7859e7a0cd server_cookie=cbd969aaadee2c5f in_seq=0 out_seq=0 2024-04-23T18:33:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.140+0000 7f441d7fa640 1 -- 172.21.15.33:0/966177201 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636ba4b3200 con 0x5636ba551800 2024-04-23T18:33:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.140+0000 7f441d7fa640 10 client.16539 handle_command_reply: tid=0 2024-04-23T18:33:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.140+0000 7f443ee02740 1 client.16539 shutdown 2024-04-23T18:33:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 msgr2=0x5636ba5597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4424031870 tx=0x7f4424004610 comp rx=0 tx=0).stop 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 shutdown_connections 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5636ba551800 0x5636ba5c9aa0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636ba546f40 0x5636ba54ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636ba546620 0x5636ba559ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f441d7fa640 0 client.16539 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 --2- 172.21.15.33:0/966177201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636ba542b30 0x5636ba5597a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 >> 172.21.15.33:0/966177201 conn(0x5636ba54eaa0 msgr2=0x5636ba5490f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 shutdown_connections 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 1 -- 172.21.15.33:0/966177201 wait complete. 2024-04-23T18:33:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443ee02740 20 client.16539 trim_cache size 0 max 16384 2024-04-23T18:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:33:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 msgr2=0x7f44200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.141+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 0x7f44200a6280 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f442c09c900 tx=0x7f442c06e3b0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 msgr2=0x7f443419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f442c009fd0 tx=0x7f442c00ee60 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 shutdown_connections 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44200a3dc0 0x7f44200a6280 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f443410e930 0x7f443419a700 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44341016a0 0x7f44341a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 --2- 172.21.15.33:0/2966667328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4434100d80 0x7f44341a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 >> 172.21.15.33:0/2966667328 conn(0x7f44340771d0 msgr2=0x7f443410fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 shutdown_connections 2024-04-23T18:33:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:35.142+0000 7f443c010640 1 -- 172.21.15.33:0/2966667328 wait complete. 2024-04-23T18:33:35.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:34 smithi122 ceph-mon[32546]: pgmap v219: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:34 smithi033 ceph-mon[29274]: pgmap v219: 97 pgs: 97 active+clean; 5.1 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:37.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:36 smithi106 ceph-mon[32716]: pgmap v220: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-23T18:33:37.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:36 smithi122 ceph-mon[32546]: pgmap v220: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-23T18:33:37.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:36 smithi033 ceph-mon[29274]: pgmap v220: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-23T18:33:39.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:38 smithi033 ceph-mon[29274]: pgmap v221: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:39.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:38 smithi106 ceph-mon[32716]: pgmap v221: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:39.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:38 smithi122 ceph-mon[32546]: pgmap v221: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:41.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:40 smithi106 ceph-mon[32716]: pgmap v222: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:41.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:40 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:41.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:40 smithi122 ceph-mon[32546]: pgmap v222: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:41.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:40 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:41.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:40 smithi033 ceph-mon[29274]: pgmap v222: 97 pgs: 97 active+clean; 5.2 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:41.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:40 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:33:42.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:41 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:42.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:41 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:42.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:41 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:33:42.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:41 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:41 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:42.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:41 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:33:42.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:41 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:33:42.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:41 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:33:42.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:41 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:33:43.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:43 smithi106 ceph-mon[32716]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-04-23T18:33:43.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:43 smithi122 ceph-mon[32546]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-04-23T18:33:43.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:43 smithi033 ceph-mon[29274]: pgmap v223: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 20 op/s 2024-04-23T18:33:44.196 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-23T18:33:44.196 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T18:33:44.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:33:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1683115853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 msgr2=0x7fb2481014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1683115853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb2481014f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb244009900 tx=0x7fb24402a410 comp rx=0 tx=0).stop 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1683115853 shutdown_connections 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1683115853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb248106190 0x7fb24810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1683115853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb248101ac0 0x7fb248105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1683115853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb2481014f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1683115853 >> 172.21.15.33:0/1683115853 conn(0x7fb248077fb0 msgr2=0x7fb2480783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1683115853 shutdown_connections 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1683115853 wait complete. 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.321+0000 7fb24f7e2640 1 Processor -- start 2024-04-23T18:33:44.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 -- start start 2024-04-23T18:33:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb248106190 0x7fb24819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 0x7fb248198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb248112210 con 0x7fb248101110 2024-04-23T18:33:44.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb248112090 con 0x7fb2481985b0 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb248112390 con 0x7fb248106190 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36534/0 (socket says 172.21.15.33:36534) 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 -- 172.21.15.33:0/1903577912 learned_addr learned my addr 172.21.15.33:0/1903577912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24dd7f640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 0x7fb248198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 -- 172.21.15.33:0/1903577912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb248106190 msgr2=0x7fb24819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb248106190 0x7fb24819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 -- 172.21.15.33:0/1903577912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 msgr2=0x7fb248198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 0x7fb248198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 -- 172.21.15.33:0/1903577912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2481990d0 con 0x7fb248101110 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24dd7f640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 0x7fb248198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24d57e640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb244005940 tx=0x7fb2440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24403f030 con 0x7fb248101110 2024-04-23T18:33:44.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb244003cb0 con 0x7fb248101110 2024-04-23T18:33:44.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.322+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb248199360 con 0x7fb248101110 2024-04-23T18:33:44.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.323+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244035070 con 0x7fb248101110 2024-04-23T18:33:44.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.323+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2481a5550 con 0x7fb248101110 2024-04-23T18:33:44.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.324+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2480684e0 con 0x7fb248101110 2024-04-23T18:33:44.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.326+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb244042030 con 0x7fb248101110 2024-04-23T18:33:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.326+0000 7fb23e7fc640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 0x7fb2340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.326+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fb2440f13b0 con 0x7fb248101110 2024-04-23T18:33:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.326+0000 7fb24cd7d640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 0x7fb2340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.327+0000 7fb24cd7d640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 0x7fb2340a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb238005fd0 tx=0x7fb238005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.330+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2440bda40 con 0x7fb248101110 2024-04-23T18:33:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.559+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb248101ac0 con 0x7fb248101110 2024-04-23T18:33:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.559+0000 7fb23e7fc640 1 -- 172.21.15.33:0/1903577912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11929 (secure 0 0 0) 0x7fb2440bd340 con 0x7fb248101110 2024-04-23T18:33:44.552 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:33:44.552 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:32:15.719012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25900":{"gid":25900,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/1758592712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1758592712},{"type":"v1","addr":"172.21.15.122:6835","nonce":1758592712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:33:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 msgr2=0x7fb2340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 0x7fb2340a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb238005fd0 tx=0x7fb238005850 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 msgr2=0x7fb24819e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb244005940 tx=0x7fb2440042f0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 shutdown_connections 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb2340a3d20 0x7fb2340a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2481985b0 0x7fb248198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb248106190 0x7fb24819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.561+0000 7fb24f7e2640 1 --2- 172.21.15.33:0/1903577912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb248101110 0x7fb24819e400 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.562+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 >> 172.21.15.33:0/1903577912 conn(0x7fb248077fb0 msgr2=0x7fb2480ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.562+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 shutdown_connections 2024-04-23T18:33:44.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.562+0000 7fb24f7e2640 1 -- 172.21.15.33:0/1903577912 wait complete. 2024-04-23T18:33:44.565 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T18:33:44.565 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T18:33:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 -- 172.21.15.33:0/3889447958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81810c570 msgr2=0x7fa81810e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 --2- 172.21.15.33:0/3889447958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81810c570 0x7fa81810e9e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa810009ca0 tx=0x7fa81002a950 comp rx=0 tx=0).stop 2024-04-23T18:33:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 -- 172.21.15.33:0/3889447958 shutdown_connections 2024-04-23T18:33:44.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 --2- 172.21.15.33:0/3889447958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa81810c570 0x7fa81810e9e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 --2- 172.21.15.33:0/3889447958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 0x7fa81810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 --2- 172.21.15.33:0/3889447958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa818103230 0x7fa818103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 -- 172.21.15.33:0/3889447958 >> 172.21.15.33:0/3889447958 conn(0x7fa818077fb0 msgr2=0x7fa8180783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 -- 172.21.15.33:0/3889447958 shutdown_connections 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 -- 172.21.15.33:0/3889447958 wait complete. 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.685+0000 7fa81de31640 1 Processor -- start 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 -- start start 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 0x7fa818198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa81810c570 0x7fa81819cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa818112220 con 0x7fa818103230 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181120a0 con 0x7fa81810c570 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa81de31640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181123a0 con 0x7fa818103b50 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36552/0 (socket says 172.21.15.33:36552) 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 -- 172.21.15.33:0/854999575 learned_addr learned my addr 172.21.15.33:0/854999575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa817fff640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa81810c570 0x7fa81819cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa816ffd640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 0x7fa818198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 -- 172.21.15.33:0/854999575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 msgr2=0x7fa818198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 0x7fa818198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 -- 172.21.15.33:0/854999575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa81810c570 msgr2=0x7fa81819cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa81810c570 0x7fa81819cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 -- 172.21.15.33:0/854999575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81819d6e0 con 0x7fa818103230 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.686+0000 7fa8177fe640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.687+0000 7fa814ff9640 1 -- 172.21.15.33:0/854999575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808017030 con 0x7fa818103230 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.687+0000 7fa814ff9640 1 -- 172.21.15.33:0/854999575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa808004020 con 0x7fa818103230 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.687+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81819d9d0 con 0x7fa818103230 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.687+0000 7fa814ff9640 1 -- 172.21.15.33:0/854999575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080047a0 con 0x7fa818103230 2024-04-23T18:33:44.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.687+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8181a5260 con 0x7fa818103230 2024-04-23T18:33:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa814ff9640 1 -- 172.21.15.33:0/854999575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa808009a30 con 0x7fa818103230 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 Processor -- start 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 -- start start 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c86876b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c86877090 0x563c86879480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c867d1370 con 0x563c8686c0a0 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563c8676e2e0 con 0x563c86877090 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa820c25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563c867cd330 con 0x563c8686c9c0 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36560/0 (socket says 172.21.15.33:36560) 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa817fff640 1 -- 172.21.15.33:0/3470510527 learned_addr learned my addr 172.21.15.33:0/3470510527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.689+0000 7fa814ff9640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 0x7fa8000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa8177fe640 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c86876b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa816ffd640 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c86877090 0x563c86879480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 0x7fa8000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa814ff9640 1 -- 172.21.15.33:0/854999575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa8080d0240 con 0x7fa818103230 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 -- 172.21.15.33:0/3470510527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 msgr2=0x563c86876b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c86876b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 -- 172.21.15.33:0/3470510527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c86877090 msgr2=0x563c86879480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c86877090 0x563c86879480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 -- 172.21.15.33:0/3470510527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c865afd80 con 0x563c8686c0a0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa817fff640 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa81002ae70 tx=0x7fa810003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd37f5e8f50cdaf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa7fdffb640 1 -- 172.21.15.33:0/3470510527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810043050 con 0x563c8686c0a0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa7fdffb640 1 -- 172.21.15.33:0/3470510527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810044040 con 0x563c8686c0a0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.690+0000 7fa7fdffb640 1 -- 172.21.15.33:0/3470510527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81004d020 con 0x563c8686c0a0 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- 172.21.15.33:0/3470510527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 msgr2=0x563c8686c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa81002ae70 tx=0x7fa810003b20 comp rx=0 tx=0).stop 2024-04-23T18:33:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa817fff640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 0x7fa8000a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa81004e000 tx=0x7fa81004c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- 172.21.15.33:0/3470510527 shutdown_connections 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c86877090 0x563c86879480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c86876b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 --2- 172.21.15.33:0/3470510527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c0a0 0x563c8686c480 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- 172.21.15.33:0/3470510527 >> 172.21.15.33:0/3470510527 conn(0x563c868683c0 msgr2=0x563c868687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- 172.21.15.33:0/3470510527 shutdown_connections 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- 172.21.15.33:0/3470510527 wait complete. 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.691+0000 7fa820c25740 1 -- start start 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 Processor -- start 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 0x563c86874020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c9c0 0x563c86874560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c86879e80 con 0x563c86877090 2024-04-23T18:33:44.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563c8676bfb0 con 0x563c8686c9c0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563c8687a000 con 0x563c8686c0a0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa820c25740 10 client.0 _ll_register_callbacks cb 0x563c8676b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36570/0 (socket says 172.21.15.33:36570) 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 -- 172.21.15.33:0/910273058 learned_addr learned my addr 172.21.15.33:0/910273058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa816ffd640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c9c0 0x563c86874560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa8177fe640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 0x563c86874020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 -- 172.21.15.33:0/910273058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 msgr2=0x563c86874020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 0x563c86874020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 -- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c9c0 msgr2=0x563c86874560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c9c0 0x563c86874560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 -- 172.21.15.33:0/910273058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c86610510 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa8177fe640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 0x563c86874020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.692+0000 7fa817fff640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa810005dc0 tx=0x7fa810004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943fa4a36ba7a55f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81005a070 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa810049800 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa810044040 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa810066050 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 10 client.16557 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563c864a3670 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810044040 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa8100304d0 con 0x563c86877090 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 10 client.16557 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:44.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 10 client.16557 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 20 client.16557 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868872a0 0x563c86889690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 4 client.16557 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.693+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c867ded90 con 0x563c868872a0 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.694+0000 7fa8177fe640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868872a0 0x563c86889690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.694+0000 7fa7f6ffd640 10 client.16557 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.694+0000 7fa8177fe640 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868872a0 0x563c86889690 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=180ea7a974088dd3 server_cookie=54c89320a5458675 in_seq=0 out_seq=0 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/910273058 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c867ded90 con 0x563c868872a0 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa7f6ffd640 10 client.16557 handle_command_reply: tid=0 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 client.16557 shutdown 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 msgr2=0x563c86874aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa810005dc0 tx=0x7fa810004600 comp rx=0 tx=0).stop 2024-04-23T18:33:44.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 shutdown_connections 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868872a0 0x563c86889690 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c86874aa0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c9c0 0x563c86874560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 --2- 172.21.15.33:0/910273058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c0a0 0x563c86874020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa7f6ffd640 0 client.16557 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 >> 172.21.15.33:0/910273058 conn(0x563c868683c0 msgr2=0x563c8686e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 shutdown_connections 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 1 -- 172.21.15.33:0/910273058 wait complete. 2024-04-23T18:33:44.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.695+0000 7fa820c25740 20 client.16557 trim_cache size 0 max 16384 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 Processor -- start 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 -- start start 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8686e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 0x563c868712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181adbd0 con 0x563c8686c9c0 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181af710 con 0x563c8686c0a0 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.712+0000 7fa820c25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181ae2a0 con 0x563c86877090 2024-04-23T18:33:44.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36582/0 (socket says 172.21.15.33:36582) 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 -- 172.21.15.33:0/3644437931 learned_addr learned my addr 172.21.15.33:0/3644437931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa8177fe640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8686e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa817fff640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 0x563c868712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 -- 172.21.15.33:0/3644437931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 msgr2=0x563c868712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 0x563c868712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 -- 172.21.15.33:0/3644437931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 msgr2=0x563c8686e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8686e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 -- 172.21.15.33:0/3644437931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8686b6b0 con 0x563c8686c9c0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa817fff640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 0x563c868712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa816ffd640 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009b20 tx=0x7fa80c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1913eb3c7122505d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa7fd7fa640 1 -- 172.21.15.33:0/3644437931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c012c40 con 0x563c8686c9c0 2024-04-23T18:33:44.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa7fd7fa640 1 -- 172.21.15.33:0/3644437931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c012de0 con 0x563c8686c9c0 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.713+0000 7fa7fd7fa640 1 -- 172.21.15.33:0/3644437931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c01c460 con 0x563c8686c9c0 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- 172.21.15.33:0/3644437931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 msgr2=0x563c86882960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009b20 tx=0x7fa80c012450 comp rx=0 tx=0).stop 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- 172.21.15.33:0/3644437931 shutdown_connections 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c86877090 0x563c868712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c8686c9c0 0x563c86882960 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 --2- 172.21.15.33:0/3644437931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8686e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- 172.21.15.33:0/3644437931 >> 172.21.15.33:0/3644437931 conn(0x563c868683c0 msgr2=0x563c86882250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- 172.21.15.33:0/3644437931 shutdown_connections 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- 172.21.15.33:0/3644437931 wait complete. 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 -- start start 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.714+0000 7fa820c25740 1 Processor -- start 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8688a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c8688a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181af040 con 0x563c86877090 2024-04-23T18:33:44.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181ae970 con 0x563c8686c0a0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8181abbd0 con 0x563c8686c9c0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 10 client.0 _ll_register_callbacks cb 0x7fa810003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36598/0 (socket says 172.21.15.33:36598) 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 -- 172.21.15.33:0/2765793247 learned_addr learned my addr 172.21.15.33:0/2765793247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa816ffd640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c8688a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa8177fe640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8688a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 -- 172.21.15.33:0/2765793247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 msgr2=0x563c8688a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c8688a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 -- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 msgr2=0x563c8688a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8688a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 -- 172.21.15.33:0/2765793247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c8688b230 con 0x563c86877090 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa817fff640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8100496a0 tx=0x7fa81002ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf644c11a5f42c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:44.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa7f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81005a070 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7fa810004250 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80c012de0 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.715+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa81004d030 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 10 client.16563 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563c868ed9e0 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c012de0 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fa810030440 con 0x563c86877090 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 10 client.16563 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 10 client.16563 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 20 client.16563 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868ee1a0 0x563c868f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 4 client.16563 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x563c867d8c00 con 0x563c868ee1a0 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa8177fe640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868ee1a0 0x563c868f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa8177fe640 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868ee1a0 0x563c868f05c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c5da48d1e4e09a server_cookie=6fce5d816b8206dd in_seq=0 out_seq=0 2024-04-23T18:33:44.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.716+0000 7fa7f6ffd640 10 client.16563 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:44.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.730+0000 7fa7f6ffd640 1 -- 172.21.15.33:0/2765793247 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563c867d8c00 con 0x563c868ee1a0 2024-04-23T18:33:44.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa7f6ffd640 10 client.16563 handle_command_reply: tid=0 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 client.16563 shutdown 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 msgr2=0x563c868ec480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa8100496a0 tx=0x7fa81002ff50 comp rx=0 tx=0).stop 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 shutdown_connections 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563c868ee1a0 0x563c868f05c0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c86877090 0x563c868ec480 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c8686c9c0 0x563c8688a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 --2- 172.21.15.33:0/2765793247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8686c0a0 0x563c8688a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa7f6ffd640 0 client.16563 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 >> 172.21.15.33:0/2765793247 conn(0x563c868683c0 msgr2=0x563c86882ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 shutdown_connections 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 1 -- 172.21.15.33:0/2765793247 wait complete. 2024-04-23T18:33:44.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.731+0000 7fa820c25740 20 client.16563 trim_cache size 0 max 16384 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 33, 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 26, 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "284427f0": { 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 26, 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:33:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:33:44.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 msgr2=0x7fa8000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 0x7fa8000a6260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa81004e000 tx=0x7fa81004c000 comp rx=0 tx=0).stop 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 msgr2=0x7fa818198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d950 tx=0x7fa80800de10 comp rx=0 tx=0).stop 2024-04-23T18:33:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 shutdown_connections 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa8000a3da0 0x7fa8000a6260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa81810c570 0x7fa81819cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa818103b50 0x7fa818198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 --2- 172.21.15.33:0/854999575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103230 0x7fa818198740 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 >> 172.21.15.33:0/854999575 conn(0x7fa818077fb0 msgr2=0x7fa8180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 shutdown_connections 2024-04-23T18:33:44.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:33:44.732+0000 7fa81de31640 1 -- 172.21.15.33:0/854999575 wait complete. 2024-04-23T18:33:44.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '284427f0', quiesce took 0.5 seconds. Will release after: 30.0 2024-04-23T18:33:45.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:45 smithi106 ceph-mon[32716]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:45.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:45 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1903577912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:45.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:45 smithi122 ceph-mon[32546]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:45.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:45 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1903577912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:45.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:45 smithi033 ceph-mon[29274]: pgmap v224: 97 pgs: 97 active+clean; 5.3 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-23T18:33:45.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:45 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1903577912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:33:47.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:47 smithi106 ceph-mon[32716]: pgmap v225: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:47.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:47 smithi122 ceph-mon[32546]: pgmap v225: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:47.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:47 smithi033 ceph-mon[29274]: pgmap v225: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 20 op/s 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- 172.21.15.33:0/3582583042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 msgr2=0x7ff91c10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 --2- 172.21.15.33:0/3582583042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c10d490 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff914009cd0 tx=0x7ff91402a8e0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- 172.21.15.33:0/3582583042 shutdown_connections 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 --2- 172.21.15.33:0/3582583042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c10d490 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 --2- 172.21.15.33:0/3582583042 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 0x7ff91c100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 --2- 172.21.15.33:0/3582583042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 0x7ff91c106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- 172.21.15.33:0/3582583042 >> 172.21.15.33:0/3582583042 conn(0x7ff91c077390 msgr2=0x7ff91c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- 172.21.15.33:0/3582583042 shutdown_connections 2024-04-23T18:33:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- 172.21.15.33:0/3582583042 wait complete. 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 Processor -- start 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.242+0000 7ff923272640 1 -- start start 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 0x7ff91c1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 0x7ff91c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c110020 con 0x7ff91c100f80 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c10fea0 con 0x7ff91c1005c0 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff923272640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1101a0 con 0x7ff91c106660 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33130/0 (socket says 172.21.15.33:33130) 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 -- 172.21.15.33:0/1018028140 learned_addr learned my addr 172.21.15.33:0/1018028140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92100e640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 0x7ff91c1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92180f640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 0x7ff91c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 -- 172.21.15.33:0/1018028140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 msgr2=0x7ff91c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 0x7ff91c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 -- 172.21.15.33:0/1018028140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 msgr2=0x7ff91c1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 0x7ff91c1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 -- 172.21.15.33:0/1018028140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c19ad80 con 0x7ff91c100f80 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff92080d640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d950 tx=0x7ff90c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.243+0000 7ff9127fc640 1 -- 172.21.15.33:0/1018028140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c017070 con 0x7ff91c100f80 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.244+0000 7ff9127fc640 1 -- 172.21.15.33:0/1018028140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90c009c30 con 0x7ff91c100f80 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.244+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91c19b070 con 0x7ff91c100f80 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.244+0000 7ff9127fc640 1 -- 172.21.15.33:0/1018028140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c00f620 con 0x7ff91c100f80 2024-04-23T18:33:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.244+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff91c1a7310 con 0x7ff91c100f80 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.246+0000 7ff9127fc640 1 -- 172.21.15.33:0/1018028140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff90c010460 con 0x7ff91c100f80 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.246+0000 7ff925f88740 1 Processor -- start 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 -- start start 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1ef620 0x56376d1efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56376d154970 con 0x56376d1fa800 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56376d0f14a0 con 0x56376d1eff40 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff925f88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56376d150930 con 0x56376d1ef620 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff9127fc640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 0x7ff9080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92100e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92100e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:55456/0 (socket says 172.21.15.33:55456) 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33132/0 (socket says 172.21.15.33:33132) 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92180f640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 0x7ff9080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92100e640 1 -- 172.21.15.33:0/2790926240 learned_addr learned my addr 172.21.15.33:0/2790926240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff9127fc640 1 -- 172.21.15.33:0/1018028140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff90c0d03c0 con 0x7ff91c100f80 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92180f640 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1ef620 0x56376d1efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92100e640 1 -- 172.21.15.33:0/2790926240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1ef620 msgr2=0x56376d1efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.247+0000 7ff92100e640 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1ef620 0x56376d1efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92100e640 1 -- 172.21.15.33:0/2790926240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 msgr2=0x56376d1fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92100e640 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92100e640 1 -- 172.21.15.33:0/2790926240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376ce67210 con 0x56376d1eff40 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92080d640 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92180f640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 0x7ff9080a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff914004af0 tx=0x7ff914009910 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff92100e640 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff91800bd20 tx=0x7ff91800e820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5317427aebfd935 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff9077fe640 1 -- 172.21.15.33:0/2790926240 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918004260 con 0x56376d1eff40 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff9077fe640 1 -- 172.21.15.33:0/2790926240 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff918004400 con 0x56376d1eff40 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.248+0000 7ff9077fe640 1 -- 172.21.15.33:0/2790926240 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9180188e0 con 0x56376d1eff40 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- 172.21.15.33:0/2790926240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 msgr2=0x56376d1fa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff91800bd20 tx=0x7ff91800e820 comp rx=0 tx=0).stop 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- 172.21.15.33:0/2790926240 shutdown_connections 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1fa800 0x56376d1fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1fa2c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 --2- 172.21.15.33:0/2790926240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1ef620 0x56376d1efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- 172.21.15.33:0/2790926240 >> 172.21.15.33:0/2790926240 conn(0x56376d1ebb30 msgr2=0x56376d1ebf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- 172.21.15.33:0/2790926240 shutdown_connections 2024-04-23T18:33:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- 172.21.15.33:0/2790926240 wait complete. 2024-04-23T18:33:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 -- start start 2024-04-23T18:33:48.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.249+0000 7ff925f88740 1 Processor -- start 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56376d1fd430 con 0x56376d1ef620 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56376d0ef0b0 con 0x56376d1eff40 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56376d1fd5b0 con 0x56376d1fa800 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff925f88740 10 client.0 _ll_register_callbacks cb 0x56376d0dcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33138/0 (socket says 172.21.15.33:33138) 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 -- 172.21.15.33:0/530839598 learned_addr learned my addr 172.21.15.33:0/530839598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92080d640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92180f640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 -- 172.21.15.33:0/530839598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 msgr2=0x56376d1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d1f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 -- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 msgr2=0x56376d1f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 -- 172.21.15.33:0/530839598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376cf33620 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92180f640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff92100e640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff918003c50 tx=0x7ff918003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d452755891e59f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918004260 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.250+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff918014040 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff918004400 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff91800b9e0 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 10 client.16575 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56376d209300 con 0x56376d1ef620 2024-04-23T18:33:48.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff918004400 con 0x56376d1ef620 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff91800b040 con 0x56376d1ef620 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 10 client.16575 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 20 client.16575 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d20a760 0x56376d20cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 4 client.16575 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376d162390 con 0x56376d20a760 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.251+0000 7ff92080d640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d20a760 0x56376d20cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.252+0000 7ff904ff9640 10 client.16575 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.252+0000 7ff92080d640 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d20a760 0x56376d20cb50 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea18dfdc65cc8f7 server_cookie=7ed4a693fab7ad1f in_seq=0 out_seq=0 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff904ff9640 1 -- 172.21.15.33:0/530839598 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56376d162390 con 0x56376d20a760 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff904ff9640 10 client.16575 handle_command_reply: tid=0 2024-04-23T18:33:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 client.16575 shutdown 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 msgr2=0x56376d1fa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff918003c50 tx=0x7ff918003c80 comp rx=0 tx=0).stop 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 shutdown_connections 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d20a760 0x56376d20cb50 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d1f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 --2- 172.21.15.33:0/530839598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d1fa0f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff904ff9640 0 client.16575 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 >> 172.21.15.33:0/530839598 conn(0x56376d1ebb30 msgr2=0x56376d1f1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 shutdown_connections 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 1 -- 172.21.15.33:0/530839598 wait complete. 2024-04-23T18:33:48.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.253+0000 7ff925f88740 20 client.16575 trim_cache size 0 max 16384 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 Processor -- start 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 -- start start 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1ef620 0x56376d1eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d2100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1afd40 con 0x56376d1eff40 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b1880 con 0x56376d1ef620 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff925f88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b0410 con 0x56376d1fa800 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff92080d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33140/0 (socket says 172.21.15.33:33140) 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.268+0000 7ff92080d640 1 -- 172.21.15.33:0/459252168 learned_addr learned my addr 172.21.15.33:0/459252168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92180f640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d2100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92100e640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1ef620 0x56376d1eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 -- 172.21.15.33:0/459252168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 msgr2=0x56376d2100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d2100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 -- 172.21.15.33:0/459252168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1ef620 msgr2=0x56376d1eaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1ef620 0x56376d1eaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 -- 172.21.15.33:0/459252168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56376d1f7c80 con 0x56376d1eff40 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92180f640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d2100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:33:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff92080d640 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c06de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b624c88f7ca936 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff906ffd640 1 -- 172.21.15.33:0/459252168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c020040 con 0x56376d1eff40 2024-04-23T18:33:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff906ffd640 1 -- 172.21.15.33:0/459252168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90c09a700 con 0x56376d1eff40 2024-04-23T18:33:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.269+0000 7ff906ffd640 1 -- 172.21.15.33:0/459252168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c098390 con 0x56376d1eff40 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- 172.21.15.33:0/459252168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 msgr2=0x56376d20fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c06de40 comp rx=0 tx=0).stop 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- 172.21.15.33:0/459252168 shutdown_connections 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d2100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1eff40 0x56376d20fb80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- 172.21.15.33:0/459252168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1ef620 0x56376d1eaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- 172.21.15.33:0/459252168 >> 172.21.15.33:0/459252168 conn(0x56376d1ebb30 msgr2=0x56376d206090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- 172.21.15.33:0/459252168 shutdown_connections 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- 172.21.15.33:0/459252168 wait complete. 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- start start 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 Processor -- start 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d2682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d20f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b11b0 con 0x56376d1ef620 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1b0ae0 con 0x56376d1eff40 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c1add40 con 0x56376d1fa800 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.270+0000 7ff925f88740 10 client.0 _ll_register_callbacks cb 0x7ff91800bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33142/0 (socket says 172.21.15.33:33142) 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 -- 172.21.15.33:0/1700663219 learned_addr learned my addr 172.21.15.33:0/1700663219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92180f640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d20f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92080d640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d2682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 -- 172.21.15.33:0/1700663219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 msgr2=0x56376d20f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d20f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 -- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 msgr2=0x56376d2682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d2682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 -- 172.21.15.33:0/1700663219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56376d20f830 con 0x56376d1ef620 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92080d640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d2682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92180f640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d20f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff92100e640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff9180039f0 tx=0x7ff918005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bb87e4ccc1125 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff904ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91802f020 con 0x56376d1ef620 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7ff91801f040 con 0x56376d1ef620 2024-04-23T18:33:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.271+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff90c09a700 con 0x56376d1ef620 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff91803a050 con 0x56376d1ef620 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 10 client.16581 fetch_fsmap learned FSMap version 33 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56376d2705d0 con 0x56376d1ef620 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff90c09a700 con 0x56376d1ef620 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7ff918004680 con 0x56376d1ef620 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 10 client.16581 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 20 client.16581 populate_metadata read hostname 'smithi033' 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d270d10 0x56376d273130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 4 client.16581 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56376d15c200 con 0x56376d270d10 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.272+0000 7ff92080d640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d270d10 0x56376d273130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.273+0000 7ff92080d640 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d270d10 0x56376d273130 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c03e97c599dfaf1 server_cookie=47994fcca62806a7 in_seq=0 out_seq=0 2024-04-23T18:33:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.273+0000 7ff904ff9640 10 client.16581 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff904ff9640 1 -- 172.21.15.33:0/1700663219 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56376d15c200 con 0x56376d270d10 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff904ff9640 10 client.16581 handle_command_reply: tid=0 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 client.16581 shutdown 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 msgr2=0x56376d267d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff9180039f0 tx=0x7ff918005770 comp rx=0 tx=0).stop 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 shutdown_connections 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56376d270d10 0x56376d273130 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56376d1fa800 0x56376d20f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56376d1eff40 0x56376d2682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff904ff9640 0 client.16581 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:33:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 --2- 172.21.15.33:0/1700663219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56376d1ef620 0x56376d267d80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 >> 172.21.15.33:0/1700663219 conn(0x56376d1ebb30 msgr2=0x56376d1f6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 shutdown_connections 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 1 -- 172.21.15.33:0/1700663219 wait complete. 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.280+0000 7ff925f88740 20 client.16581 trim_cache size 0 max 16384 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:33:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 msgr2=0x7ff9080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 0x7ff9080a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff914004af0 tx=0x7ff914009910 comp rx=0 tx=0).stop 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 msgr2=0x7ff91c1a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00d950 tx=0x7ff90c00de10 comp rx=0 tx=0).stop 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 shutdown_connections 2024-04-23T18:33:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff9080a3dc0 0x7ff9080a6280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff91c106660 0x7ff91c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff91c100f80 0x7ff91c1a0990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 --2- 172.21.15.33:0/1018028140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff91c1005c0 0x7ff91c1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 >> 172.21.15.33:0/1018028140 conn(0x7ff91c077390 msgr2=0x7ff91c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 shutdown_connections 2024-04-23T18:33:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:33:48.281+0000 7ff923272640 1 -- 172.21.15.33:0/1018028140 wait complete. 2024-04-23T18:33:49.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:49 smithi106 ceph-mon[32716]: pgmap v226: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:49.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:49 smithi122 ceph-mon[32546]: pgmap v226: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:49.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:49 smithi033 ceph-mon[29274]: pgmap v226: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:51.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:51 smithi106 ceph-mon[32716]: pgmap v227: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:51.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:51 smithi122 ceph-mon[32546]: pgmap v227: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:51.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:51 smithi033 ceph-mon[29274]: pgmap v227: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:33:53.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:53 smithi106 ceph-mon[32716]: pgmap v228: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-23T18:33:53.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:53 smithi122 ceph-mon[32546]: pgmap v228: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-23T18:33:53.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:53 smithi033 ceph-mon[29274]: pgmap v228: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 20 KiB/s wr, 18 op/s 2024-04-23T18:33:55.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:55 smithi106 ceph-mon[32716]: pgmap v229: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-23T18:33:55.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:55 smithi122 ceph-mon[32546]: pgmap v229: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-23T18:33:55.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:55 smithi033 ceph-mon[29274]: pgmap v229: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-23T18:33:57.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:57 smithi106 ceph-mon[32716]: pgmap v230: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:57.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:57 smithi122 ceph-mon[32546]: pgmap v230: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:57.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:57 smithi033 ceph-mon[29274]: pgmap v230: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 KiB/s wr, 16 op/s 2024-04-23T18:33:59.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:33:59 smithi106 ceph-mon[32716]: pgmap v231: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:33:59.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:33:59 smithi122 ceph-mon[32546]: pgmap v231: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:33:59.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:33:59 smithi033 ceph-mon[29274]: pgmap v231: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:01.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:01 smithi106 ceph-mon[32716]: pgmap v232: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 -- 172.21.15.33:0/1796995414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4768100700 msgr2=0x7f4768100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 --2- 172.21.15.33:0/1796995414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4768100700 0x7f4768100ae0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4758009900 tx=0x7f475802a410 comp rx=0 tx=0).stop 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 -- 172.21.15.33:0/1796995414 shutdown_connections 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 --2- 172.21.15.33:0/1796995414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768107310 0x7f476810ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 --2- 172.21.15.33:0/1796995414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47681010b0 0x7f4768106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 --2- 172.21.15.33:0/1796995414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4768100700 0x7f4768100ae0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 -- 172.21.15.33:0/1796995414 >> 172.21.15.33:0/1796995414 conn(0x7f4768077390 msgr2=0x7f47680fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 -- 172.21.15.33:0/1796995414 shutdown_connections 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.380+0000 7f476e119640 1 -- 172.21.15.33:0/1796995414 wait complete. 2024-04-23T18:34:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 Processor -- start 2024-04-23T18:34:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 -- start start 2024-04-23T18:34:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768100700 0x7f4768101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768107310 0x7f4768101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112200 con 0x7f47681020f0 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112080 con 0x7f4768100700 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.381+0000 7f476e119640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112380 con 0x7f4768107310 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41270/0 (socket says 172.21.15.33:41270) 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 -- 172.21.15.33:0/2721412570 learned_addr learned my addr 172.21.15.33:0/2721412570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f47677fe640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768100700 0x7f4768101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4766ffd640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768107310 0x7f4768101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 -- 172.21.15.33:0/2721412570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768107310 msgr2=0x7f4768101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768107310 0x7f4768101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 -- 172.21.15.33:0/2721412570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768100700 msgr2=0x7f4768101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768100700 0x7f4768101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 -- 172.21.15.33:0/2721412570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47681a7010 con 0x7f47681020f0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4767fff640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4760009fd0 tx=0x7f476000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4764ff9640 1 -- 172.21.15.33:0/2721412570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760017070 con 0x7f47681020f0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4764ff9640 1 -- 172.21.15.33:0/2721412570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4760003e20 con 0x7f47681020f0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47681a7300 con 0x7f47681020f0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.382+0000 7f4764ff9640 1 -- 172.21.15.33:0/2721412570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760010460 con 0x7f47681020f0 2024-04-23T18:34:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.383+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681a7800 con 0x7f47681020f0 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4764ff9640 1 -- 172.21.15.33:0/2721412570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4760003970 con 0x7f47681020f0 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 Processor -- start 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 -- start start 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a280722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28072800 0x559a28074bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a27fcc970 con 0x559a28067620 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559a27f694a0 con 0x559a28067f40 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4770e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a27fc8930 con 0x559a28072800 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4764ff9640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 0x7f47500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41282/0 (socket says 172.21.15.33:41282) 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 -- 172.21.15.33:0/604442685 learned_addr learned my addr 172.21.15.33:0/604442685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f47677fe640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a280722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4767fff640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 0x7f47500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28072800 0x559a28074bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4764ff9640 1 -- 172.21.15.33:0/2721412570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f47600d04a0 con 0x7f47681020f0 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 -- 172.21.15.33:0/604442685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28072800 msgr2=0x559a28074bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28072800 0x559a28074bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 -- 172.21.15.33:0/604442685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 msgr2=0x559a280722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a280722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 -- 172.21.15.33:0/604442685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a27cdf210 con 0x559a28067620 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f47677fe640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a280722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4766ffd640 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058a0 tx=0x7f475c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde0b4395f315bac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f4767fff640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 0x7f47500a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47600ce1e0 tx=0x7f476001d040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f474dffb640 1 -- 172.21.15.33:0/604442685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004030 con 0x559a28067620 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f474dffb640 1 -- 172.21.15.33:0/604442685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c0054d0 con 0x559a28067620 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.386+0000 7f474dffb640 1 -- 172.21.15.33:0/604442685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004ba0 con 0x559a28067620 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- 172.21.15.33:0/604442685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 msgr2=0x559a28067a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f475c0058a0 tx=0x7f475c00b820 comp rx=0 tx=0).stop 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- 172.21.15.33:0/604442685 shutdown_connections 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28072800 0x559a28074bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a280722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 --2- 172.21.15.33:0/604442685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a28067a00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- 172.21.15.33:0/604442685 >> 172.21.15.33:0/604442685 conn(0x559a28063b30 msgr2=0x559a28063f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- 172.21.15.33:0/604442685 shutdown_connections 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- 172.21.15.33:0/604442685 wait complete. 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.387+0000 7f4770e2e740 1 -- start start 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 Processor -- start 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a28072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2806f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a28075430 con 0x559a28072800 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559a27f670b0 con 0x559a28067f40 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a280755b0 con 0x559a28067620 2024-04-23T18:34:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4770e2e740 10 client.0 _ll_register_callbacks cb 0x559a27f54d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41286/0 (socket says 172.21.15.33:41286) 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4766ffd640 1 -- 172.21.15.33:0/4278325715 learned_addr learned my addr 172.21.15.33:0/4278325715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f47677fe640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2806f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.388+0000 7f4766ffd640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a28072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 -- 172.21.15.33:0/4278325715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 msgr2=0x559a28072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a28072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 -- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 msgr2=0x559a2806f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2806f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 -- 172.21.15.33:0/4278325715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a27dab620 con 0x559a28072800 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4766ffd640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f475c004650 tx=0x7f475c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461b371bb1124ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c005330 con 0x559a28072800 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f475c004b20 con 0x559a28072800 2024-04-23T18:34:01.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c0054d0 con 0x559a28072800 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f475c004d20 con 0x559a28072800 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4770e2e740 10 client.16593 fetch_fsmap learned FSMap version 33 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a28081310 con 0x559a28072800 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.389+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c0054d0 con 0x559a28072800 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f475c01f460 con 0x559a28072800 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 10 client.16593 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 20 client.16593 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a28082710 0x559a28084b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 4 client.16593 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a27fda390 con 0x559a28082710 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f47677fe640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a28082710 0x559a28084b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f47677fe640 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a28082710 0x559a28084b00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0cb184e6827a2ab server_cookie=254a9dc3f64d9035 in_seq=0 out_seq=0 2024-04-23T18:34:01.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.390+0000 7f4746ffd640 10 client.16593 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4746ffd640 1 -- 172.21.15.33:0/4278325715 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a27fda390 con 0x559a28082710 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4746ffd640 10 client.16593 handle_command_reply: tid=0 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 client.16593 shutdown 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 msgr2=0x559a2806f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f475c004650 tx=0x7f475c004840 comp rx=0 tx=0).stop 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 shutdown_connections 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a28082710 0x559a28084b00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806f940 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2806f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 --2- 172.21.15.33:0/4278325715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a28072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4746ffd640 0 client.16593 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.391+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 >> 172.21.15.33:0/4278325715 conn(0x559a28063b30 msgr2=0x559a280698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.392+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 shutdown_connections 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.392+0000 7f4770e2e740 1 -- 172.21.15.33:0/4278325715 wait complete. 2024-04-23T18:34:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.392+0000 7f4770e2e740 20 client.16593 trim_cache size 0 max 16384 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 Processor -- start 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 -- start start 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 0x559a28069860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28072800 0x559a28071c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47681afe70 con 0x559a28067620 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47681b19b0 con 0x559a28072800 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4770e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47681b0540 con 0x559a28067f40 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41300/0 (socket says 172.21.15.33:41300) 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4766ffd640 1 -- 172.21.15.33:0/2728607833 learned_addr learned my addr 172.21.15.33:0/2728607833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f4766ffd640 1 -- 172.21.15.33:0/2728607833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 msgr2=0x559a28069860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.407+0000 7f47677fe640 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 0x559a28069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4766ffd640 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 0x559a28069860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4766ffd640 1 -- 172.21.15.33:0/2728607833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28072800 msgr2=0x559a28071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4766ffd640 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28072800 0x559a28071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4766ffd640 1 -- 172.21.15.33:0/2728607833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a2806d950 con 0x559a28067620 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f47677fe640 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 0x559a28069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4766ffd640 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f475c000c00 tx=0x7f475c0058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463ce33275ddd2be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f474d7fa640 1 -- 172.21.15.33:0/2728607833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004680 con 0x559a28067620 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f474d7fa640 1 -- 172.21.15.33:0/2728607833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c025020 con 0x559a28067620 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f474d7fa640 1 -- 172.21.15.33:0/2728607833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c01fa40 con 0x559a28067620 2024-04-23T18:34:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.408+0000 7f4770e2e740 1 -- 172.21.15.33:0/2728607833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 msgr2=0x559a2806ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f475c000c00 tx=0x7f475c0058f0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- 172.21.15.33:0/2728607833 shutdown_connections 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28072800 0x559a28071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067f40 0x559a28069860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- 172.21.15.33:0/2728607833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28067620 0x559a2806ece0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- 172.21.15.33:0/2728607833 >> 172.21.15.33:0/2728607833 conn(0x559a28063b30 msgr2=0x559a2806a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- 172.21.15.33:0/2728607833 shutdown_connections 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- 172.21.15.33:0/2728607833 wait complete. 2024-04-23T18:34:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- start start 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 Processor -- start 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a2807a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2807ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47681b12e0 con 0x559a28072800 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f47681b0c10 con 0x559a28067f40 2024-04-23T18:34:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47681ade70 con 0x559a28067620 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.409+0000 7f4770e2e740 10 client.0 _ll_register_callbacks cb 0x7f475c00bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41316/0 (socket says 172.21.15.33:41316) 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 -- 172.21.15.33:0/1665234264 learned_addr learned my addr 172.21.15.33:0/1665234264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a2807a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f47677fe640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2807ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 -- 172.21.15.33:0/1665234264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 msgr2=0x559a2807a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a2807a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 -- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 msgr2=0x559a2807ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2807ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 -- 172.21.15.33:0/1665234264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a2806c450 con 0x559a28072800 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4766ffd640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x559a2806d640 tx=0x7f475c020f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d31f466e083b726 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4746ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:01.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c004630 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8542+0+0 (secure 0 0 0) 0x7f475c0047d0 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f475c025020 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f475c0049d0 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4770e2e740 10 client.16599 fetch_fsmap learned FSMap version 33 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.410+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559a280e8650 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f475c025020 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f475c037020 con 0x559a28072800 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 10 client.16599 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 20 client.16599 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a280e8c80 0x559a280eb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 4 client.16599 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a2806fcd0 con 0x559a280e8c80 2024-04-23T18:34:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.411+0000 7f47677fe640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a280e8c80 0x559a280eb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.412+0000 7f47677fe640 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a280e8c80 0x559a280eb0a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be81ddb686018c81 server_cookie=a13f10dce1de67d3 in_seq=0 out_seq=0 2024-04-23T18:34:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:01.412+0000 7f4746ffd640 10 client.16599 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:01.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:01 smithi122 ceph-mon[32546]: pgmap v232: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:01.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:01 smithi033 ceph-mon[29274]: pgmap v232: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:03.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.171+0000 7f4746ffd640 1 -- 172.21.15.33:0/1665234264 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a2806fcd0 con 0x559a280e8c80 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.171+0000 7f4746ffd640 10 client.16599 handle_command_reply: tid=0 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.171+0000 7f4770e2e740 1 client.16599 shutdown 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 msgr2=0x559a2806bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x559a2806d640 tx=0x7f475c020f70 comp rx=0 tx=0).stop 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 shutdown_connections 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559a280e8c80 0x559a280eb0a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a28072800 0x559a2806bc50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559a28067f40 0x559a2807ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 --2- 172.21.15.33:0/1665234264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a28067620 0x559a2807a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4746ffd640 0 client.16599 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:03.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 >> 172.21.15.33:0/1665234264 conn(0x559a28063b30 msgr2=0x559a28069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 shutdown_connections 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 1 -- 172.21.15.33:0/1665234264 wait complete. 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.172+0000 7f4770e2e740 20 client.16599 trim_cache size 0 max 16384 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:34:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:34:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 msgr2=0x7f47500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 0x7f47500a6280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47600ce1e0 tx=0x7f476001d040 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 msgr2=0x7f47681a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4760009fd0 tx=0x7f476000b260 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 shutdown_connections 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f47500a3dc0 0x7f47500a6280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47681020f0 0x7f47681a6970 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768107310 0x7f4768101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 --2- 172.21.15.33:0/2721412570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4768100700 0x7f4768101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 >> 172.21.15.33:0/2721412570 conn(0x7f4768077390 msgr2=0x7f47680fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 shutdown_connections 2024-04-23T18:34:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:03.173+0000 7f476e119640 1 -- 172.21.15.33:0/2721412570 wait complete. 2024-04-23T18:34:03.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:03 smithi106 ceph-mon[32716]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T18:34:03.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-mon[32546]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T18:34:03.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:03 smithi033 ceph-mon[29274]: pgmap v233: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 51 KiB/s wr, 31 op/s 2024-04-23T18:34:04.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:34:04.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: in thread 7f21b9a1c700 thread_name:md_log_replay 2024-04-23T18:34:04.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:34:04.198 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21cadddd20] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2: /usr/bin/ceph-mds(+0x54938f) [0x557cb89b238f] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557cb89b5b23] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 4: (MDLog::_replay_thread()+0x617) [0x557cb8a10647] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557cb8a19d31] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f21cadd31ca] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 7: clone() 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:03.933+0000 7f21b9a1c700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: in thread 7f21b9a1c700 thread_name:md_log_replay 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21cadddd20] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2: /usr/bin/ceph-mds(+0x54938f) [0x557cb89b238f] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557cb89b5b23] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 4: (MDLog::_replay_thread()+0x617) [0x557cb8a10647] 2024-04-23T18:34:04.199 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557cb8a19d31] 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f21cadd31ca] 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 7: clone() 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 0> 2024-04-23T18:34:03.933+0000 7f21b9a1c700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: in thread 7f21b9a1c700 thread_name:md_log_replay 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21cadddd20] 2024-04-23T18:34:04.200 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2: /usr/bin/ceph-mds(+0x54938f) [0x557cb89b238f] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557cb89b5b23] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 4: (MDLog::_replay_thread()+0x617) [0x557cb8a10647] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557cb8a19d31] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f21cadd31ca] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 7: clone() 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 0> 2024-04-23T18:34:03.933+0000 7f21b9a1c700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: in thread 7f21b9a1c700 thread_name:md_log_replay 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21cadddd20] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2: /usr/bin/ceph-mds(+0x54938f) [0x557cb89b238f] 2024-04-23T18:34:04.201 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x557cb89b5b23] 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 4: (MDLog::_replay_thread()+0x617) [0x557cb8a10647] 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 5: (MDLog::ReplayThread::entry()+0x11) [0x557cb8a19d31] 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f21cadd31ca] 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 7: clone() 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[67841]: 2024-04-23T18:34:04.202 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:03 smithi122 podman[68249]: 2024-04-23 18:34:03.985861104 +0000 UTC m=+0.022253624 container died 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-23T18:34:04.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:04 smithi122 podman[68249]: 2024-04-23 18:34:04.678848599 +0000 UTC m=+0.715241100 container remove 43f993b72434494e9bff86952eb3a64f47386ac7c3d2df17f2e17d172b7b27b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-23T18:34:04.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:34:04.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:34:05.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:05 smithi106 ceph-mon[32716]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:34:05.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:05 smithi106 ceph-mon[32716]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:05.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:05 smithi122 ceph-mon[32546]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:34:05.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:05 smithi122 ceph-mon[32546]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:05.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:05 smithi033 ceph-mon[29274]: pgmap v234: 97 pgs: 97 active+clean; 5.4 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:34:05.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:05 smithi033 ceph-mon[29274]: osdmap e93: 12 total, 12 up, 12 in 2024-04-23T18:34:07.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:07 smithi106 ceph-mon[32716]: pgmap v236: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:07.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:07 smithi122 ceph-mon[32546]: pgmap v236: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:07.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:07 smithi033 ceph-mon[29274]: pgmap v236: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:09.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:09 smithi106 ceph-mon[32716]: pgmap v237: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:09.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:09 smithi122 ceph-mon[32546]: pgmap v237: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:09.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:09 smithi033 ceph-mon[29274]: pgmap v237: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:11.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:11 smithi106 ceph-mon[32716]: pgmap v238: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:11.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:11 smithi106 ceph-mon[32716]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:34:11.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:11 smithi122 ceph-mon[32546]: pgmap v238: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:11.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:11 smithi122 ceph-mon[32546]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:34:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:11 smithi033 ceph-mon[29274]: pgmap v238: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 58 KiB/s wr, 33 op/s 2024-04-23T18:34:11.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:11 smithi033 ceph-mon[29274]: osdmap e94: 12 total, 12 up, 12 in 2024-04-23T18:34:13.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:13 smithi106 ceph-mon[32716]: pgmap v240: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T18:34:13.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:13 smithi122 ceph-mon[32546]: pgmap v240: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T18:34:13.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:13 smithi033 ceph-mon[29274]: pgmap v240: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-04-23T18:34:14.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3065660412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103230 msgr2=0x7fb5bc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3065660412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103230 0x7fb5bc103610 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac009900 tx=0x7fb5ac02a480 comp rx=0 tx=0).stop 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3065660412 shutdown_connections 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3065660412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc10c570 0x7fb5bc10e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3065660412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc103b50 0x7fb5bc10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3065660412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103230 0x7fb5bc103610 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.858+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3065660412 >> 172.21.15.33:0/3065660412 conn(0x7fb5bc077fb0 msgr2=0x7fb5bc0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:14.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.859+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3065660412 shutdown_connections 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.859+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3065660412 wait complete. 2024-04-23T18:34:14.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.859+0000 7fb5c20fa640 1 Processor -- start 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.859+0000 7fb5c20fa640 1 -- start start 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc103230 0x7fb5bc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc10c570 0x7fb5bc19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc112200 con 0x7fb5bc103b50 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc112080 con 0x7fb5bc103230 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5c20fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5bc112380 con 0x7fb5bc10c570 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35080/0 (socket says 172.21.15.33:35080) 2024-04-23T18:34:14.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 -- 172.21.15.33:0/3852081196 learned_addr learned my addr 172.21.15.33:0/3852081196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc103230 0x7fb5bc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5bbfff640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc10c570 0x7fb5bc19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 -- 172.21.15.33:0/3852081196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc10c570 msgr2=0x7fb5bc19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc10c570 0x7fb5bc19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 -- 172.21.15.33:0/3852081196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc103230 msgr2=0x7fb5bc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc103230 0x7fb5bc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 -- 172.21.15.33:0/3852081196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5bc19d740 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5baffd640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.860+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b0017070 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.861+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5b0009c30 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.861+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5bc19da30 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.861+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b000f620 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.861+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5bc1a52c0 con 0x7fb5bc103b50 2024-04-23T18:34:14.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.862+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5bc0691c0 con 0x7fb5bc103b50 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.863+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5b0005030 con 0x7fb5bc103b50 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.863+0000 7fb5b8ff9640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 0x7fb5a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.864+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 0x7fb5a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.864+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7fb5b00d0430 con 0x7fb5bc103b50 2024-04-23T18:34:14.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.864+0000 7fb5bb7fe640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 0x7fb5a40a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005ab0 tx=0x7fb5ac005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:14.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:14.867+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5b009cac0 con 0x7fb5bc103b50 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.098+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5bc199aa0 con 0x7fb5bc103b50 2024-04-23T18:34:15.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.098+0000 7fb5b8ff9640 1 -- 172.21.15.33:0/3852081196 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11929 (secure 0 0 0) 0x7fb5bc199aa0 con 0x7fb5bc103b50 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:32:15.719012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25900":{"gid":25900,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/1758592712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1758592712},{"type":"v1","addr":"172.21.15.122:6835","nonce":1758592712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:34:15.091 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 msgr2=0x7fb5a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 0x7fb5a40a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac005ab0 tx=0x7fb5ac005a20 comp rx=0 tx=0).stop 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 msgr2=0x7fb5bc198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5b000d950 tx=0x7fb5b000de10 comp rx=0 tx=0).stop 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 shutdown_connections 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb5a40a3d20 0x7fb5a40a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb5bc10c570 0x7fb5bc19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc103b50 0x7fb5bc198ce0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 --2- 172.21.15.33:0/3852081196 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb5bc103230 0x7fb5bc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 >> 172.21.15.33:0/3852081196 conn(0x7fb5bc077fb0 msgr2=0x7fb5bc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 shutdown_connections 2024-04-23T18:34:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.101+0000 7fb5c20fa640 1 -- 172.21.15.33:0/3852081196 wait complete. 2024-04-23T18:34:15.104 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 284427f0 --release --await' 2024-04-23T18:34:15.104 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 284427f0 --release --await 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:15 smithi122 ceph-mon[32546]: pgmap v241: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:15 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3852081196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:14 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 3. 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:14 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:14 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:34:15.204 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 podman[68345]: 2024-04-23 18:34:15.093032852 +0000 UTC m=+0.018386746 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 -- 172.21.15.33:0/77107740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5580106170 msgr2=0x7f558010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 --2- 172.21.15.33:0/77107740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5580106170 0x7f558010dd40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5578009ca0 tx=0x7f557802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 -- 172.21.15.33:0/77107740 shutdown_connections 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 --2- 172.21.15.33:0/77107740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5580106170 0x7f558010dd40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 --2- 172.21.15.33:0/77107740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580101a50 0x7f5580105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 --2- 172.21.15.33:0/77107740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55801010a0 0x7f5580101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 -- 172.21.15.33:0/77107740 >> 172.21.15.33:0/77107740 conn(0x7f5580077eb0 msgr2=0x7f55800782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 -- 172.21.15.33:0/77107740 shutdown_connections 2024-04-23T18:34:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.221+0000 7f5584faf640 1 -- 172.21.15.33:0/77107740 wait complete. 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 Processor -- start 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 -- start start 2024-04-23T18:34:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580106170 0x7f5580198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55801110c0 con 0x7f55801010a0 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5580110f40 con 0x7f558019d010 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f5584faf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5580111240 con 0x7f5580106170 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35102/0 (socket says 172.21.15.33:35102) 2024-04-23T18:34:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 -- 172.21.15.33:0/3038200404 learned_addr learned my addr 172.21.15.33:0/3038200404 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557dd9b640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580106170 0x7f5580198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557ed9d640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 -- 172.21.15.33:0/3038200404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580106170 msgr2=0x7f5580198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580106170 0x7f5580198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.222+0000 7f557e59c640 1 -- 172.21.15.33:0/3038200404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558019d010 msgr2=0x7f558019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f557e59c640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f557e59c640 1 -- 172.21.15.33:0/3038200404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558019db00 con 0x7f55801010a0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f557e59c640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f557000d950 tx=0x7f557000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f556f7fe640 1 -- 172.21.15.33:0/3038200404 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570017070 con 0x7f55801010a0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f556f7fe640 1 -- 172.21.15.33:0/3038200404 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5570009c30 con 0x7f55801010a0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558019ddf0 con 0x7f55801010a0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f556f7fe640 1 -- 172.21.15.33:0/3038200404 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557000f620 con 0x7f55801010a0 2024-04-23T18:34:15.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.223+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55801a5630 con 0x7f55801010a0 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.225+0000 7f556f7fe640 1 -- 172.21.15.33:0/3038200404 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5570010460 con 0x7f55801010a0 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 Processor -- start 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 -- start start 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32490 0x559d02a32870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32db0 0x559d02a3cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d029976d0 con 0x559d02a3d480 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559d029341a0 con 0x559d02a32490 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f5587d9c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d02993690 con 0x559d02a32db0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f556f7fe640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 0x7f55680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35110/0 (socket says 172.21.15.33:35110) 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557dd9b640 1 -- 172.21.15.33:0/2224591047 learned_addr learned my addr 172.21.15.33:0/2224591047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557e59c640 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32db0 0x559d02a3cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557ed9d640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 0x7f55680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.226+0000 7f557ed9d640 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32490 0x559d02a32870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f556f7fe640 1 -- 172.21.15.33:0/3038200404 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f55700d03d0 con 0x7f55801010a0 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 -- 172.21.15.33:0/2224591047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32db0 msgr2=0x559d02a3cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32db0 0x559d02a3cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 -- 172.21.15.33:0/2224591047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32490 msgr2=0x559d02a32870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32490 0x559d02a32870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 -- 172.21.15.33:0/2224591047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d0293e5b0 con 0x559d02a3d480 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557dd9b640 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f557400bbf0 tx=0x7f55740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34c69086dd981a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f557ed9d640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 0x7f55680a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f557802f500 tx=0x7f5578005c30 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f556cff9640 1 -- 172.21.15.33:0/2224591047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574005020 con 0x559d02a3d480 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f556cff9640 1 -- 172.21.15.33:0/2224591047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740047a0 con 0x559d02a3d480 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.227+0000 7f556cff9640 1 -- 172.21.15.33:0/2224591047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740116a0 con 0x559d02a3d480 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- 172.21.15.33:0/2224591047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 msgr2=0x559d02a3f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f557400bbf0 tx=0x7f55740107a0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- 172.21.15.33:0/2224591047 shutdown_connections 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a3f870 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32db0 0x559d02a3cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- 172.21.15.33:0/2224591047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32490 0x559d02a32870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- 172.21.15.33:0/2224591047 >> 172.21.15.33:0/2224591047 conn(0x559d02a2e7b0 msgr2=0x559d02a2eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- 172.21.15.33:0/2224591047 shutdown_connections 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- 172.21.15.33:0/2224591047 wait complete. 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 -- start start 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 Processor -- start 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.228+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a3a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a3d480 0x559d02a3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d02a40310 con 0x559d02a32490 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559d02931b20 con 0x559d02a32db0 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559d02a40490 con 0x559d02a3d480 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 10 client.0 _ll_register_callbacks cb 0x559d02931530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35116/0 (socket says 172.21.15.33:35116) 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 -- 172.21.15.33:0/894428290 learned_addr learned my addr 172.21.15.33:0/894428290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557ed9d640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a3d480 0x559d02a3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557dd9b640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a3a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 -- 172.21.15.33:0/894428290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a3d480 msgr2=0x559d02a3a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a3d480 0x559d02a3a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 -- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 msgr2=0x559d02a3a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a3a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 -- 172.21.15.33:0/894428290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d026a9920 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f557e59c640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5570021000 tx=0x7f5570066a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed3950a7e6db0e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570020040 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f55700a4b40 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55740047a0 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f55700b5050 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 10 client.16617 fetch_fsmap learned FSMap version 33 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.229+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559d02775d30 con 0x559d02a32490 2024-04-23T18:34:15.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740047a0 con 0x559d02a32490 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f55700980b0 con 0x559d02a32490 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 10 client.16617 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 10 client.16617 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 20 client.16617 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02a4d6a0 0x559d02a4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 4 client.16617 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d02955500 con 0x559d02a4d6a0 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f557dd9b640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02a4d6a0 0x559d02a4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f5565ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.230+0000 7f557dd9b640 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02a4d6a0 0x559d02a4fa90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c1bd59cbe62510 server_cookie=481e52dbdd2b9685 in_seq=0 out_seq=0 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5565ffb640 1 -- 172.21.15.33:0/894428290 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d02955500 con 0x559d02a4d6a0 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5565ffb640 10 client.16617 handle_command_reply: tid=0 2024-04-23T18:34:15.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 client.16617 shutdown 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 msgr2=0x559d02a3ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5570021000 tx=0x7f5570066a10 comp rx=0 tx=0).stop 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 shutdown_connections 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02a4d6a0 0x559d02a4fa90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a3d480 0x559d02a3a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a3a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5565ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 --2- 172.21.15.33:0/894428290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a32490 0x559d02a3ce60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 >> 172.21.15.33:0/894428290 conn(0x559d02a2e7b0 msgr2=0x559d02a345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 shutdown_connections 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.231+0000 7f5587d9c740 1 -- 172.21.15.33:0/894428290 wait complete. 2024-04-23T18:34:15.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.232+0000 7f5587d9c740 20 client.16617 trim_cache size 0 max 16384 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 Processor -- start 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 -- start start 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55801adaf0 con 0x559d02a3d480 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55801af630 con 0x559d02a32db0 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f5587d9c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55801ae1c0 con 0x559d02a32490 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35124/0 (socket says 172.21.15.33:35124) 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 -- 172.21.15.33:0/1957909769 learned_addr learned my addr 172.21.15.33:0/1957909769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557e59c640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557dd9b640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 -- 172.21.15.33:0/1957909769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 msgr2=0x559d02a491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 -- 172.21.15.33:0/1957909769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 msgr2=0x559d02a552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 -- 172.21.15.33:0/1957909769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d02a500c0 con 0x559d02a3d480 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557e59c640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.249+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5578035000 tx=0x7f5578040070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4107124d6a91787 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5567fff640 1 -- 172.21.15.33:0/1957909769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578031750 con 0x559d02a3d480 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5567fff640 1 -- 172.21.15.33:0/1957909769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5578036040 con 0x559d02a3d480 2024-04-23T18:34:15.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5567fff640 1 -- 172.21.15.33:0/1957909769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780047f0 con 0x559d02a3d480 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- 172.21.15.33:0/1957909769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 msgr2=0x559d02a55830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5578035000 tx=0x7f5578040070 comp rx=0 tx=0).stop 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- 172.21.15.33:0/1957909769 shutdown_connections 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a55830 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1957909769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- 172.21.15.33:0/1957909769 >> 172.21.15.33:0/1957909769 conn(0x559d02a2e7b0 msgr2=0x559d02a33be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- 172.21.15.33:0/1957909769 shutdown_connections 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- 172.21.15.33:0/1957909769 wait complete. 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.250+0000 7f5587d9c740 1 -- start start 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 Processor -- start 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a45c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55801aef60 con 0x559d02a3d480 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55801ae890 con 0x559d02a32db0 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55801abaf0 con 0x559d02a32490 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5587d9c740 10 client.0 _ll_register_callbacks cb 0x7f557009bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35130/0 (socket says 172.21.15.33:35130) 2024-04-23T18:34:15.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 -- 172.21.15.33:0/1432143802 learned_addr learned my addr 172.21.15.33:0/1432143802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557dd9b640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a45c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557e59c640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 -- 172.21.15.33:0/1432143802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 msgr2=0x559d02a456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 -- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 msgr2=0x559d02a45c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a45c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 -- 172.21.15.33:0/1432143802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d02a4c240 con 0x559d02a3d480 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557e59c640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557dd9b640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a45c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f557ed9d640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5578005fc0 tx=0x7f5578032d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66144693e1fb9216 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.251+0000 7f5565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:15.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578064060 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f5578031750 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5578036040 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5578004b40 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 10 client.16623 fetch_fsmap learned FSMap version 33 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x559d02a37d80 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5578036040 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f557802f660 con 0x559d02a3d480 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 10 client.16623 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 10 client.16623 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 20 client.16623 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02ab4c80 0x559d02ab70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 4 client.16623 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "284427f0", "release": true, "await": true}] 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.252+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "284427f0", "release": true, "await": true}) -- 0x559d029a50f0 con 0x559d02ab4c80 2024-04-23T18:34:15.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.253+0000 7f557e59c640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02ab4c80 0x559d02ab70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.253+0000 7f557e59c640 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02ab4c80 0x559d02ab70a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f2a2e45580cd6d server_cookie=c435ca9f3e3f48b6 in_seq=0 out_seq=0 2024-04-23T18:34:15.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.253+0000 7f5565ffb640 10 client.16623 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.255+0000 7f5565ffb640 1 -- 172.21.15.33:0/1432143802 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559d029a50f0 con 0x559d02ab4c80 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.255+0000 7f5565ffb640 10 client.16623 handle_command_reply: tid=0 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.255+0000 7f5587d9c740 1 client.16623 shutdown 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 msgr2=0x559d02a363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f5578005fc0 tx=0x7f5578032d90 comp rx=0 tx=0).stop 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 shutdown_connections 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559d02ab4c80 0x559d02ab70a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d02a3d480 0x559d02a363e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d02a32db0 0x559d02a45c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 --2- 172.21.15.33:0/1432143802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559d02a32490 0x559d02a456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5565ffb640 0 client.16623 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 >> 172.21.15.33:0/1432143802 conn(0x559d02a2e7b0 msgr2=0x559d02a34470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 shutdown_connections 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 1 -- 172.21.15.33:0/1432143802 wait complete. 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5587d9c740 20 client.16623 trim_cache size 0 max 16384 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 33, 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:34:15.255 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 28, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "284427f0": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 28, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:34:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:15.257 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 msgr2=0x7f55680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 0x7f55680a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f557802f500 tx=0x7f5578005c30 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 msgr2=0x7f5580198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.256+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f557000d950 tx=0x7f557000de10 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 shutdown_connections 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55680a3da0 0x7f55680a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558019d010 0x7f558019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5580106170 0x7f5580198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 --2- 172.21.15.33:0/3038200404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55801010a0 0x7f5580198750 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 >> 172.21.15.33:0/3038200404 conn(0x7f5580077eb0 msgr2=0x7f558006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 shutdown_connections 2024-04-23T18:34:15.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.257+0000 7f5584faf640 1 -- 172.21.15.33:0/3038200404 wait complete. 2024-04-23T18:34:15.259 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '284427f0', seconds elapsed: 31.1 2024-04-23T18:34:15.260 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-23T18:34:15.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:15.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 -- 172.21.15.33:0/1003274394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2020101110 msgr2=0x7f20201014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 --2- 172.21.15.33:0/1003274394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2020101110 0x7f20201014f0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2010009900 tx=0x7f201002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 -- 172.21.15.33:0/1003274394 shutdown_connections 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 --2- 172.21.15.33:0/1003274394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 0x7f202010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 --2- 172.21.15.33:0/1003274394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101ac0 0x7f2020105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 --2- 172.21.15.33:0/1003274394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2020101110 0x7f20201014f0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 -- 172.21.15.33:0/1003274394 >> 172.21.15.33:0/1003274394 conn(0x7f2020077fb0 msgr2=0x7f20200783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 -- 172.21.15.33:0/1003274394 shutdown_connections 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.370+0000 7f2024a10640 1 -- 172.21.15.33:0/1003274394 wait complete. 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 Processor -- start 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 -- start start 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 0x7f2020198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2020112210 con 0x7f202019cfd0 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2020112090 con 0x7f2020106190 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f2024a10640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2020112390 con 0x7f2020101110 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35146/0 (socket says 172.21.15.33:35146) 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:45608/0 (socket says 172.21.15.33:45608) 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 -- 172.21.15.33:0/418254181 learned_addr learned my addr 172.21.15.33:0/418254181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201dd9b640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 0x7f2020198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 -- 172.21.15.33:0/418254181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 msgr2=0x7f2020198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 -- 172.21.15.33:0/418254181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 msgr2=0x7f2020198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 0x7f2020198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201ed9d640 1 -- 172.21.15.33:0/418254181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202019dac0 con 0x7f202019cfd0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.371+0000 7f201e59c640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f201ed9d640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f201800cab0 tx=0x7f201800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800ae20 con 0x7f202019cfd0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201800f900 con 0x7f202019cfd0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202019ddb0 con 0x7f202019cfd0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180189f0 con 0x7f202019cfd0 2024-04-23T18:34:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.372+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20201a5550 con 0x7f202019cfd0 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.373+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20200691c0 con 0x7f202019cfd0 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.374+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f201800fab0 con 0x7f202019cfd0 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.375+0000 7f200f7fe640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.375+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f20180d0ff0 con 0x7f202019cfd0 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.375+0000 7f201e59c640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.375+0000 7f201e59c640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 0x7f20080a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2010031440 tx=0x7f2010005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:15 smithi106 ceph-mon[32716]: pgmap v241: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-23T18:34:15.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:15 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3852081196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:15.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.378+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2018014030 con 0x7f202019cfd0 2024-04-23T18:34:15.465 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 podman[68345]: 2024-04-23 18:34:15.213063157 +0000 UTC m=+0.138417038 container create 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:34:15.466 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 podman[68345]: 2024-04-23 18:34:15.474510222 +0000 UTC m=+0.399864103 container init 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T18:34:15.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:15 smithi033 ceph-mon[29274]: pgmap v241: 97 pgs: 97 active+clean; 5.4 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.8 KiB/s wr, 12 op/s 2024-04-23T18:34:15.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:15 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3852081196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.605+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2020199a60 con 0x7f202019cfd0 2024-04-23T18:34:15.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.606+0000 7f200f7fe640 1 -- 172.21.15.33:0/418254181 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11929 (secure 0 0 0) 0x7f2020199a60 con 0x7f202019cfd0 2024-04-23T18:34:15.598 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:34:15.598 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:32:15.719012+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25900":{"gid":25900,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/1758592712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":1758592712},{"type":"v1","addr":"172.21.15.122:6835","nonce":1758592712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24551,24490,24482,24466,24515]},"id":1}]} 2024-04-23T18:34:15.599 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-23T18:34:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 msgr2=0x7f20080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 0x7f20080a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2010031440 tx=0x7f2010005b50 comp rx=0 tx=0).stop 2024-04-23T18:34:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 msgr2=0x7f202019d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f201800cab0 tx=0x7f201800cf70 comp rx=0 tx=0).stop 2024-04-23T18:34:15.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 shutdown_connections 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f20080a3d20 0x7f20080a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f202019cfd0 0x7f202019d480 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2020106190 0x7f2020198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 --2- 172.21.15.33:0/418254181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2020101110 0x7f2020198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 >> 172.21.15.33:0/418254181 conn(0x7f2020077fb0 msgr2=0x7f20200ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 shutdown_connections 2024-04-23T18:34:15.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.608+0000 7f2024a10640 1 -- 172.21.15.33:0/418254181 wait complete. 2024-04-23T18:34:15.611 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-23T18:34:15.611 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-23T18:34:15.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.730+0000 7f2e574e8640 1 -- 172.21.15.33:0/932714044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e50101a50 msgr2=0x7f2e50105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.730+0000 7f2e574e8640 1 --2- 172.21.15.33:0/932714044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e50101a50 0x7f2e50105a90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009900 tx=0x7f2e4002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- 172.21.15.33:0/932714044 shutdown_connections 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- 172.21.15.33:0/932714044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 0x7f2e5010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- 172.21.15.33:0/932714044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e50101a50 0x7f2e50105a90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- 172.21.15.33:0/932714044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- 172.21.15.33:0/932714044 >> 172.21.15.33:0/932714044 conn(0x7f2e50077f50 msgr2=0x7f2e50078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- 172.21.15.33:0/932714044 shutdown_connections 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- 172.21.15.33:0/932714044 wait complete. 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 Processor -- start 2024-04-23T18:34:15.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- start start 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 0x7f2e50199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501110b0 con 0x7f2e5019a040 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50110f30 con 0x7f2e50106170 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.731+0000 7f2e574e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50111230 con 0x7f2e501010a0 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:45620/0 (socket says 172.21.15.33:45620) 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55284640 1 -- 172.21.15.33:0/475113061 learned_addr learned my addr 172.21.15.33:0/475113061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35170/0 (socket says 172.21.15.33:35170) 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e54a83640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 0x7f2e50199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 -- 172.21.15.33:0/475113061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 msgr2=0x7f2e50199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 -- 172.21.15.33:0/475113061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 msgr2=0x7f2e50199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 0x7f2e50199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 -- 172.21.15.33:0/475113061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e501a4df0 con 0x7f2e5019a040 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55284640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e55a85640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e940 tx=0x7f2e4c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e467fc640 1 -- 172.21.15.33:0/475113061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004d20 con 0x7f2e5019a040 2024-04-23T18:34:15.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e501a50e0 con 0x7f2e5019a040 2024-04-23T18:34:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e467fc640 1 -- 172.21.15.33:0/475113061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c005020 con 0x7f2e5019a040 2024-04-23T18:34:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e467fc640 1 -- 172.21.15.33:0/475113061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0116d0 con 0x7f2e5019a040 2024-04-23T18:34:15.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.732+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e501a55b0 con 0x7f2e5019a040 2024-04-23T18:34:15.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.734+0000 7f2e467fc640 1 -- 172.21.15.33:0/475113061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e4c005460 con 0x7f2e5019a040 2024-04-23T18:34:15.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 Processor -- start 2024-04-23T18:34:15.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 -- start start 2024-04-23T18:34:15.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e467fc640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 0x7f2e3c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417759c0 0x55dc4177fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc41780090 0x55dc41782480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc416da370 con 0x55dc417750a0 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc416772e0 con 0x55dc41780090 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc416d6330 con 0x55dc417759c0 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e54a83640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 0x7f2e3c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e55284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e467fc640 1 -- 172.21.15.33:0/475113061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f2e4c0d0890 con 0x7f2e5019a040 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e55284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35182/0 (socket says 172.21.15.33:35182) 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.735+0000 7f2e55284640 1 -- 172.21.15.33:0/4229994696 learned_addr learned my addr 172.21.15.33:0/4229994696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e54a83640 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417759c0 0x55dc4177fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55a85640 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc41780090 0x55dc41782480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 -- 172.21.15.33:0/4229994696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417759c0 msgr2=0x55dc4177fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417759c0 0x55dc4177fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 -- 172.21.15.33:0/4229994696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc41780090 msgr2=0x55dc41782480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc41780090 0x55dc41782480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 -- 172.21.15.33:0/4229994696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc414b8d80 con 0x55dc417750a0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e54a83640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 0x7f2e3c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e40032ed0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e55284640 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2e48010560 tx=0x7f2e48010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d171f02f58db238f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e3b7fe640 1 -- 172.21.15.33:0/4229994696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48011760 con 0x55dc417750a0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e3b7fe640 1 -- 172.21.15.33:0/4229994696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e48011db0 con 0x55dc417750a0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.736+0000 7f2e3b7fe640 1 -- 172.21.15.33:0/4229994696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4801ce20 con 0x55dc417750a0 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/4229994696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 msgr2=0x55dc41775480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2e48010560 tx=0x7f2e48010a20 comp rx=0 tx=0).stop 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/4229994696 shutdown_connections 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc41780090 0x55dc41782480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417759c0 0x55dc4177fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/4229994696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc41775480 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/4229994696 >> 172.21.15.33:0/4229994696 conn(0x55dc417713c0 msgr2=0x55dc417717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/4229994696 shutdown_connections 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/4229994696 wait complete. 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 -- start start 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.737+0000 7f2e5a1ff740 1 Processor -- start 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc4177dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc41782e80 con 0x55dc417750a0 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc41674fb0 con 0x55dc417759c0 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc41783000 con 0x55dc41780090 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e5a1ff740 10 client.0 _ll_register_callbacks cb 0x55dc416749a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:15.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35192/0 (socket says 172.21.15.33:35192) 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 -- 172.21.15.33:0/1284857530 learned_addr learned my addr 172.21.15.33:0/1284857530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55a85640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc4177dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 -- 172.21.15.33:0/1284857530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 msgr2=0x55dc4177dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc4177dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 -- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 msgr2=0x55dc4177d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 -- 172.21.15.33:0/1284857530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc41519510 con 0x55dc417750a0 2024-04-23T18:34:15.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e55284640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801ce30 tx=0x7f2e4800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a781dcefe36df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.738+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48029060 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8542+0+0 (secure 0 0 0) 0x7f2e48005050 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e48011db0 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2e48005250 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 10 client.16647 fetch_fsmap learned FSMap version 33 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55dc413ac670 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e48011db0 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f2e480047c0 con 0x55dc417750a0 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 10 client.16647 fetch_fsmap finished waiting for FSMap version 33 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 10 client.16647 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 20 client.16647 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417947e0 0x55dc41796bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 4 client.16647 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:15.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc416e7d90 con 0x55dc417947e0 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.739+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417947e0 0x55dc41796bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.740+0000 7f2e38ff9640 10 client.16647 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.740+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417947e0 0x55dc41796bd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891b8947b9379d59 server_cookie=a53cdad7e7963f79 in_seq=0 out_seq=0 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1284857530 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc416e7d90 con 0x55dc417947e0 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e38ff9640 10 client.16647 handle_command_reply: tid=0 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 client.16647 shutdown 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 msgr2=0x55dc4177d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801ce30 tx=0x7f2e4800bd50 comp rx=0 tx=0).stop 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 shutdown_connections 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417947e0 0x55dc41796bd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc4177dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e38ff9640 0 client.16647 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1284857530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417750a0 0x55dc4177d030 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 >> 172.21.15.33:0/1284857530 conn(0x55dc417713c0 msgr2=0x55dc417771c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 shutdown_connections 2024-04-23T18:34:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1284857530 wait complete. 2024-04-23T18:34:15.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.741+0000 7f2e5a1ff740 20 client.16647 trim_cache size 0 max 16384 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 Processor -- start 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 -- start start 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 0x55dc41770d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501aba40 con 0x55dc41780090 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501ad580 con 0x55dc417759c0 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501ac110 con 0x55dc417750a0 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e55a85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e55284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 0x55dc41770d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e55284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 0x55dc41770d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:45640/0 (socket says 172.21.15.33:45640) 2024-04-23T18:34:15.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e55a85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35208/0 (socket says 172.21.15.33:35208) 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e55284640 1 -- 172.21.15.33:0/1328661075 learned_addr learned my addr 172.21.15.33:0/1328661075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.760+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 -- 172.21.15.33:0/1328661075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 msgr2=0x55dc41770d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 0x55dc41770d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 -- 172.21.15.33:0/1328661075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 msgr2=0x55dc4177d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 -- 172.21.15.33:0/1328661075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc417757f0 con 0x55dc41780090 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e55a85640 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0cdf70 tx=0x7f2e4c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe24140becceb36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e3affd640 1 -- 172.21.15.33:0/1328661075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c01f040 con 0x55dc41780090 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e3affd640 1 -- 172.21.15.33:0/1328661075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c0a0b70 con 0x55dc41780090 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.761+0000 7f2e3affd640 1 -- 172.21.15.33:0/1328661075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c099070 con 0x55dc41780090 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1328661075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 msgr2=0x55dc4179c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0cdf70 tx=0x7f2e4c087040 comp rx=0 tx=0).stop 2024-04-23T18:34:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1328661075 shutdown_connections 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc41780090 0x55dc4179c230 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417759c0 0x55dc4177d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1328661075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc417750a0 0x55dc41770d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1328661075 >> 172.21.15.33:0/1328661075 conn(0x55dc417713c0 msgr2=0x55dc4177cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1328661075 shutdown_connections 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1328661075 wait complete. 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 -- start start 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.762+0000 7f2e5a1ff740 1 Processor -- start 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417759c0 0x55dc4178c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc417888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501aceb0 con 0x55dc417759c0 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501ac7e0 con 0x55dc417750a0 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e501a9a60 con 0x55dc41780090 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e5a1ff740 10 client.0 _ll_register_callbacks cb 0x7f2e4800bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e54a83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417759c0 0x55dc4178c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:49810/0 (socket says 172.21.15.33:49810) 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 -- 172.21.15.33:0/1949418843 learned_addr learned my addr 172.21.15.33:0/1949418843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55a85640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc417888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 -- 172.21.15.33:0/1949418843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 msgr2=0x55dc417888d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc417888d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 -- 172.21.15.33:0/1949418843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417759c0 msgr2=0x55dc4178c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417759c0 0x55dc4178c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 -- 172.21.15.33:0/1949418843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc417890b0 con 0x55dc417750a0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e55284640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e48004d50 tx=0x7f2e48005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fe06f606fdfebed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.763+0000 7f2e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4803b060 con 0x55dc417750a0 2024-04-23T18:34:15.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f2e48004860 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e4c0a0b70 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2e48029040 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 10 client.26129 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dc417fb510 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e4c0a0b70 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2e4802d420 con 0x55dc417750a0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 10 client.26129 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 10 client.26129 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 20 client.26129 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417fbcd0 0x55dc417fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 4 client.26129 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.764+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55dc416e1c00 con 0x55dc417fbcd0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.765+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417fbcd0 0x55dc417fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.765+0000 7f2e38ff9640 10 client.26129 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:15.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:15.765+0000 7f2e54a83640 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417fbcd0 0x55dc417fe0f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bdb0c2324abf1d server_cookie=73643e54a6e10088 in_seq=0 out_seq=0 2024-04-23T18:34:15.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 podman[68345]: 2024-04-23 18:34:15.477277553 +0000 UTC m=+0.402631437 container start 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:34:15.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 ceph-mds[68364]: -- 172.21.15.122:0/1560245691 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c128a0fa40 con 0x55c1289e3000 2024-04-23T18:34:15.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: starting mds.f at 2024-04-23T18:34:15.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 bash[68345]: 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de 2024-04-23T18:34:15.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:34:15 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:34:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e38ff9640 1 -- 172.21.15.33:0/1949418843 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55dc416e1c00 con 0x55dc417fbcd0 2024-04-23T18:34:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e38ff9640 10 client.26129 handle_command_reply: tid=0 2024-04-23T18:34:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 client.26129 shutdown 2024-04-23T18:34:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 msgr2=0x55dc4178c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2e48004d50 tx=0x7f2e48005850 comp rx=0 tx=0).stop 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 shutdown_connections 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55dc417fbcd0 0x55dc417fe0f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc41780090 0x55dc417888d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e38ff9640 0 client.26129 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc417759c0 0x55dc4178c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 --2- 172.21.15.33:0/1949418843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc417750a0 0x55dc4178c400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 >> 172.21.15.33:0/1949418843 conn(0x55dc417713c0 msgr2=0x55dc4177d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 shutdown_connections 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 1 -- 172.21.15.33:0/1949418843 wait complete. 2024-04-23T18:34:16.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.270+0000 7f2e5a1ff740 20 client.26129 trim_cache size 0 max 16384 2024-04-23T18:34:16.265 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:34:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-04-23T18:34:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:34:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 30, 2024-04-23T18:34:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "97a135a5": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 30, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:16.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:16.267 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 msgr2=0x7f2e3c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 0x7f2e3c0a61e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e40000c00 tx=0x7f2e40032ed0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 msgr2=0x7f2e501a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c00e940 tx=0x7f2e4c00ee00 comp rx=0 tx=0).stop 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 shutdown_connections 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2e3c0a3d20 0x7f2e3c0a61e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5019a040 0x7f2e501a45f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e50106170 0x7f2e50199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 --2- 172.21.15.33:0/475113061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e501010a0 0x7f2e50199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.271+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 >> 172.21.15.33:0/475113061 conn(0x7f2e50077f50 msgr2=0x7f2e500ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 shutdown_connections 2024-04-23T18:34:16.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f2e574e8640 1 -- 172.21.15.33:0/475113061 wait complete. 2024-04-23T18:34:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1938839670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08101a90 msgr2=0x7f1e08105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1938839670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08101a90 0x7f1e08105ad0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009900 tx=0x7f1df002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1938839670 shutdown_connections 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1938839670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081061b0 0x7f1e0810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1938839670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e08101a90 0x7f1e08105ad0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1938839670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e081010e0 0x7f1e081014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1938839670 >> 172.21.15.33:0/1938839670 conn(0x7f1e080771d0 msgr2=0x7f1e080775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1938839670 shutdown_connections 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.272+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1938839670 wait complete. 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 Processor -- start 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 -- start start 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 0x7f1e08198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 0x7f1e0819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081121d0 con 0x7f1e081061b0 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112050 con 0x7f1e081010e0 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0cd9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08112350 con 0x7f1e0819cf10 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 0x7f1e08198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 0x7f1e08198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35216/0 (socket says 172.21.15.33:35216) 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:49828/0 (socket says 172.21.15.33:49828) 2024-04-23T18:34:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e05d9b640 1 -- 172.21.15.33:0/1528992701 learned_addr learned my addr 172.21.15.33:0/1528992701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.273+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 0x7f1e0819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 -- 172.21.15.33:0/1528992701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 msgr2=0x7f1e0819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 0x7f1e0819d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 -- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 msgr2=0x7f1e08198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 0x7f1e08198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 -- 172.21.15.33:0/1528992701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0819da00 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 0x7f1e0819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1dff7fe640 1 -- 172.21.15.33:0/1528992701 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800ccc0 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1dff7fe640 1 -- 172.21.15.33:0/1528992701 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df80108e0 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0819dcf0 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1dff7fe640 1 -- 172.21.15.33:0/1528992701 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8018880 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.274+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081a54e0 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.277+0000 7f1dff7fe640 1 -- 172.21.15.33:0/1528992701 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1df8018a20 con 0x7f1e081010e0 2024-04-23T18:34:16.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.277+0000 7f1e0fb82740 1 Processor -- start 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.277+0000 7f1e0fb82740 1 -- start start 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 0x563bb2246900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246e40 0x563bb22556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563bb21ab970 con 0x563bb2255c10 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bb21484a0 con 0x563bb2246520 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bb21a7930 con 0x563bb2246e40 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1dff7fe640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 0x7f1df40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 0x563bb2246900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35230/0 (socket says 172.21.15.33:35230) 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e05d9b640 1 -- 172.21.15.33:0/2855783561 learned_addr learned my addr 172.21.15.33:0/2855783561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e0659c640 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246e40 0x563bb22556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 0x7f1df40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1dff7fe640 1 -- 172.21.15.33:0/1528992701 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f1df80d0b20 con 0x7f1e081010e0 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 -- 172.21.15.33:0/2855783561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246e40 msgr2=0x563bb22556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246e40 0x563bb22556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 -- 172.21.15.33:0/2855783561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 msgr2=0x563bb2258000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e06d9d640 1 -- 172.21.15.33:0/2855783561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb1ebe210 con 0x563bb2246520 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.278+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.279+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 0x563bb2246900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000f440 tx=0x7f1e0000f900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f306b3862dd14227 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.279+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 0x7f1df40a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000fe00 tx=0x7f1e0001e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.279+0000 7f1dfcff9640 1 -- 172.21.15.33:0/2855783561 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0002b070 con 0x563bb2246520 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.279+0000 7f1dfcff9640 1 -- 172.21.15.33:0/2855783561 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e0000ee20 con 0x563bb2246520 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.279+0000 7f1dfcff9640 1 -- 172.21.15.33:0/2855783561 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00026c20 con 0x563bb2246520 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- 172.21.15.33:0/2855783561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 msgr2=0x563bb2246900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 0x563bb2246900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000f440 tx=0x7f1e0000f900 comp rx=0 tx=0).stop 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- 172.21.15.33:0/2855783561 shutdown_connections 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb2258000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246e40 0x563bb22556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/2855783561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246520 0x563bb2246900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- 172.21.15.33:0/2855783561 >> 172.21.15.33:0/2855783561 conn(0x563bb2242b30 msgr2=0x563bb2242f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- 172.21.15.33:0/2855783561 shutdown_connections 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- 172.21.15.33:0/2855783561 wait complete. 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- start start 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 Processor -- start 2024-04-23T18:34:16.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246e40 0x563bb22528a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2255c10 0x563bb2252e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563bb22587e0 con 0x563bb2246e40 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563bb21460b0 con 0x563bb2255c10 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.280+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563bb2258960 con 0x563bb2246520 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0fb82740 10 client.0 _ll_register_callbacks cb 0x563bb2133d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e05d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246e40 0x563bb22528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e06d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2255c10 0x563bb2252e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:45682/0 (socket says 172.21.15.33:45682) 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 -- 172.21.15.33:0/3112881659 learned_addr learned my addr 172.21.15.33:0/3112881659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 -- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2255c10 msgr2=0x563bb2252e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2255c10 0x563bb2252e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 -- 172.21.15.33:0/3112881659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246e40 msgr2=0x563bb22528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246e40 0x563bb22528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 -- 172.21.15.33:0/3112881659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb1f8a620 con 0x563bb2246520 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0659c640 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df809ca40 tx=0x7f1df80073b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37247f0c1d909954 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80071c0 con 0x563bb2246520 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f1df8010a90 con 0x563bb2246520 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e0000ee20 con 0x563bb2246520 2024-04-23T18:34:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1df80993b0 con 0x563bb2246520 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0fb82740 10 client.26146 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.281+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bb22647d0 con 0x563bb2246520 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e0000ee20 con 0x563bb2246520 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1df80a2040 con 0x563bb2246520 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 10 client.26146 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 10 client.26146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 20 client.26146 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb2265c40 0x563bb2268030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 4 client.26146 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bb21b9390 con 0x563bb2265c40 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb2265c40 0x563bb2268030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1dedffb640 10 client.26146 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.282+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb2265c40 0x563bb2268030 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0cc07f0035a0f server_cookie=8998ec1687061ab0 in_seq=0 out_seq=0 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1dedffb640 1 -- 172.21.15.33:0/3112881659 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563bb21b9390 con 0x563bb2265c40 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1dedffb640 10 client.26146 handle_command_reply: tid=0 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 client.26146 shutdown 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 msgr2=0x563bb22555b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1df809ca40 tx=0x7f1df80073b0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 shutdown_connections 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb2265c40 0x563bb2268030 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2255c10 0x563bb2252e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1dedffb640 0 client.26146 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246e40 0x563bb22528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/3112881659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22555b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.283+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 >> 172.21.15.33:0/3112881659 conn(0x563bb2242b30 msgr2=0x563bb22489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.284+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 shutdown_connections 2024-04-23T18:34:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.284+0000 7f1e0fb82740 1 -- 172.21.15.33:0/3112881659 wait complete. 2024-04-23T18:34:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.284+0000 7f1e0fb82740 20 client.26146 trim_cache size 0 max 16384 2024-04-23T18:34:16.290 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '97a135a5', quiesce took 1.0 seconds. Will release after: 30.0 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 Processor -- start 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 -- start start 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081ab8d0 con 0x563bb2255c10 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081ad410 con 0x563bb2246e40 2024-04-23T18:34:16.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081abfa0 con 0x563bb2246520 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35258/0 (socket says 172.21.15.33:35258) 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 -- 172.21.15.33:0/1698457091 learned_addr learned my addr 172.21.15.33:0/1698457091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 -- 172.21.15.33:0/1698457091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 msgr2=0x563bb22616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 -- 172.21.15.33:0/1698457091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 msgr2=0x563bb22550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e06d9d640 1 -- 172.21.15.33:0/1698457091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb22531b0 con 0x563bb2255c10 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.299+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1e00029040 tx=0x7f1e00011390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e9c2095de07098 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1deffff640 1 -- 172.21.15.33:0/1698457091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0002b040 con 0x563bb2255c10 2024-04-23T18:34:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1deffff640 1 -- 172.21.15.33:0/1698457091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00011030 con 0x563bb2255c10 2024-04-23T18:34:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1deffff640 1 -- 172.21.15.33:0/1698457091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000055a0 con 0x563bb2255c10 2024-04-23T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1698457091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 msgr2=0x563bb226d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1e00029040 tx=0x7f1e00011390 comp rx=0 tx=0).stop 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1698457091 shutdown_connections 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2255c10 0x563bb226d880 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1698457091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2246520 0x563bb22616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.300+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1698457091 >> 172.21.15.33:0/1698457091 conn(0x563bb2242b30 msgr2=0x563bb224df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1698457091 shutdown_connections 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1698457091 wait complete. 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- start start 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 Processor -- start 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2255c10 0x563bb2259d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081acd40 con 0x563bb2246520 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081ac670 con 0x563bb2246e40 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081a98d0 con 0x563bb2255c10 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0fb82740 10 client.0 _ll_register_callbacks cb 0x7f1df809cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35270/0 (socket says 172.21.15.33:35270) 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e0659c640 1 -- 172.21.15.33:0/1554538825 learned_addr learned my addr 172.21.15.33:0/1554538825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.301+0000 7f1e06d9d640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2255c10 0x563bb2259d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 -- 172.21.15.33:0/1554538825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2255c10 msgr2=0x563bb2259d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2255c10 0x563bb2259d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 -- 172.21.15.33:0/1554538825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 msgr2=0x563bb22c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:16.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 -- 172.21.15.33:0/1554538825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb225a480 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e05d9b640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22c5940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563bb2252100 tx=0x7f1df0032f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d70e6867099a4fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0035040 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f1df0004040 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e00011030 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1df0055050 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 10 client.26147 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563bb22ccc10 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e00011030 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1df0046550 con 0x563bb2246e40 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 10 client.26147 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:16.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 20 client.26147 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb22cd330 0x563bb22cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 4 client.26147 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.302+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563bb21b3200 con 0x563bb22cd330 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.303+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb22cd330 0x563bb22cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.303+0000 7f1dedffb640 10 client.26147 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:16.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.303+0000 7f1e0659c640 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb22cd330 0x563bb22cf720 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=265edc5ca22fb2e5 server_cookie=52b72a6ea3aa2ae2 in_seq=0 out_seq=0 2024-04-23T18:34:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1dedffb640 1 -- 172.21.15.33:0/1554538825 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563bb21b3200 con 0x563bb22cd330 2024-04-23T18:34:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1dedffb640 10 client.26147 handle_command_reply: tid=0 2024-04-23T18:34:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 client.26147 shutdown 2024-04-23T18:34:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 msgr2=0x563bb22c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22c5940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x563bb2252100 tx=0x7f1df0032f30 comp rx=0 tx=0).stop 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 shutdown_connections 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563bb22cd330 0x563bb22cf720 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563bb2255c10 0x563bb2259d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563bb2246e40 0x563bb22c5940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 --2- 172.21.15.33:0/1554538825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563bb2246520 0x563bb22c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1dedffb640 0 client.26147 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 >> 172.21.15.33:0/1554538825 conn(0x563bb2242b30 msgr2=0x563bb2241420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 shutdown_connections 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 1 -- 172.21.15.33:0/1554538825 wait complete. 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0fb82740 20 client.26147 trim_cache size 0 max 16384 2024-04-23T18:34:16.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:34:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:34:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:34:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:34:16.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 msgr2=0x7f1df40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 0x7f1df40a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000fe00 tx=0x7f1e0001e000 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 msgr2=0x7f1e08198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 shutdown_connections 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1df40a3dc0 0x7f1df40a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e0819cf10 0x7f1e0819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e081061b0 0x7f1e08198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 --2- 172.21.15.33:0/1528992701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e081010e0 0x7f1e08198650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.308+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 >> 172.21.15.33:0/1528992701 conn(0x7f1e080771d0 msgr2=0x7f1e080fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.309+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 shutdown_connections 2024-04-23T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:16.309+0000 7f1e0cd9b640 1 -- 172.21.15.33:0/1528992701 wait complete. 2024-04-23T18:34:16.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: daemon mds.f restarted 2024-04-23T18:34:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/418254181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:34:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/3736210770,v1:172.21.15.122:6835/3736210770] up:boot 2024-04-23T18:34:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:16.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: daemon mds.f restarted 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/418254181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/3736210770,v1:172.21.15.122:6835/3736210770] up:boot 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:16.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: daemon mds.f restarted 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/418254181' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: osdmap e95: 12 total, 12 up, 12 in 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/3736210770,v1:172.21.15.122:6835/3736210770] up:boot 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:34:16.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:34:17.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:17 smithi106 ceph-mon[32716]: pgmap v242: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:34:17.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:17 smithi122 ceph-mon[32546]: pgmap v242: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:34:17.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:17 smithi033 ceph-mon[29274]: pgmap v242: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:34:19.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:19 smithi106 ceph-mon[32716]: pgmap v244: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:34:19.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:19 smithi122 ceph-mon[32546]: pgmap v244: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:34:19.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:19 smithi033 ceph-mon[29274]: pgmap v244: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:34:21.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:21 smithi106 ceph-mon[32716]: pgmap v245: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-04-23T18:34:21.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:21 smithi122 ceph-mon[32546]: pgmap v245: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-04-23T18:34:21.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:21 smithi033 ceph-mon[29274]: pgmap v245: 97 pgs: 97 active+clean; 5.5 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.9 KiB/s wr, 5 op/s 2024-04-23T18:34:23.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:23 smithi106 ceph-mon[32716]: pgmap v246: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:23.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:23 smithi122 ceph-mon[32546]: pgmap v246: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:23.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:23 smithi033 ceph-mon[29274]: pgmap v246: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:25.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:25 smithi122 ceph-mon[32546]: pgmap v247: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:25.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:25 smithi033 ceph-mon[29274]: pgmap v247: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:25.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:25 smithi106 ceph-mon[32716]: pgmap v247: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.5 KiB/s wr, 11 op/s 2024-04-23T18:34:27.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:27 smithi122 ceph-mon[32546]: pgmap v248: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-23T18:34:27.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:27 smithi033 ceph-mon[29274]: pgmap v248: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-23T18:34:27.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:27 smithi106 ceph-mon[32716]: pgmap v248: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.2 KiB/s wr, 12 op/s 2024-04-23T18:34:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 -- 172.21.15.33:0/209479736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac10c560 msgr2=0x7f34ac10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 --2- 172.21.15.33:0/209479736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac10c560 0x7f34ac10e9b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009ca0 tx=0x7f34a402a950 comp rx=0 tx=0).stop 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 -- 172.21.15.33:0/209479736 shutdown_connections 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 --2- 172.21.15.33:0/209479736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac10c560 0x7f34ac10e9b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 --2- 172.21.15.33:0/209479736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac103b40 0x7f34ac10c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 --2- 172.21.15.33:0/209479736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 0x7f34ac103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 -- 172.21.15.33:0/209479736 >> 172.21.15.33:0/209479736 conn(0x7f34ac077fd0 msgr2=0x7f34ac0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 -- 172.21.15.33:0/209479736 shutdown_connections 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.407+0000 7f34b1576640 1 -- 172.21.15.33:0/209479736 wait complete. 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 Processor -- start 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 -- start start 2024-04-23T18:34:29.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 0x7f34ac1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac10c560 0x7f34ac19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112180 con 0x7f34ac103b40 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112000 con 0x7f34ac10c560 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.408+0000 7f34b1576640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac112300 con 0x7f34ac103220 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49628/0 (socket says 172.21.15.33:49628) 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 -- 172.21.15.33:0/170790818 learned_addr learned my addr 172.21.15.33:0/170790818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac10c560 0x7f34ac19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aaffd640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 0x7f34ac1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 -- 172.21.15.33:0/170790818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 msgr2=0x7f34ac1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 0x7f34ac1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 -- 172.21.15.33:0/170790818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac10c560 msgr2=0x7f34ac19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac10c560 0x7f34ac19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 -- 172.21.15.33:0/170790818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ac19d640 con 0x7f34ac103b40 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f34a000d950 tx=0x7f34a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f3493fff640 1 -- 172.21.15.33:0/170790818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0017070 con 0x7f34ac103b40 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ac19d930 con 0x7f34ac103b40 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f3493fff640 1 -- 172.21.15.33:0/170790818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a0009c30 con 0x7f34ac103b40 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f3493fff640 1 -- 172.21.15.33:0/170790818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a000f620 con 0x7f34ac103b40 2024-04-23T18:34:29.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.409+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ac1a51c0 con 0x7f34ac103b40 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f3493fff640 1 -- 172.21.15.33:0/170790818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f34a0010460 con 0x7f34ac103b40 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 Processor -- start 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 -- start start 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x55714754d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x55714755f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5571474b2970 con 0x55714754de40 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55714744f4a0 con 0x55714755cc10 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34b436a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571474ae930 con 0x55714754d520 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f3493fff640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49638/0 (socket says 172.21.15.33:49638) 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34aaffd640 1 -- 172.21.15.33:0/2589930935 learned_addr learned my addr 172.21.15.33:0/2589930935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.412+0000 7f3493fff640 1 -- 172.21.15.33:0/170790818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f34a00d02f0 con 0x7f34ac103b40 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x55714754d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x55714755f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 -- 172.21.15.33:0/2589930935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 msgr2=0x55714754d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x55714754d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 -- 172.21.15.33:0/2589930935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 msgr2=0x55714755f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x55714755f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 -- 172.21.15.33:0/2589930935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571471c5210 con 0x55714754de40 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x55714755f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f349c00bcd0 tx=0x7f349c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23a94367ec97f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 0x7f34940a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34a4030040 tx=0x7f34a4009800 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34917fa640 1 -- 172.21.15.33:0/2589930935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c005220 con 0x55714754de40 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34917fa640 1 -- 172.21.15.33:0/2589930935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c004c50 con 0x55714754de40 2024-04-23T18:34:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.413+0000 7f34917fa640 1 -- 172.21.15.33:0/2589930935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0116a0 con 0x55714754de40 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- 172.21.15.33:0/2589930935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 msgr2=0x55714755c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f349c00bcd0 tx=0x7f349c0107a0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- 172.21.15.33:0/2589930935 shutdown_connections 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x55714755f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754de40 0x55714755c6d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 --2- 172.21.15.33:0/2589930935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x55714754d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- 172.21.15.33:0/2589930935 >> 172.21.15.33:0/2589930935 conn(0x557147549b30 msgr2=0x557147549f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- 172.21.15.33:0/2589930935 shutdown_connections 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- 172.21.15.33:0/2589930935 wait complete. 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 -- start start 2024-04-23T18:34:29.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.414+0000 7f34b436a740 1 Processor -- start 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x5571475597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x557147559d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55714755f840 con 0x55714754d520 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55714744d0b0 con 0x55714755cc10 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55714755f9c0 con 0x55714754de40 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34b436a740 10 client.0 _ll_register_callbacks cb 0x55714743ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49646/0 (socket says 172.21.15.33:49646) 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 -- 172.21.15.33:0/3251018595 learned_addr learned my addr 172.21.15.33:0/3251018595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x5571475597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x557147559d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 -- 172.21.15.33:0/3251018595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 msgr2=0x5571475597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x5571475597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 -- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 msgr2=0x557147559d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x557147559d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 -- 172.21.15.33:0/3251018595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557147291620 con 0x55714754d520 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f34aaffd640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f349c005690 tx=0x7f349c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda7e6d37daa677d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f348eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c004a60 con 0x55714754d520 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.415+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f349c005010 con 0x55714754d520 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349c004c50 con 0x55714754d520 2024-04-23T18:34:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f349c005210 con 0x55714754d520 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 10 client.16683 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55714756b6c0 con 0x55714754d520 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f349c004c50 con 0x55714754d520 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f349c026590 con 0x55714754d520 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 10 client.16683 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 20 client.16683 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55714756ca70 0x55714756ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 4 client.16683 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.416+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571474c0390 con 0x55714756ca70 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.417+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55714756ca70 0x55714756ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.417+0000 7f348eff5640 10 client.16683 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.417+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55714756ca70 0x55714756ee60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434ec05a4132f3a7 server_cookie=f2b220bace40f223 in_seq=0 out_seq=0 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f348eff5640 1 -- 172.21.15.33:0/3251018595 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571474c0390 con 0x55714756ca70 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f348eff5640 10 client.16683 handle_command_reply: tid=0 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 client.16683 shutdown 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 msgr2=0x55714755c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f349c005690 tx=0x7f349c0048f0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 shutdown_connections 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55714756ca70 0x55714756ee60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714755cc10 0x557147559d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x5571475597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 --2- 172.21.15.33:0/3251018595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714754d520 0x55714755c4d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f348eff5640 0 client.16683 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 >> 172.21.15.33:0/3251018595 conn(0x557147549b30 msgr2=0x55714754f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 shutdown_connections 2024-04-23T18:34:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 1 -- 172.21.15.33:0/3251018595 wait complete. 2024-04-23T18:34:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.418+0000 7f34b436a740 20 client.16683 trim_cache size 0 max 16384 2024-04-23T18:34:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 Processor -- start 2024-04-23T18:34:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 -- start start 2024-04-23T18:34:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x5571475590a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754de40 0x55714755c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ab970 con 0x55714755cc10 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ad4b0 con 0x55714754de40 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.433+0000 7f34b436a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ac040 con 0x55714754d520 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49650/0 (socket says 172.21.15.33:49650) 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 -- 172.21.15.33:0/76107048 learned_addr learned my addr 172.21.15.33:0/76107048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34aaffd640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x5571475590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754de40 0x55714755c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 -- 172.21.15.33:0/76107048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 msgr2=0x5571475590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x5571475590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 -- 172.21.15.33:0/76107048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754de40 msgr2=0x55714755c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754de40 0x55714755c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 -- 172.21.15.33:0/76107048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55714754e380 con 0x55714755cc10 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34aaffd640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x5571475590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009ca0 tx=0x7f34a4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763e47bc0ce9b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f3490ff9640 1 -- 172.21.15.33:0/76107048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4034040 con 0x55714755cc10 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f3490ff9640 1 -- 172.21.15.33:0/76107048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a404f070 con 0x55714755cc10 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.434+0000 7f3490ff9640 1 -- 172.21.15.33:0/76107048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a404b450 con 0x55714755cc10 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- 172.21.15.33:0/76107048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 msgr2=0x557147574400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f34a4009ca0 tx=0x7f34a4005c50 comp rx=0 tx=0).stop 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- 172.21.15.33:0/76107048 shutdown_connections 2024-04-23T18:34:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147574400 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754de40 0x55714755c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- 172.21.15.33:0/76107048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754d520 0x5571475590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- 172.21.15.33:0/76107048 >> 172.21.15.33:0/76107048 conn(0x557147549b30 msgr2=0x55714754f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- 172.21.15.33:0/76107048 shutdown_connections 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- 172.21.15.33:0/76107048 wait complete. 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- start start 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 Processor -- start 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754d520 0x557147564fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x557147560350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1acde0 con 0x55714755cc10 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1ac710 con 0x55714754d520 2024-04-23T18:34:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34ac1a9950 con 0x55714754de40 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.435+0000 7f34b436a740 10 client.0 _ll_register_callbacks cb 0x7f349c00b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49652/0 (socket says 172.21.15.33:49652) 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 -- 172.21.15.33:0/2129704473 learned_addr learned my addr 172.21.15.33:0/2129704473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34aa7fc640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x557147560350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754d520 0x557147564fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 -- 172.21.15.33:0/2129704473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 msgr2=0x557147560350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x557147560350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 -- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754d520 msgr2=0x557147564fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754d520 0x557147564fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 -- 172.21.15.33:0/2129704473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557147561070 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34ab7fe640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34a402ae70 tx=0x7f34a4004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f173a1c6773920af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f348eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4003c40 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f34a4045040 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34a404f070 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f34a406a050 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34b436a740 10 client.16689 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.436+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5571475d2c90 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34a404f070 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f34a4048040 con 0x55714755cc10 2024-04-23T18:34:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 10 client.16689 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 20 client.16689 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5571475d33d0 0x5571475d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 4 client.16689 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55714755a090 con 0x5571475d33d0 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5571475d33d0 0x5571475d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f348eff5640 10 client.16689 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:29.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.437+0000 7f34aaffd640 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5571475d33d0 0x5571475d57f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d89618fb6ad752 server_cookie=b7fc741e5e67f63b in_seq=0 out_seq=0 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f348eff5640 1 -- 172.21.15.33:0/2129704473 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55714755a090 con 0x5571475d33d0 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f348eff5640 10 client.16689 handle_command_reply: tid=0 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 client.16689 shutdown 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 msgr2=0x557147560890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f34a402ae70 tx=0x7f34a4004540 comp rx=0 tx=0).stop 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 shutdown_connections 2024-04-23T18:34:29.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5571475d33d0 0x5571475d57f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55714755cc10 0x557147560890 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55714754de40 0x557147560350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f348eff5640 0 client.16689 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 --2- 172.21.15.33:0/2129704473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55714754d520 0x557147564fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 >> 172.21.15.33:0/2129704473 conn(0x557147549b30 msgr2=0x55714754f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 shutdown_connections 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.445+0000 7f34b436a740 1 -- 172.21.15.33:0/2129704473 wait complete. 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b436a740 20 client.16689 trim_cache size 0 max 16384 2024-04-23T18:34:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:34:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:34:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:34:29.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 msgr2=0x7f34940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 0x7f34940a6280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34a4030040 tx=0x7f34a4009800 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 msgr2=0x7f34ac198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f34a000d950 tx=0x7f34a000de10 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 shutdown_connections 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34940a3dc0 0x7f34940a6280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f34ac10c560 0x7f34ac19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34ac103b40 0x7f34ac198be0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 --2- 172.21.15.33:0/170790818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34ac103220 0x7f34ac1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 >> 172.21.15.33:0/170790818 conn(0x7f34ac077fd0 msgr2=0x7f34ac0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 shutdown_connections 2024-04-23T18:34:29.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:29.446+0000 7f34b1576640 1 -- 172.21.15.33:0/170790818 wait complete. 2024-04-23T18:34:29.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:29 smithi122 ceph-mon[32546]: pgmap v249: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:29.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:29 smithi033 ceph-mon[29274]: pgmap v249: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:29.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:29 smithi106 ceph-mon[32716]: pgmap v249: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:31.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:31 smithi122 ceph-mon[32546]: pgmap v250: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:31.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:31 smithi033 ceph-mon[29274]: pgmap v250: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:31.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:31 smithi106 ceph-mon[32716]: pgmap v250: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.2 KiB/s wr, 10 op/s 2024-04-23T18:34:33.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:33 smithi122 ceph-mon[32546]: pgmap v251: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-23T18:34:33.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:33 smithi033 ceph-mon[29274]: pgmap v251: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-23T18:34:33.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:33 smithi106 ceph-mon[32716]: pgmap v251: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.1 KiB/s wr, 16 op/s 2024-04-23T18:34:35.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:35 smithi122 ceph-mon[32546]: pgmap v252: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:35.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:35 smithi033 ceph-mon[29274]: pgmap v252: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:35.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:35 smithi106 ceph-mon[32716]: pgmap v252: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:34:37.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:37 smithi122 ceph-mon[32546]: pgmap v253: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-04-23T18:34:37.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:37 smithi033 ceph-mon[29274]: pgmap v253: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-04-23T18:34:37.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:37 smithi106 ceph-mon[32716]: pgmap v253: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-04-23T18:34:39.422 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:39 smithi106 ceph-mon[32716]: pgmap v254: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:39.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:39 smithi122 ceph-mon[32546]: pgmap v254: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:39.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:39 smithi033 ceph-mon[29274]: pgmap v254: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:41.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:41 smithi122 ceph-mon[32546]: pgmap v255: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:41.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:41 smithi033 ceph-mon[29274]: pgmap v255: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:41.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:41 smithi106 ceph-mon[32716]: pgmap v255: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:34:42.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:42.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:42.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:42.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:34:42.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:42.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:42.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:42.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/529501873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb641010e0 msgr2=0x7fcb641014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/529501873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb641010e0 0x7fcb641014c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009900 tx=0x7fcb6002a410 comp rx=0 tx=0).stop 2024-04-23T18:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/529501873 shutdown_connections 2024-04-23T18:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/529501873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 0x7fcb6410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/529501873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb64101a90 0x7fcb64105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/529501873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb641010e0 0x7fcb641014c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/529501873 >> 172.21.15.33:0/529501873 conn(0x7fcb640771d0 msgr2=0x7fcb640775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/529501873 shutdown_connections 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.539+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/529501873 wait complete. 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 Processor -- start 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 -- start start 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb641010e0 0x7fcb641986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 0x7fcb64198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64111120 con 0x7fcb6419cf70 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb64110fa0 con 0x7fcb641010e0 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb6b7f9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641112a0 con 0x7fcb641061b0 2024-04-23T18:34:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39042/0 (socket says 172.21.15.33:39042) 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 -- 172.21.15.33:0/2232051601 learned_addr learned my addr 172.21.15.33:0/2232051601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb68d94640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 0x7fcb64198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69595640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb641010e0 0x7fcb641986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 -- 172.21.15.33:0/2232051601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 msgr2=0x7fcb64198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 0x7fcb64198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 -- 172.21.15.33:0/2232051601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb641010e0 msgr2=0x7fcb641986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb641010e0 0x7fcb641986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.540+0000 7fcb69d96640 1 -- 172.21.15.33:0/2232051601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb6419da60 con 0x7fcb6419cf70 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb69d96640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eaa0 tx=0x7fcb5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb5a7fc640 1 -- 172.21.15.33:0/2232051601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c00cdb0 con 0x7fcb6419cf70 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb5a7fc640 1 -- 172.21.15.33:0/2232051601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c005020 con 0x7fcb6419cf70 2024-04-23T18:34:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb6419dd50 con 0x7fcb6419cf70 2024-04-23T18:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb5a7fc640 1 -- 172.21.15.33:0/2232051601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0116d0 con 0x7fcb6419cf70 2024-04-23T18:34:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.541+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb641a5540 con 0x7fcb6419cf70 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.543+0000 7fcb5a7fc640 1 -- 172.21.15.33:0/2232051601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb5c0051d0 con 0x7fcb6419cf70 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 Processor -- start 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 -- start start 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb5a7fc640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 0x7fcb500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fb12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fb1800 0x557b14fb3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b14f0b970 con 0x557b14fa6620 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557b14ea84a0 con 0x557b14fa6f40 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b14f07930 con 0x557b14fb1800 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb68d94640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 0x7fcb500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb5a7fc640 1 -- 172.21.15.33:0/2232051601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fcb5c0d09e0 con 0x7fcb6419cf70 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39054/0 (socket says 172.21.15.33:39054) 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 -- 172.21.15.33:0/3468073209 learned_addr learned my addr 172.21.15.33:0/3468073209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69d96640 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fb1800 0x557b14fb3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb68d94640 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fb12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 -- 172.21.15.33:0/3468073209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fb1800 msgr2=0x557b14fb3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fb1800 0x557b14fb3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 -- 172.21.15.33:0/3468073209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 msgr2=0x557b14fb12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fb12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.544+0000 7fcb69595640 1 -- 172.21.15.33:0/3468073209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b14c1e210 con 0x557b14fa6620 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb68d94640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 0x7fcb500a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb540058c0 tx=0x7fcb5400bb70 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb69595640 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009900 tx=0x7fcb60032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fe5e7567df2caa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb4f7fe640 1 -- 172.21.15.33:0/3468073209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6003f070 con 0x557b14fa6620 2024-04-23T18:34:42.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb4f7fe640 1 -- 172.21.15.33:0/3468073209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60032e30 con 0x557b14fa6620 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb4f7fe640 1 -- 172.21.15.33:0/3468073209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb60035070 con 0x557b14fa6620 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3468073209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 msgr2=0x557b14fa6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009900 tx=0x7fcb60032c40 comp rx=0 tx=0).stop 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3468073209 shutdown_connections 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fb1800 0x557b14fb3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fb12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3468073209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fa6620 0x557b14fa6a00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.545+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3468073209 >> 172.21.15.33:0/3468073209 conn(0x557b14fa2b30 msgr2=0x557b14fa2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3468073209 shutdown_connections 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3468073209 wait complete. 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- start start 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 Processor -- start 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6620 0x557b14fb1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b14fb43d0 con 0x557b14fb1800 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557b14ea60b0 con 0x557b14fa6f40 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b14fb4550 con 0x557b14fa6620 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.546+0000 7fcb6e5e0740 10 client.0 _ll_register_callbacks cb 0x557b14e93d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39058/0 (socket says 172.21.15.33:39058) 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 -- 172.21.15.33:0/4184964326 learned_addr learned my addr 172.21.15.33:0/4184964326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69595640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6620 0x557b14fb1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb68d94640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 -- 172.21.15.33:0/4184964326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6620 msgr2=0x557b14fb1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6620 0x557b14fb1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 -- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 msgr2=0x557b14fae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 -- 172.21.15.33:0/4184964326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b14cea620 con 0x557b14fb1800 2024-04-23T18:34:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0bd180 tx=0x7fcb5c09d110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a0d81f2d1754fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c01f070 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fcb5c087190 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb60032e30 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb5c0a5600 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb6e5e0740 10 client.16701 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b14fc0260 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.547+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb60032e30 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcb5c00f040 con 0x557b14fb1800 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 10 client.16701 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 20 client.16701 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b14fc1770 0x557b14fc3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 4 client.16701 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b14f19390 con 0x557b14fc1770 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb69595640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b14fc1770 0x557b14fc3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb4cff9640 10 client.16701 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.548+0000 7fcb69595640 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b14fc1770 0x557b14fc3ba0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e329cadc8ce73463 server_cookie=2bd7b7210f1182cf in_seq=0 out_seq=0 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4184964326 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b14f19390 con 0x557b14fc1770 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb4cff9640 10 client.16701 handle_command_reply: tid=0 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 client.16701 shutdown 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 msgr2=0x557b14fae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0bd180 tx=0x7fcb5c09d110 comp rx=0 tx=0).stop 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 shutdown_connections 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b14fc1770 0x557b14fc3ba0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fae8a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6f40 0x557b14fae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb4cff9640 0 client.16701 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4184964326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6620 0x557b14fb1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 >> 172.21.15.33:0/4184964326 conn(0x557b14fa2b30 msgr2=0x557b14fa88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 shutdown_connections 2024-04-23T18:34:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4184964326 wait complete. 2024-04-23T18:34:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.549+0000 7fcb6e5e0740 20 client.16701 trim_cache size 0 max 16384 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 Processor -- start 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 -- start start 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fbd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fc94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ada40 con 0x557b14fb1800 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641af580 con 0x557b14fa6620 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.564+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ae110 con 0x557b14fa6f40 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39070/0 (socket says 172.21.15.33:39070) 2024-04-23T18:34:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 -- 172.21.15.33:0/3554373424 learned_addr learned my addr 172.21.15.33:0/3554373424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb68d94640 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fc94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69595640 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fbd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 -- 172.21.15.33:0/3554373424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 msgr2=0x557b14fc94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fc94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 -- 172.21.15.33:0/3554373424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 msgr2=0x557b14fbd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fbd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 -- 172.21.15.33:0/3554373424 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b14fac8b0 con 0x557b14fb1800 2024-04-23T18:34:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb69d96640 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c098df0 tx=0x7fcb5c0a0ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51019cae3da5efa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb4effd640 1 -- 172.21.15.33:0/3554373424 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0b5070 con 0x557b14fb1800 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb4effd640 1 -- 172.21.15.33:0/3554373424 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c0988f0 con 0x557b14fb1800 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.565+0000 7fcb4effd640 1 -- 172.21.15.33:0/3554373424 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c01f040 con 0x557b14fb1800 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3554373424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 msgr2=0x557b14fc99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c098df0 tx=0x7fcb5c0a0ec0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3554373424 shutdown_connections 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fc99f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fc94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/3554373424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fbd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3554373424 >> 172.21.15.33:0/3554373424 conn(0x557b14fa2b30 msgr2=0x557b14fa1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3554373424 shutdown_connections 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/3554373424 wait complete. 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 -- start start 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.566+0000 7fcb6e5e0740 1 Processor -- start 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fb9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fb4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641aeeb0 con 0x557b14fb1800 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641ae7e0 con 0x557b14fa6620 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb641aba20 con 0x557b14fa6f40 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 10 client.0 _ll_register_callbacks cb 0x7fcb5c09c2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39086/0 (socket says 172.21.15.33:39086) 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 -- 172.21.15.33:0/4243787352 learned_addr learned my addr 172.21.15.33:0/4243787352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69595640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fb9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb68d94640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fb4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 -- 172.21.15.33:0/4243787352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 msgr2=0x557b14fb4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fb4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 -- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 msgr2=0x557b14fb9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fb9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 -- 172.21.15.33:0/4243787352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b14fb5ac0 con 0x557b14fb1800 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69595640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fb9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb69d96640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c09c690 tx=0x7fcb5c09aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749067b016cec8af server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0b5070 con 0x557b14fb1800 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fcb5c0bbd50 con 0x557b14fb1800 2024-04-23T18:34:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb5c0988f0 con 0x557b14fb1800 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb5c01f070 con 0x557b14fb1800 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 10 client.16707 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.567+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b15027c60 con 0x557b14fb1800 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb5c0988f0 con 0x557b14fb1800 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcb5c067300 con 0x557b14fb1800 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 10 client.16707 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 20 client.16707 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b150280e0 0x557b1502a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 4 client.16707 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b14f13200 con 0x557b150280e0 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb69595640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b150280e0 0x557b1502a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb69595640 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b150280e0 0x557b1502a4d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a744329e3fbadfe server_cookie=a353c6c294efb072 in_seq=0 out_seq=0 2024-04-23T18:34:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.568+0000 7fcb4cff9640 10 client.16707 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb4cff9640 1 -- 172.21.15.33:0/4243787352 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b14f13200 con 0x557b150280e0 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb4cff9640 10 client.16707 handle_command_reply: tid=0 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 client.16707 shutdown 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 msgr2=0x557b14fb5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c09c690 tx=0x7fcb5c09aea0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 shutdown_connections 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557b150280e0 0x557b1502a4d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b14fb1800 0x557b14fb5380 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b14fa6f40 0x557b14fb4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb4cff9640 0 client.16707 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 --2- 172.21.15.33:0/4243787352 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557b14fa6620 0x557b14fb9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 >> 172.21.15.33:0/4243787352 conn(0x557b14fa2b30 msgr2=0x557b14fa1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 shutdown_connections 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 1 -- 172.21.15.33:0/4243787352 wait complete. 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.572+0000 7fcb6e5e0740 20 client.16707 trim_cache size 0 max 16384 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:34:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 msgr2=0x7fcb500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 0x7fcb500a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb540058c0 tx=0x7fcb5400bb70 comp rx=0 tx=0).stop 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 msgr2=0x7fcb6419d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00eaa0 tx=0x7fcb5c00ef60 comp rx=0 tx=0).stop 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 shutdown_connections 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb500a3dc0 0x7fcb500a6280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb6419cf70 0x7fcb6419d420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb641061b0 0x7fcb64198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 --2- 172.21.15.33:0/2232051601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb641010e0 0x7fcb641986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 >> 172.21.15.33:0/2232051601 conn(0x7fcb640771d0 msgr2=0x7fcb6406aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 shutdown_connections 2024-04-23T18:34:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:42.573+0000 7fcb6b7f9640 1 -- 172.21.15.33:0/2232051601 wait complete. 2024-04-23T18:34:42.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:34:42.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:34:42.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:34:42.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:34:43.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:43 smithi122 ceph-mon[32546]: pgmap v256: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-23T18:34:43.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:43 smithi033 ceph-mon[29274]: pgmap v256: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-23T18:34:43.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:43 smithi106 ceph-mon[32716]: pgmap v256: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.9 KiB/s wr, 14 op/s 2024-04-23T18:34:45.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:45 smithi122 ceph-mon[32546]: pgmap v257: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:45.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:45 smithi033 ceph-mon[29274]: pgmap v257: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:45.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:45 smithi106 ceph-mon[32716]: pgmap v257: 97 pgs: 97 active+clean; 5.5 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:34:46.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 -- 172.21.15.33:0/1032179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac10c570 msgr2=0x7f8dac10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 --2- 172.21.15.33:0/1032179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac10c570 0x7f8dac10e9e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009ca0 tx=0x7f8da402a950 comp rx=0 tx=0).stop 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 -- 172.21.15.33:0/1032179810 shutdown_connections 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 --2- 172.21.15.33:0/1032179810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac10c570 0x7f8dac10e9e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 --2- 172.21.15.33:0/1032179810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac103b50 0x7f8dac10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 --2- 172.21.15.33:0/1032179810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 0x7f8dac103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 -- 172.21.15.33:0/1032179810 >> 172.21.15.33:0/1032179810 conn(0x7f8dac077fb0 msgr2=0x7f8dac0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 -- 172.21.15.33:0/1032179810 shutdown_connections 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.415+0000 7f8db2843640 1 -- 172.21.15.33:0/1032179810 wait complete. 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 Processor -- start 2024-04-23T18:34:46.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 -- start start 2024-04-23T18:34:46.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 0x7f8dac198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac10c570 0x7f8dac19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac1121d0 con 0x7f8dac103b50 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac112050 con 0x7f8dac10c570 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db2843640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac112350 con 0x7f8dac103230 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8dab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8dab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39104/0 (socket says 172.21.15.33:39104) 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8dab7fe640 1 -- 172.21.15.33:0/831178263 learned_addr learned my addr 172.21.15.33:0/831178263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8db0de0640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac10c570 0x7f8dac19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.416+0000 7f8dabfff640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 0x7f8dac198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 -- 172.21.15.33:0/831178263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 msgr2=0x7f8dac198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 0x7f8dac198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 -- 172.21.15.33:0/831178263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac10c570 msgr2=0x7f8dac19cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac10c570 0x7f8dac19cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 -- 172.21.15.33:0/831178263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac19d6d0 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8dab7fe640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9400cd80 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac19d9c0 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d940108e0 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94018980 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.417+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac1a5250 con 0x7f8dac103b50 2024-04-23T18:34:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.418+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dac0691c0 con 0x7f8dac103b50 2024-04-23T18:34:46.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.419+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d94010430 con 0x7f8dac103b50 2024-04-23T18:34:46.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.420+0000 7f8da97fa640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 0x7f8d980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.420+0000 7f8dabfff640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 0x7f8d980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.420+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f8d940d0b90 con 0x7f8dac103b50 2024-04-23T18:34:46.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.420+0000 7f8dabfff640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 0x7f8d980a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009b60 tx=0x7f8d9c005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.425+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8d9409d1e0 con 0x7f8dac103b50 2024-04-23T18:34:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.651+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dac1076f0 con 0x7f8dac103b50 2024-04-23T18:34:46.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.652+0000 7f8da97fa640 1 -- 172.21.15.33:0/831178263 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11929 (secure 0 0 0) 0x7f8d9409cae0 con 0x7f8dac103b50 2024-04-23T18:34:46.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:34:46.644 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:34:15.747206+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26116":{"gid":26116,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/3736210770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3736210770},{"type":"v1","addr":"172.21.15.122:6835","nonce":3736210770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24466,24482,24551,24490]},"id":1}]} 2024-04-23T18:34:46.645 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-04-23T18:34:46.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 msgr2=0x7f8d980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 0x7f8d980a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009b60 tx=0x7f8d9c005f90 comp rx=0 tx=0).stop 2024-04-23T18:34:46.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 msgr2=0x7f8dac198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8d9400e9f0 tx=0x7f8d9400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 shutdown_connections 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8d980a3d20 0x7f8d980a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8dac10c570 0x7f8dac19cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac103b50 0x7f8dac198c70 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 --2- 172.21.15.33:0/831178263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dac103230 0x7f8dac198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 >> 172.21.15.33:0/831178263 conn(0x7f8dac077fb0 msgr2=0x7f8dac0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 shutdown_connections 2024-04-23T18:34:46.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.654+0000 7f8db2843640 1 -- 172.21.15.33:0/831178263 wait complete. 2024-04-23T18:34:46.657 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 97a135a5 --release --await' 2024-04-23T18:34:46.657 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 97a135a5 --release --await 2024-04-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 -- 172.21.15.33:0/803413696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 msgr2=0x7f980406ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 --2- 172.21.15.33:0/803413696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f980406ac20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009900 tx=0x7f97f402a410 comp rx=0 tx=0).stop 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 -- 172.21.15.33:0/803413696 shutdown_connections 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 --2- 172.21.15.33:0/803413696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9804107270 0x7f980410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 --2- 172.21.15.33:0/803413696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f980406b1f0 0x7f9804106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 --2- 172.21.15.33:0/803413696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f980406ac20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 -- 172.21.15.33:0/803413696 >> 172.21.15.33:0/803413696 conn(0x7f9804077eb0 msgr2=0x7f98040782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 -- 172.21.15.33:0/803413696 shutdown_connections 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 -- 172.21.15.33:0/803413696 wait complete. 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.774+0000 7f980a9d8640 1 Processor -- start 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 -- start start 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 0x7f9804198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980419cff0 0x7f980419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98041121b0 con 0x7f980406a840 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112030 con 0x7f980419cff0 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f980a9d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112330 con 0x7f9804107270 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38524/0 (socket says 172.21.15.33:38524) 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 -- 172.21.15.33:0/4008014437 learned_addr learned my addr 172.21.15.33:0/4008014437 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f98037fe640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 0x7f9804198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 -- 172.21.15.33:0/4008014437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 msgr2=0x7f9804198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 0x7f9804198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9808f75640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980419cff0 0x7f980419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 -- 172.21.15.33:0/4008014437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980419cff0 msgr2=0x7f980419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980419cff0 0x7f980419d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f9803fff640 1 -- 172.21.15.33:0/4008014437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980419dae0 con 0x7f980406a840 2024-04-23T18:34:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.775+0000 7f98037fe640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 0x7f9804198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f9803fff640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97f4006010 tx=0x7f97f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f98017fa640 1 -- 172.21.15.33:0/4008014437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f403f030 con 0x7f980406a840 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f98017fa640 1 -- 172.21.15.33:0/4008014437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f4003800 con 0x7f980406a840 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980419dd70 con 0x7f980406a840 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f98017fa640 1 -- 172.21.15.33:0/4008014437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4035070 con 0x7f980406a840 2024-04-23T18:34:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.776+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98041a5530 con 0x7f980406a840 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f98017fa640 1 -- 172.21.15.33:0/4008014437 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f97f4003cb0 con 0x7f980406a840 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 Processor -- start 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 -- start start 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e58d8b6d0 con 0x558e58e26b70 2024-04-23T18:34:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558e58d281a0 con 0x558e58e35890 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f980d7c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e58d87690 con 0x558e58e261c0 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38534/0 (socket says 172.21.15.33:38534) 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 -- 172.21.15.33:0/99853912 learned_addr learned my addr 172.21.15.33:0/99853912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9808f75640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f98037fe640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f98017fa640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 -- 172.21.15.33:0/99853912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 msgr2=0x558e58e265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 -- 172.21.15.33:0/99853912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 msgr2=0x558e58e37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.779+0000 7f9803fff640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f9803fff640 1 -- 172.21.15.33:0/99853912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e58d325b0 con 0x558e58e26b70 2024-04-23T18:34:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f98037fe640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f9808f75640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f98017fa640 1 -- 172.21.15.33:0/4008014437 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f97f40f1220 con 0x7f980406a840 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f9803fff640 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97f40ee8f0 tx=0x7f97f408bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa65c310d43e903e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f97ea7fc640 1 -- 172.21.15.33:0/99853912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4049070 con 0x558e58e26b70 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f9808f75640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00bd50 tx=0x7f97fc004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f97ea7fc640 1 -- 172.21.15.33:0/99853912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f40bd8d0 con 0x558e58e26b70 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.780+0000 7f97ea7fc640 1 -- 172.21.15.33:0/99853912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40c1580 con 0x558e58e26b70 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- 172.21.15.33:0/99853912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 msgr2=0x558e58e35350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97f40ee8f0 tx=0x7f97f408bb80 comp rx=0 tx=0).stop 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- 172.21.15.33:0/99853912 shutdown_connections 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e26b70 0x558e58e35350 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 --2- 172.21.15.33:0/99853912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- 172.21.15.33:0/99853912 >> 172.21.15.33:0/99853912 conn(0x558e58e227b0 msgr2=0x558e58e22b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- 172.21.15.33:0/99853912 shutdown_connections 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- 172.21.15.33:0/99853912 wait complete. 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 -- start start 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.781+0000 7f980d7c7740 1 Processor -- start 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e26b70 0x558e58e325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e58e38410 con 0x558e58e35890 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558e58d25b20 con 0x558e58e26b70 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e58e38590 con 0x558e58e261c0 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 10 client.0 _ll_register_callbacks cb 0x558e58d25530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38540/0 (socket says 172.21.15.33:38540) 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 -- 172.21.15.33:0/3947040266 learned_addr learned my addr 172.21.15.33:0/3947040266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f98037fe640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e26b70 0x558e58e325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9803fff640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 -- 172.21.15.33:0/3947040266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 msgr2=0x558e58e352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 -- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e26b70 msgr2=0x558e58e325b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e26b70 0x558e58e325b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 -- 172.21.15.33:0/3947040266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e58a9d920 con 0x558e58e35890 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f9808f75640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f97fc011e40 tx=0x7f97fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d623dc66c029d6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f97e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc01c430 con 0x558e58e35890 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f97fc01c5d0 con 0x558e58e35890 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.782+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f40bd8d0 con 0x558e58e35890 2024-04-23T18:34:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f97fc01c7d0 con 0x558e58e35890 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 10 client.16725 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e58b69d30 con 0x558e58e35890 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f40bd8d0 con 0x558e58e35890 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f97fc02fac0 con 0x558e58e35890 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 10 client.16725 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 10 client.16725 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 20 client.16725 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58e45740 0x558e58e47b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 4 client.16725 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e58d49500 con 0x558e58e45740 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.783+0000 7f9803fff640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58e45740 0x558e58e47b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.784+0000 7f97e37fe640 10 client.16725 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.784+0000 7f9803fff640 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58e45740 0x558e58e47b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd6a3fad9f504860 server_cookie=bedf8fa613248a9c in_seq=0 out_seq=0 2024-04-23T18:34:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.784+0000 7f97e37fe640 1 -- 172.21.15.33:0/3947040266 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e58d49500 con 0x558e58e45740 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.784+0000 7f97e37fe640 10 client.16725 handle_command_reply: tid=0 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.784+0000 7f980d7c7740 1 client.16725 shutdown 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 msgr2=0x558e58e32cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f97fc011e40 tx=0x7f97fc004ab0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 shutdown_connections 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58e45740 0x558e58e47b60 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e32cf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e26b70 0x558e58e325b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f97e37fe640 0 client.16725 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 --2- 172.21.15.33:0/3947040266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e261c0 0x558e58e352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 >> 172.21.15.33:0/3947040266 conn(0x558e58e227b0 msgr2=0x558e58e28620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 shutdown_connections 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 1 -- 172.21.15.33:0/3947040266 wait complete. 2024-04-23T18:34:46.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.785+0000 7f980d7c7740 20 client.16725 trim_cache size 0 max 16384 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 Processor -- start 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 -- start start 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e34df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98041ab920 con 0x558e58e261c0 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98041ad460 con 0x558e58e35890 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f980d7c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98041abff0 con 0x558e58e26b70 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f9803fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38546/0 (socket says 172.21.15.33:38546) 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.802+0000 7f9803fff640 1 -- 172.21.15.33:0/1901120813 learned_addr learned my addr 172.21.15.33:0/1901120813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9808f75640 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 -- 172.21.15.33:0/1901120813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 msgr2=0x558e58e34df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e34df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 -- 172.21.15.33:0/1901120813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 msgr2=0x558e58e35330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e35330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 -- 172.21.15.33:0/1901120813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e58e26910 con 0x558e58e261c0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9808f75640 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f9803fff640 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f97f4004650 tx=0x7f97f40bce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f76dccccbc337 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f97e9ffb640 1 -- 172.21.15.33:0/1901120813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4049040 con 0x558e58e261c0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f97e9ffb640 1 -- 172.21.15.33:0/1901120813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f40f4e50 con 0x558e58e261c0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f97e9ffb640 1 -- 172.21.15.33:0/1901120813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f40f4a80 con 0x558e58e261c0 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f980d7c7740 1 -- 172.21.15.33:0/1901120813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 msgr2=0x558e58e21850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.803+0000 7f980d7c7740 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f97f4004650 tx=0x7f97f40bce10 comp rx=0 tx=0).stop 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- 172.21.15.33:0/1901120813 shutdown_connections 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e35890 0x558e58e35330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e34df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- 172.21.15.33:0/1901120813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e261c0 0x558e58e21850 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- 172.21.15.33:0/1901120813 >> 172.21.15.33:0/1901120813 conn(0x558e58e227b0 msgr2=0x558e58e21140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- 172.21.15.33:0/1901120813 shutdown_connections 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- 172.21.15.33:0/1901120813 wait complete. 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- start start 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 Processor -- start 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e261c0 0x558e58e36d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98041acd90 con 0x558e58e35890 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f98041ac6c0 con 0x558e58e261c0 2024-04-23T18:34:46.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98041a9920 con 0x558e58e26b70 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f980d7c7740 10 client.0 _ll_register_callbacks cb 0x7f97fc011ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.804+0000 7f9808f75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38552/0 (socket says 172.21.15.33:38552) 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 -- 172.21.15.33:0/4145685948 learned_addr learned my addr 172.21.15.33:0/4145685948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f98037fe640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9803fff640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e261c0 0x558e58e36d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 -- 172.21.15.33:0/4145685948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 msgr2=0x558e58e372a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e372a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 -- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e261c0 msgr2=0x558e58e36d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e261c0 0x558e58e36d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 -- 172.21.15.33:0/4145685948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e58e3c420 con 0x558e58e35890 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f9808f75640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f97fc011860 tx=0x7f97fc010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04fc41b686bd68e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:46.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f97e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00ec90 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7f97fc00ee30 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f40f4e50 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f97fc01c970 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f980d7c7740 10 client.16731 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558e58eac410 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.805+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f40f4e50 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f97fc02f500 con 0x558e58e35890 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 10 client.16731 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 10 client.16731 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 20 client.16731 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58eacc10 0x558e58eaf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 4 client.16731 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "97a135a5", "release": true, "await": true}] 2024-04-23T18:34:46.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "97a135a5", "release": true, "await": true}) -- 0x558e58d990f0 con 0x558e58eacc10 2024-04-23T18:34:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f9803fff640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58eacc10 0x558e58eaf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f97e37fe640 10 client.16731 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:46.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.806+0000 7f9803fff640 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58eacc10 0x558e58eaf030 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85f35cfc4126c8f server_cookie=f322669f62791423 in_seq=0 out_seq=0 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f97e37fe640 1 -- 172.21.15.33:0/4145685948 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558e58d990f0 con 0x558e58eacc10 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f97e37fe640 10 client.16731 handle_command_reply: tid=0 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 client.16731 shutdown 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 msgr2=0x558e58e3bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f97fc011860 tx=0x7f97fc010e10 comp rx=0 tx=0).stop 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 shutdown_connections 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e58eacc10 0x558e58eaf030 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e58e35890 0x558e58e3bee0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e58e26b70 0x558e58e372a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 --2- 172.21.15.33:0/4145685948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e58e261c0 0x558e58e36d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f97e37fe640 0 client.16731 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 >> 172.21.15.33:0/4145685948 conn(0x558e58e227b0 msgr2=0x558e58e30a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 shutdown_connections 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 1 -- 172.21.15.33:0/4145685948 wait complete. 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.809+0000 7f980d7c7740 20 client.16731 trim_cache size 0 max 16384 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 34, 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:34:46.806 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 32, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "97a135a5": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 32, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 90.0, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:34:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RELEASED", 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:34:46.808 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 msgr2=0x7f97ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00bd50 tx=0x7f97fc004a40 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 msgr2=0x7f9804198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97f4006010 tx=0x7f97f40042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 shutdown_connections 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f97ec0a3d20 0x7f97ec0a61e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980419cff0 0x7f980419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9804107270 0x7f9804198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 --2- 172.21.15.33:0/4008014437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f980406a840 0x7f9804198730 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 >> 172.21.15.33:0/4008014437 conn(0x7f9804077eb0 msgr2=0x7f9804108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 shutdown_connections 2024-04-23T18:34:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:34:46.810+0000 7f980a9d8640 1 -- 172.21.15.33:0/4008014437 wait complete. 2024-04-23T18:34:46.812 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '97a135a5', seconds elapsed: 31.6 2024-04-23T18:34:47.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:47 smithi122 ceph-mon[32546]: pgmap v258: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:34:47.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:47 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/831178263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:47.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:47 smithi033 ceph-mon[29274]: pgmap v258: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:34:47.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:47 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/831178263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:47.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:47 smithi106 ceph-mon[32716]: pgmap v258: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:34:47.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:47 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/831178263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:34:49.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:49 smithi122 ceph-mon[32546]: pgmap v259: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:49.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:49 smithi033 ceph-mon[29274]: pgmap v259: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:49.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:49 smithi106 ceph-mon[32716]: pgmap v259: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:51.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:51 smithi122 ceph-mon[32546]: pgmap v260: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:51.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:51 smithi033 ceph-mon[29274]: pgmap v260: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:51.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:51 smithi106 ceph-mon[32716]: pgmap v260: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:34:53.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:53 smithi122 ceph-mon[32546]: pgmap v261: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T18:34:53.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:53 smithi033 ceph-mon[29274]: pgmap v261: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T18:34:53.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:53 smithi106 ceph-mon[32716]: pgmap v261: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 14 op/s 2024-04-23T18:34:55.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:55 smithi122 ceph-mon[32546]: pgmap v262: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-04-23T18:34:55.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:55 smithi033 ceph-mon[29274]: pgmap v262: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-04-23T18:34:55.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:55 smithi106 ceph-mon[32716]: pgmap v262: 97 pgs: 97 active+clean; 5.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 -- 172.21.15.33:0/4021794125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 msgr2=0x7fa0681012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 --2- 172.21.15.33:0/4021794125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa0681012b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa058009900 tx=0x7fa05802a3f0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 -- 172.21.15.33:0/4021794125 shutdown_connections 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 --2- 172.21.15.33:0/4021794125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 --2- 172.21.15.33:0/4021794125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa068101880 0x7fa0681058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 --2- 172.21.15.33:0/4021794125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa0681012b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 -- 172.21.15.33:0/4021794125 >> 172.21.15.33:0/4021794125 conn(0x7fa0680fc770 msgr2=0x7fa0680feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 -- 172.21.15.33:0/4021794125 shutdown_connections 2024-04-23T18:34:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.672+0000 7fa06759e640 1 -- 172.21.15.33:0/4021794125 wait complete. 2024-04-23T18:34:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.673+0000 7fa06759e640 1 Processor -- start 2024-04-23T18:34:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.673+0000 7fa06759e640 1 -- start start 2024-04-23T18:34:55.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.673+0000 7fa06759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.673+0000 7fa06759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06819ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa06819efd0 0x7fa06819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa068110f00 con 0x7fa068100ed0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06759e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa068110d80 con 0x7fa06819efd0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06759e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa068111080 con 0x7fa068105fa0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38564/0 (socket says 172.21.15.33:38564) 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 -- 172.21.15.33:0/764291086 learned_addr learned my addr 172.21.15.33:0/764291086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa066d9d640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa06819efd0 0x7fa06819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa065d9b640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06819ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 -- 172.21.15.33:0/764291086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 msgr2=0x7fa06819ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06819ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 -- 172.21.15.33:0/764291086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa06819efd0 msgr2=0x7fa06819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa06819efd0 0x7fa06819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 -- 172.21.15.33:0/764291086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06819fac0 con 0x7fa068100ed0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa065d9b640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06659c640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa058005940 tx=0x7fa0580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa0577fe640 1 -- 172.21.15.33:0/764291086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05803f030 con 0x7fa068100ed0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa0577fe640 1 -- 172.21.15.33:0/764291086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa058003cb0 con 0x7fa068100ed0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06819fd50 con 0x7fa068100ed0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.674+0000 7fa0577fe640 1 -- 172.21.15.33:0/764291086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058035070 con 0x7fa068100ed0 2024-04-23T18:34:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.675+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0681a7510 con 0x7fa068100ed0 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.677+0000 7fa0577fe640 1 -- 172.21.15.33:0/764291086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa058042030 con 0x7fa068100ed0 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.677+0000 7fa06fbea740 1 Processor -- start 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.677+0000 7fa06fbea740 1 -- start start 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.677+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bb5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.677+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06fbea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6b1a970 con 0x55c1e6bb6100 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06fbea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6ab74a0 con 0x55c1e6bc09c0 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06fbea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6b16930 con 0x55c1e6bb57e0 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa0577fe640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 0x7fa0500a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa066d9d640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 0x7fa0500a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa065d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38570/0 (socket says 172.21.15.33:38570) 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 -- 172.21.15.33:0/328046342 learned_addr learned my addr 172.21.15.33:0/328046342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa066d9d640 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bb5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa0577fe640 1 -- 172.21.15.33:0/764291086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fa0580f1090 con 0x7fa068100ed0 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 -- 172.21.15.33:0/328046342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 msgr2=0x55c1e6bb5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bb5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 -- 172.21.15.33:0/328046342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 msgr2=0x55c1e6bc2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 -- 172.21.15.33:0/328046342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e682d210 con 0x55c1e6bb6100 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa065d9b640 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:34:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa066d9d640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 0x7fa0500a61d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0600098b0 tx=0x7fa060009d90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa06659c640 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0580bcd40 tx=0x7fa05808fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9f08cd317d8a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.678+0000 7fa054ff9640 1 -- 172.21.15.33:0/328046342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058049040 con 0x55c1e6bb6100 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa054ff9640 1 -- 172.21.15.33:0/328046342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05803b460 con 0x55c1e6bb6100 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa054ff9640 1 -- 172.21.15.33:0/328046342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580c5cb0 con 0x55c1e6bb6100 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 -- 172.21.15.33:0/328046342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 msgr2=0x55c1e6bc0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa0580bcd40 tx=0x7fa05808fb30 comp rx=0 tx=0).stop 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 -- 172.21.15.33:0/328046342 shutdown_connections 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc0480 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 --2- 172.21.15.33:0/328046342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bb5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 -- 172.21.15.33:0/328046342 >> 172.21.15.33:0/328046342 conn(0x55c1e6bb1cf0 msgr2=0x55c1e6bb20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 -- 172.21.15.33:0/328046342 shutdown_connections 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.679+0000 7fa06fbea740 1 -- 172.21.15.33:0/328046342 wait complete. 2024-04-23T18:34:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 -- start start 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 Processor -- start 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb6100 0x55c1e6bbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bbdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6bc33b0 con 0x55c1e6bb57e0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6ab50b0 con 0x55c1e6bc09c0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c1e6bc3530 con 0x55c1e6bb6100 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06fbea740 10 client.0 _ll_register_callbacks cb 0x55c1e6aa2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38580/0 (socket says 172.21.15.33:38580) 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa06659c640 1 -- 172.21.15.33:0/876560159 learned_addr learned my addr 172.21.15.33:0/876560159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.680+0000 7fa066d9d640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bbdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa065d9b640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb6100 0x55c1e6bbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 -- 172.21.15.33:0/876560159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb6100 msgr2=0x55c1e6bbd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb6100 0x55c1e6bbd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 -- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 msgr2=0x55c1e6bbdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bbdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 -- 172.21.15.33:0/876560159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e68f9620 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06659c640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa0580bb2f0 tx=0x7fa058005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae05d895cb881d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580d5020 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fa0580cd430 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c1e6b28390 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa058049070 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06fbea740 10 client.16743 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1e6bc33b0 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.681+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c1e6b28390 con 0x55c1e6bb57e0 2024-04-23T18:34:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa0580bd2a0 con 0x55c1e6bb57e0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 10 client.16743 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 10 client.16743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 20 client.16743 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6bd04e0 0x55c1e6bd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 4 client.16743 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1e6b22200 con 0x55c1e6bd04e0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa065d9b640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6bd04e0 0x55c1e6bd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa065d9b640 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6bd04e0 0x55c1e6bd28d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedf988377b127ac server_cookie=ecc98e54161be54d in_seq=0 out_seq=0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.682+0000 7fa04dffb640 10 client.16743 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa04dffb640 1 -- 172.21.15.33:0/876560159 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1e6b22200 con 0x55c1e6bd04e0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa04dffb640 10 client.16743 handle_command_reply: tid=0 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 client.16743 shutdown 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 msgr2=0x55c1e6bc02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa0580bb2f0 tx=0x7fa058005df0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 shutdown_connections 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6bd04e0 0x55c1e6bd28d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bbdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb6100 0x55c1e6bbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa04dffb640 0 client.16743 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 --2- 172.21.15.33:0/876560159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc02a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 >> 172.21.15.33:0/876560159 conn(0x55c1e6bb1cf0 msgr2=0x55c1e6bb7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 shutdown_connections 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 1 -- 172.21.15.33:0/876560159 wait complete. 2024-04-23T18:34:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.683+0000 7fa06fbea740 20 client.16743 trim_cache size 0 max 16384 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.698+0000 7fa06fbea740 1 Processor -- start 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.698+0000 7fa06fbea740 1 -- start start 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bcc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bd81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681afa60 con 0x55c1e6bb6100 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681b15a0 con 0x55c1e6bb57e0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06fbea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681b0130 con 0x55c1e6bc09c0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38590/0 (socket says 172.21.15.33:38590) 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 -- 172.21.15.33:0/3883662628 learned_addr learned my addr 172.21.15.33:0/3883662628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06659c640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa066d9d640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bd81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 -- 172.21.15.33:0/3883662628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bc09c0 msgr2=0x55c1e6bd81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bd81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 -- 172.21.15.33:0/3883662628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 msgr2=0x55c1e6bcc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bcc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 -- 172.21.15.33:0/3883662628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e6bb5f30 con 0x55c1e6bb6100 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa06659c640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bcc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa065d9b640 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa05c008cb0 tx=0x7fa05c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dacfb8718b76bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa04ffff640 1 -- 172.21.15.33:0/3883662628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c01c430 con 0x55c1e6bb6100 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa04ffff640 1 -- 172.21.15.33:0/3883662628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c005020 con 0x55c1e6bb6100 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.699+0000 7fa04ffff640 1 -- 172.21.15.33:0/3883662628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c004b90 con 0x55c1e6bb6100 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- 172.21.15.33:0/3883662628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 msgr2=0x55c1e6bb0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa05c008cb0 tx=0x7fa05c012820 comp rx=0 tx=0).stop 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- 172.21.15.33:0/3883662628 shutdown_connections 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bd81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bb6100 0x55c1e6bb0550 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 --2- 172.21.15.33:0/3883662628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bcc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- 172.21.15.33:0/3883662628 >> 172.21.15.33:0/3883662628 conn(0x55c1e6bb1cf0 msgr2=0x55c1e6bbd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- 172.21.15.33:0/3883662628 shutdown_connections 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- 172.21.15.33:0/3883662628 wait complete. 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 -- start start 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.700+0000 7fa06fbea740 1 Processor -- start 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681b0ed0 con 0x55c1e6bc09c0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681b0800 con 0x55c1e6bb6100 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0681ada40 con 0x55c1e6bb57e0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06fbea740 10 client.0 _ll_register_callbacks cb 0x7fa0600126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38604/0 (socket says 172.21.15.33:38604) 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 -- 172.21.15.33:0/2131741823 learned_addr learned my addr 172.21.15.33:0/2131741823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa065d9b640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa06659c640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 -- 172.21.15.33:0/2131741823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 msgr2=0x55c1e6bc8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 -- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb6100 msgr2=0x55c1e6bc5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 -- 172.21.15.33:0/2131741823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e6bc6880 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa066d9d640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa060010df0 tx=0x7fa060009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d00ead9ba8b64e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.701+0000 7fa04dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060003e00 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c005020 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8498+0+0 (secure 0 0 0) 0x7fa0600046c0 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa06002a430 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 10 client.16749 fetch_fsmap learned FSMap version 34 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1e6c36930 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c005020 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa06001f070 con 0x55c1e6bc09c0 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 10 client.16749 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:34:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 10 client.16749 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 20 client.16749 populate_metadata read hostname 'smithi033' 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6c370c0 0x55c1e6c394e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 4 client.16749 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1e6a9e5f0 con 0x55c1e6c370c0 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.702+0000 7fa06659c640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6c370c0 0x55c1e6c394e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.703+0000 7fa04dffb640 10 client.16749 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.703+0000 7fa06659c640 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6c370c0 0x55c1e6c394e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf569e1a02f65 server_cookie=27633e8aabb19f3c in_seq=0 out_seq=0 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.710+0000 7fa04dffb640 1 -- 172.21.15.33:0/2131741823 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1e6a9e5f0 con 0x55c1e6c370c0 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.710+0000 7fa04dffb640 10 client.16749 handle_command_reply: tid=0 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.710+0000 7fa06fbea740 1 client.16749 shutdown 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 msgr2=0x55c1e6bc6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa060010df0 tx=0x7fa060009600 comp rx=0 tx=0).stop 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 shutdown_connections 2024-04-23T18:34:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c1e6c370c0 0x55c1e6c394e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1e6bc09c0 0x55c1e6bc6210 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1e6bb6100 0x55c1e6bc5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 --2- 172.21.15.33:0/2131741823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1e6bb57e0 0x55c1e6bc8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa04dffb640 0 client.16749 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 >> 172.21.15.33:0/2131741823 conn(0x55c1e6bb1cf0 msgr2=0x55c1e6bcb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 shutdown_connections 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 1 -- 172.21.15.33:0/2131741823 wait complete. 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06fbea740 20 client.16749 trim_cache size 0 max 16384 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:34:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:34:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:34:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 msgr2=0x7fa0500a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 0x7fa0500a61d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa0600098b0 tx=0x7fa060009d90 comp rx=0 tx=0).stop 2024-04-23T18:34:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 msgr2=0x7fa06819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa058005940 tx=0x7fa0580042f0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 shutdown_connections 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa0500a3d10 0x7fa0500a61d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa06819efd0 0x7fa06819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa068105fa0 0x7fa06819ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 --2- 172.21.15.33:0/764291086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa068100ed0 0x7fa06819a710 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 >> 172.21.15.33:0/764291086 conn(0x7fa0680fc770 msgr2=0x7fa0680fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.711+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 shutdown_connections 2024-04-23T18:34:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:34:55.712+0000 7fa06759e640 1 -- 172.21.15.33:0/764291086 wait complete. 2024-04-23T18:34:57.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:57 smithi122 ceph-mon[32546]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:34:57.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:57 smithi033 ceph-mon[29274]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:34:57.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:57 smithi106 ceph-mon[32716]: pgmap v263: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 16 op/s 2024-04-23T18:34:59.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:34:59 smithi122 ceph-mon[32546]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:34:59.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:34:59 smithi033 ceph-mon[29274]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:34:59.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:34:59 smithi106 ceph-mon[32716]: pgmap v264: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:35:01.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:01 smithi122 ceph-mon[32546]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:35:01.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:01 smithi033 ceph-mon[29274]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:35:01.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:01 smithi106 ceph-mon[32716]: pgmap v265: 97 pgs: 97 active+clean; 5.7 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-23T18:35:03.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:03 smithi122 ceph-mon[32546]: pgmap v266: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-23T18:35:03.512 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:03 smithi033 ceph-mon[29274]: pgmap v266: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-23T18:35:03.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:03 smithi106 ceph-mon[32716]: pgmap v266: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 70 KiB/s wr, 36 op/s 2024-04-23T18:35:05.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:05 smithi122 ceph-mon[32546]: pgmap v267: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 31 op/s 2024-04-23T18:35:05.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:05 smithi033 ceph-mon[29274]: pgmap v267: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 31 op/s 2024-04-23T18:35:05.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:05 smithi106 ceph-mon[32716]: pgmap v267: 97 pgs: 97 active+clean; 5.8 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 65 KiB/s wr, 31 op/s 2024-04-23T18:35:07.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:07 smithi122 ceph-mon[32546]: pgmap v268: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-04-23T18:35:07.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:07 smithi033 ceph-mon[29274]: pgmap v268: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-04-23T18:35:07.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:07 smithi106 ceph-mon[32716]: pgmap v268: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 78 KiB/s wr, 38 op/s 2024-04-23T18:35:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 -- 172.21.15.33:0/2720832052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c103b00 msgr2=0x7f705c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 --2- 172.21.15.33:0/2720832052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c103b00 0x7f705c10bfe0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7044009900 tx=0x7f704402a510 comp rx=0 tx=0).stop 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 -- 172.21.15.33:0/2720832052 shutdown_connections 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 --2- 172.21.15.33:0/2720832052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c10c520 0x7f705c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 --2- 172.21.15.33:0/2720832052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c103b00 0x7f705c10bfe0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 --2- 172.21.15.33:0/2720832052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c1031e0 0x7f705c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 -- 172.21.15.33:0/2720832052 >> 172.21.15.33:0/2720832052 conn(0x7f705c077fd0 msgr2=0x7f705c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 -- 172.21.15.33:0/2720832052 shutdown_connections 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.811+0000 7f7060cdc640 1 -- 172.21.15.33:0/2720832052 wait complete. 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 Processor -- start 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 -- start start 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c103b00 0x7f705c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 0x7f705c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112210 con 0x7f705c1031e0 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112090 con 0x7f705c103b00 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7060cdc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f705c112390 con 0x7f705c10c520 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37518/0 (socket says 172.21.15.33:37518) 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f705a59c640 1 -- 172.21.15.33:0/3675544733 learned_addr learned my addr 172.21.15.33:0/3675544733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 0x7f705c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.812+0000 7f7053fff640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c103b00 0x7f705c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 -- 172.21.15.33:0/3675544733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 msgr2=0x7f705c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 0x7f705c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 -- 172.21.15.33:0/3675544733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c103b00 msgr2=0x7f705c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c103b00 0x7f705c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 -- 172.21.15.33:0/3675544733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c19b030 con 0x7f705c1031e0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 0x7f705c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f705a59c640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b4a0 tx=0x7f704c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f70537fe640 1 -- 172.21.15.33:0/3675544733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c014070 con 0x7f705c1031e0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c19b320 con 0x7f705c1031e0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f70537fe640 1 -- 172.21.15.33:0/3675544733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c010ac0 con 0x7f705c1031e0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f70537fe640 1 -- 172.21.15.33:0/3675544733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c017c30 con 0x7f705c1031e0 2024-04-23T18:35:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.813+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1a7630 con 0x7f705c1031e0 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.816+0000 7f70537fe640 1 -- 172.21.15.33:0/3675544733 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f704c017070 con 0x7f705c1031e0 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.816+0000 7f70639ea740 1 Processor -- start 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 -- start start 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3587a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 0x55e6f3594bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6f34ec970 con 0x55e6f3587f40 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e6f34894a0 con 0x55e6f3592800 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70639ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6f34e8930 con 0x55e6f3587620 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37520/0 (socket says 172.21.15.33:37520) 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 -- 172.21.15.33:0/136448745 learned_addr learned my addr 172.21.15.33:0/136448745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70537fe640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 0x7f70480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f7053fff640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 0x55e6f3594bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705ad9d640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3587a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 0x7f70480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f70537fe640 1 -- 172.21.15.33:0/3675544733 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f704c0cfbd0 con 0x7f705c1031e0 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 -- 172.21.15.33:0/136448745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 msgr2=0x55e6f3587a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3587a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 -- 172.21.15.33:0/136448745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 msgr2=0x55e6f3594bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 0x55e6f3594bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f705a59c640 1 -- 172.21.15.33:0/136448745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f31ff210 con 0x55e6f3587f40 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.817+0000 7f7053fff640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 0x55e6f3594bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.818+0000 7f705a59c640 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f704c0cd300 tx=0x7f704c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf88f168713a769 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.818+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 0x7f70480a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7054013bc0 tx=0x7f705400a370 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.818+0000 7f7050ff9640 1 -- 172.21.15.33:0/136448745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c01e070 con 0x55e6f3587f40 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.818+0000 7f7050ff9640 1 -- 172.21.15.33:0/136448745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c09c2e0 con 0x55e6f3587f40 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.818+0000 7f7050ff9640 1 -- 172.21.15.33:0/136448745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c010c70 con 0x55e6f3587f40 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- 172.21.15.33:0/136448745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 msgr2=0x55e6f35922c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f704c0cd300 tx=0x7f704c00bf90 comp rx=0 tx=0).stop 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- 172.21.15.33:0/136448745 shutdown_connections 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3592800 0x55e6f3594bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587f40 0x55e6f35922c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- 172.21.15.33:0/136448745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3587a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- 172.21.15.33:0/136448745 >> 172.21.15.33:0/136448745 conn(0x55e6f3583b30 msgr2=0x55e6f3583f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- 172.21.15.33:0/136448745 shutdown_connections 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- 172.21.15.33:0/136448745 wait complete. 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- start start 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 Processor -- start 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3592190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f358f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6f35953d0 con 0x55e6f3592800 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e6f34870b0 con 0x55e6f3587f40 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6f3595550 con 0x55e6f3587620 2024-04-23T18:35:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.819+0000 7f70639ea740 10 client.0 _ll_register_callbacks cb 0x55e6f3474d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37534/0 (socket says 172.21.15.33:37534) 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 -- 172.21.15.33:0/3044320678 learned_addr learned my addr 172.21.15.33:0/3044320678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705a59c640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3592190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f7053fff640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f358f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 -- 172.21.15.33:0/3044320678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 msgr2=0x55e6f3592190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3592190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 -- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 msgr2=0x55e6f358f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f358f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 -- 172.21.15.33:0/3044320678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f32cb620 con 0x55e6f3592800 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f7053fff640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f358f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705a59c640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3592190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f705ad9d640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f705400a510 tx=0x7f7054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765f59cd0285dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f7041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7054004d20 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f705401e650 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.820+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c09c2e0 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f705401e850 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 10 client.16761 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6f35a1380 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c09c2e0 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f705401c040 con 0x55e6f3592800 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 10 client.16761 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 20 client.16761 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f35a2840 0x55e6f35a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 4 client.16761 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:08.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f34fa390 con 0x55e6f35a2840 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.821+0000 7f705a59c640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f35a2840 0x55e6f35a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.822+0000 7f705a59c640 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f35a2840 0x55e6f35a4c30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e61684820859f46 server_cookie=74a13238334d2db6 in_seq=0 out_seq=0 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.822+0000 7f7041ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.822+0000 7f7041ffb640 1 -- 172.21.15.33:0/3044320678 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6f34fa390 con 0x55e6f35a2840 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.822+0000 7f7041ffb640 10 client.16761 handle_command_reply: tid=0 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 client.16761 shutdown 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 msgr2=0x55e6f358fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f705400a510 tx=0x7f7054004a40 comp rx=0 tx=0).stop 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 shutdown_connections 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f35a2840 0x55e6f35a4c30 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3592800 0x55e6f358fa00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f358f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f7041ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 --2- 172.21.15.33:0/3044320678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3587620 0x55e6f3592190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 >> 172.21.15.33:0/3044320678 conn(0x55e6f3583b30 msgr2=0x55e6f3589860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 shutdown_connections 2024-04-23T18:35:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 1 -- 172.21.15.33:0/3044320678 wait complete. 2024-04-23T18:35:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.823+0000 7f70639ea740 20 client.16761 trim_cache size 0 max 16384 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 Processor -- start 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 -- start start 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f3583430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f35aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1aff30 con 0x55e6f3587620 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b1a70 con 0x55e6f3587f40 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f70639ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b0600 con 0x55e6f3592800 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37542/0 (socket says 172.21.15.33:37542) 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 -- 172.21.15.33:0/1595087970 learned_addr learned my addr 172.21.15.33:0/1595087970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f7053fff640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f3583430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 -- 172.21.15.33:0/1595087970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 msgr2=0x55e6f35aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705ad9d640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f35aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f35aa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 -- 172.21.15.33:0/1595087970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 msgr2=0x55e6f3583430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f3583430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705a59c640 1 -- 172.21.15.33:0/1595087970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f3594e20 con 0x55e6f3587620 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f7053fff640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f3583430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.838+0000 7f705ad9d640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f35aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f705a59c640 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f704c09b8e0 tx=0x7f704c097de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53cd81b86305823 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f7043fff640 1 -- 172.21.15.33:0/1595087970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b5070 con 0x55e6f3587620 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f7043fff640 1 -- 172.21.15.33:0/1595087970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c09ff90 con 0x55e6f3587620 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f7043fff640 1 -- 172.21.15.33:0/1595087970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c01e040 con 0x55e6f3587620 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 -- 172.21.15.33:0/1595087970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 msgr2=0x55e6f359e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f704c09b8e0 tx=0x7f704c097de0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 -- 172.21.15.33:0/1595087970 shutdown_connections 2024-04-23T18:35:08.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f35aa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f3583430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 --2- 172.21.15.33:0/1595087970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f359e220 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 -- 172.21.15.33:0/1595087970 >> 172.21.15.33:0/1595087970 conn(0x55e6f3583b30 msgr2=0x55e6f359db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 -- 172.21.15.33:0/1595087970 shutdown_connections 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.839+0000 7f70639ea740 1 -- 172.21.15.33:0/1595087970 wait complete. 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 -- start start 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 Processor -- start 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f35986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f3598bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b13a0 con 0x55e6f3587620 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1b0cd0 con 0x55e6f3587f40 2024-04-23T18:35:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1adf30 con 0x55e6f3592800 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f70639ea740 10 client.0 _ll_register_callbacks cb 0x7f7054010d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37544/0 (socket says 172.21.15.33:37544) 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 -- 172.21.15.33:0/1505007219 learned_addr learned my addr 172.21.15.33:0/1505007219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f7053fff640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f35986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705ad9d640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f3598bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 -- 172.21.15.33:0/1505007219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 msgr2=0x55e6f3598bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f3598bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 -- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 msgr2=0x55e6f35986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f35986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705a59c640 1 -- 172.21.15.33:0/1505007219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f3607dc0 con 0x55e6f3587620 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f705ad9d640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f3598bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:08.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.840+0000 7f7053fff640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f35986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f705a59c640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f704c098040 tx=0x7f704c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0d650e4298bfca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c01e030 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f704c0bc0f0 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c09ff90 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f704c0b5040 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 10 client.16767 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6f3608fc0 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c09ff90 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f704c0a43c0 con 0x55e6f3587620 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 10 client.16767 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 10 client.16767 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 20 client.16767 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f3609240 0x55e6f360b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 4 client.16767 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.841+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6f34f4200 con 0x55e6f3609240 2024-04-23T18:35:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.842+0000 7f7053fff640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f3609240 0x55e6f360b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.842+0000 7f7053fff640 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f3609240 0x55e6f360b630 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b630763f76aeb2 server_cookie=4ab079a165422065 in_seq=0 out_seq=0 2024-04-23T18:35:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.842+0000 7f7041ffb640 10 client.16767 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.850+0000 7f7041ffb640 1 -- 172.21.15.33:0/1505007219 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6f34f4200 con 0x55e6f3609240 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.850+0000 7f7041ffb640 10 client.16767 handle_command_reply: tid=0 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.850+0000 7f70639ea740 1 client.16767 shutdown 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 msgr2=0x55e6f3598170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f704c098040 tx=0x7f704c0047d0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 shutdown_connections 2024-04-23T18:35:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6f3609240 0x55e6f360b630 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6f3592800 0x55e6f3598bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6f3587f40 0x55e6f35986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 --2- 172.21.15.33:0/1505007219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6f3587620 0x55e6f3598170 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7041ffb640 0 client.16767 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 >> 172.21.15.33:0/1505007219 conn(0x55e6f3583b30 msgr2=0x55e6f359db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 shutdown_connections 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 1 -- 172.21.15.33:0/1505007219 wait complete. 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f70639ea740 20 client.16767 trim_cache size 0 max 16384 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:35:08.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:35:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:35:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 msgr2=0x7f70480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 0x7f70480a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7054013bc0 tx=0x7f705400a370 comp rx=0 tx=0).stop 2024-04-23T18:35:08.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 msgr2=0x7f705c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b4a0 tx=0x7f704c00b960 comp rx=0 tx=0).stop 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 shutdown_connections 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70480a3dc0 0x7f70480a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f705c10c520 0x7f705c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f705c103b00 0x7f705c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 --2- 172.21.15.33:0/3675544733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f705c1031e0 0x7f705c1a06b0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.851+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 >> 172.21.15.33:0/3675544733 conn(0x7f705c077fd0 msgr2=0x7f705c108490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.852+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 shutdown_connections 2024-04-23T18:35:08.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:08.852+0000 7f7060cdc640 1 -- 172.21.15.33:0/3675544733 wait complete. 2024-04-23T18:35:09.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:09 smithi122 ceph-mon[32546]: pgmap v269: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-04-23T18:35:09.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:09 smithi122 ceph-mon[32546]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:35:09.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:09 smithi033 ceph-mon[29274]: pgmap v269: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-04-23T18:35:09.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:09 smithi033 ceph-mon[29274]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:35:09.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:09 smithi106 ceph-mon[32716]: pgmap v269: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 68 KiB/s wr, 31 op/s 2024-04-23T18:35:09.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:09 smithi106 ceph-mon[32716]: osdmap e96: 12 total, 12 up, 12 in 2024-04-23T18:35:10.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:10 smithi106 ceph-mon[32716]: pgmap v271: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:35:10.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:10 smithi122 ceph-mon[32546]: pgmap v271: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:35:10.729 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:10 smithi033 ceph-mon[29274]: pgmap v271: 97 pgs: 97 active+clean; 5.9 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:35:12.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:12 smithi122 ceph-mon[32546]: pgmap v272: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:12.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:12 smithi033 ceph-mon[29274]: pgmap v272: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:13.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:12 smithi106 ceph-mon[32716]: pgmap v272: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:14.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:14 smithi122 ceph-mon[32546]: pgmap v273: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:14.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:14 smithi033 ceph-mon[29274]: pgmap v273: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:15.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:14 smithi106 ceph-mon[32716]: pgmap v273: 97 pgs: 97 active+clean; 6.1 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:17.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:16 smithi106 ceph-mon[32716]: pgmap v274: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:17.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:16 smithi106 ceph-mon[32716]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:35:17.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:16 smithi122 ceph-mon[32546]: pgmap v274: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:17.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:16 smithi122 ceph-mon[32546]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:35:17.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:16 smithi033 ceph-mon[29274]: pgmap v274: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:17.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:16 smithi033 ceph-mon[29274]: osdmap e97: 12 total, 12 up, 12 in 2024-04-23T18:35:19.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:18 smithi106 ceph-mon[32716]: pgmap v276: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:19.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:18 smithi122 ceph-mon[32546]: pgmap v276: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:19.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:18 smithi033 ceph-mon[29274]: pgmap v276: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 18 op/s 2024-04-23T18:35:20.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:20 smithi122 ceph-mon[32546]: pgmap v277: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:21.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:20 smithi106 ceph-mon[32716]: pgmap v277: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:21.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:20 smithi033 ceph-mon[29274]: pgmap v277: 97 pgs: 97 active+clean; 6.2 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-23T18:35:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 -- 172.21.15.33:0/2317606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c101aa0 msgr2=0x7f196c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 --2- 172.21.15.33:0/2317606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c101aa0 0x7f196c105a90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f195c009ca0 tx=0x7f195c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 -- 172.21.15.33:0/2317606453 shutdown_connections 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 --2- 172.21.15.33:0/2317606453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c106170 0x7f196c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 --2- 172.21.15.33:0/2317606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c101aa0 0x7f196c105a90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 --2- 172.21.15.33:0/2317606453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c1010f0 0x7f196c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 -- 172.21.15.33:0/2317606453 >> 172.21.15.33:0/2317606453 conn(0x7f196c077fd0 msgr2=0x7f196c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 -- 172.21.15.33:0/2317606453 shutdown_connections 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.951+0000 7f197308a640 1 -- 172.21.15.33:0/2317606453 wait complete. 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 Processor -- start 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 -- start start 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c1010f0 0x7f196c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c19cfa0 0x7f196c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1121f0 con 0x7f196c106170 2024-04-23T18:35:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f196c112070 con 0x7f196c1010f0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f197308a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f196c112370 con 0x7f196c19cfa0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59740/0 (socket says 172.21.15.33:59740) 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 -- 172.21.15.33:0/2949648603 learned_addr learned my addr 172.21.15.33:0/2949648603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f1970e26640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c1010f0 0x7f196c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f1971627640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c19cfa0 0x7f196c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 -- 172.21.15.33:0/2949648603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c19cfa0 msgr2=0x7f196c19d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c19cfa0 0x7f196c19d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 -- 172.21.15.33:0/2949648603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c1010f0 msgr2=0x7f196c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c1010f0 0x7f196c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.952+0000 7f196bfff640 1 -- 172.21.15.33:0/2949648603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196c19da90 con 0x7f196c106170 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f196bfff640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f195c006010 tx=0x7f195c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f1969ffb640 1 -- 172.21.15.33:0/2949648603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c003bf0 con 0x7f196c106170 2024-04-23T18:35:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196c19dd20 con 0x7f196c106170 2024-04-23T18:35:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f1969ffb640 1 -- 172.21.15.33:0/2949648603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c003d90 con 0x7f196c106170 2024-04-23T18:35:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f1969ffb640 1 -- 172.21.15.33:0/2949648603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c02fb60 con 0x7f196c106170 2024-04-23T18:35:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.953+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f196c1a5570 con 0x7f196c106170 2024-04-23T18:35:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1969ffb640 1 -- 172.21.15.33:0/2949648603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f195c02fd00 con 0x7f196c106170 2024-04-23T18:35:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 Processor -- start 2024-04-23T18:35:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 -- start start 2024-04-23T18:35:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb58ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb599800 0x55e6fb59bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb4f3970 con 0x55e6fb58ef40 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb4904a0 con 0x55e6fb599800 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1975d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb4ef930 con 0x55e6fb58e620 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1969ffb640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 0x7f19580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1970e26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1970e26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59756/0 (socket says 172.21.15.33:59756) 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1971627640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 0x7f19580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.956+0000 7f1970e26640 1 -- 172.21.15.33:0/1723038280 learned_addr learned my addr 172.21.15.33:0/1723038280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1969ffb640 1 -- 172.21.15.33:0/2949648603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f195c0f1c00 con 0x7f196c106170 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1971627640 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb58ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f196bfff640 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb599800 0x55e6fb59bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 -- 172.21.15.33:0/1723038280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 msgr2=0x55e6fb58ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb58ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 -- 172.21.15.33:0/1723038280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb599800 msgr2=0x55e6fb59bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb599800 0x55e6fb59bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 -- 172.21.15.33:0/1723038280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fb206210 con 0x55e6fb58ef40 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1971627640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 0x7f19580a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1964005d80 tx=0x7f196400ca70 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1970e26640 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f195400b760 tx=0x7f195400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6251332ffa992b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1962ffd640 1 -- 172.21.15.33:0/1723038280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954003af0 con 0x55e6fb58ef40 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1962ffd640 1 -- 172.21.15.33:0/1723038280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954015070 con 0x55e6fb58ef40 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.957+0000 7f1962ffd640 1 -- 172.21.15.33:0/1723038280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954004b20 con 0x55e6fb58ef40 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- 172.21.15.33:0/1723038280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 msgr2=0x55e6fb5992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f195400b760 tx=0x7f195400bb60 comp rx=0 tx=0).stop 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- 172.21.15.33:0/1723038280 shutdown_connections 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb599800 0x55e6fb59bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb58ef40 0x55e6fb5992c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1723038280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb58ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- 172.21.15.33:0/1723038280 >> 172.21.15.33:0/1723038280 conn(0x55e6fb58ab30 msgr2=0x55e6fb58af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- 172.21.15.33:0/1723038280 shutdown_connections 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- 172.21.15.33:0/1723038280 wait complete. 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 -- start start 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.958+0000 7f1975d9e740 1 Processor -- start 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5990d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb5963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb59c430 con 0x55e6fb599800 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb48e0b0 con 0x55e6fb58ef40 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e6fb59c5b0 con 0x55e6fb58e620 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1975d9e740 10 client.0 _ll_register_callbacks cb 0x55e6fb47bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59762/0 (socket says 172.21.15.33:59762) 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 -- 172.21.15.33:0/417497203 learned_addr learned my addr 172.21.15.33:0/417497203 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1970e26640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f196bfff640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb5963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 -- 172.21.15.33:0/417497203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 msgr2=0x55e6fb5990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 -- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 msgr2=0x55e6fb5963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb5963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f1971627640 1 -- 172.21.15.33:0/417497203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fb2d2620 con 0x55e6fb599800 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.959+0000 7f196bfff640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb5963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1971627640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1964014310 tx=0x7f196401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2ff9f4fec7f284 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964025030 con 0x55e6fb599800 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1964012be0 con 0x55e6fb599800 2024-04-23T18:35:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1954015070 con 0x55e6fb599800 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1964031050 con 0x55e6fb599800 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 10 client.16779 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6fb5a8320 con 0x55e6fb599800 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954015070 con 0x55e6fb599800 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f19640047a0 con 0x55e6fb599800 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 10 client.16779 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 20 client.16779 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.960+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb5a97e0 0x55e6fb5abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.961+0000 7f1975d9e740 4 client.16779 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.961+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6fb501390 con 0x55e6fb5a97e0 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.961+0000 7f1970e26640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb5a97e0 0x55e6fb5abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.961+0000 7f1970e26640 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb5a97e0 0x55e6fb5abbf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40077dbc5782d49e server_cookie=e11456df7858f278 in_seq=0 out_seq=0 2024-04-23T18:35:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.961+0000 7f194bfff640 10 client.16779 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f194bfff640 1 -- 172.21.15.33:0/417497203 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6fb501390 con 0x55e6fb5a97e0 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f194bfff640 10 client.16779 handle_command_reply: tid=0 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 client.16779 shutdown 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 msgr2=0x55e6fb596900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1964014310 tx=0x7f196401b000 comp rx=0 tx=0).stop 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 shutdown_connections 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb5a97e0 0x55e6fb5abbf0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb596900 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb5963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f194bfff640 0 client.16779 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 --2- 172.21.15.33:0/417497203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 >> 172.21.15.33:0/417497203 conn(0x55e6fb58ab30 msgr2=0x55e6fb590860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 shutdown_connections 2024-04-23T18:35:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.962+0000 7f1975d9e740 1 -- 172.21.15.33:0/417497203 wait complete. 2024-04-23T18:35:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.963+0000 7f1975d9e740 20 client.16779 trim_cache size 0 max 16384 2024-04-23T18:35:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.977+0000 7f1975d9e740 1 Processor -- start 2024-04-23T18:35:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.977+0000 7f1975d9e740 1 -- start start 2024-04-23T18:35:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58e620 0x55e6fb5a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58ef40 0x55e6fb5b1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1adab0 con 0x55e6fb599800 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1af5f0 con 0x55e6fb58e620 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1975d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1ae180 con 0x55e6fb58ef40 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59770/0 (socket says 172.21.15.33:59770) 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 -- 172.21.15.33:0/69335747 learned_addr learned my addr 172.21.15.33:0/69335747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f196bfff640 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58ef40 0x55e6fb5b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1970e26640 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58e620 0x55e6fb5a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 -- 172.21.15.33:0/69335747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58ef40 msgr2=0x55e6fb5b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58ef40 0x55e6fb5b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 -- 172.21.15.33:0/69335747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58e620 msgr2=0x55e6fb5a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58e620 0x55e6fb5a52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 -- 172.21.15.33:0/69335747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6fb59bf60 con 0x55e6fb599800 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f1971627640 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1964021480 tx=0x7f19640146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406016b147646acb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f19627fc640 1 -- 172.21.15.33:0/69335747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196402e020 con 0x55e6fb599800 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.978+0000 7f19627fc640 1 -- 172.21.15.33:0/69335747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640047a0 con 0x55e6fb599800 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f19627fc640 1 -- 172.21.15.33:0/69335747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964025040 con 0x55e6fb599800 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- 172.21.15.33:0/69335747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 msgr2=0x55e6fb5b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1964021480 tx=0x7f19640146c0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- 172.21.15.33:0/69335747 shutdown_connections 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb5b1a40 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58ef40 0x55e6fb5b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 --2- 172.21.15.33:0/69335747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58e620 0x55e6fb5a52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- 172.21.15.33:0/69335747 >> 172.21.15.33:0/69335747 conn(0x55e6fb58ab30 msgr2=0x55e6fb596380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- 172.21.15.33:0/69335747 shutdown_connections 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- 172.21.15.33:0/69335747 wait complete. 2024-04-23T18:35:21.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 -- start start 2024-04-23T18:35:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.979+0000 7f1975d9e740 1 Processor -- start 2024-04-23T18:35:21.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb59ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1aef20 con 0x55e6fb599800 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1ae850 con 0x55e6fb58ef40 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f196c1abab0 con 0x55e6fb58e620 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1975d9e740 10 client.0 _ll_register_callbacks cb 0x7f196400cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59778/0 (socket says 172.21.15.33:59778) 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 -- 172.21.15.33:0/1446419783 learned_addr learned my addr 172.21.15.33:0/1446419783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1970e26640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f196bfff640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb59ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 -- 172.21.15.33:0/1446419783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 msgr2=0x55e6fb5a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 -- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 msgr2=0x55e6fb59ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb59ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1971627640 1 -- 172.21.15.33:0/1446419783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6fb59f9f0 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f1970e26640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.980+0000 7f196bfff640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb59ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1971627640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1964014400 tx=0x7f1964014430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b0a151e59ef60d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964042060 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19640047a0 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f1964004d60 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f196402e030 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 10 client.16785 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6fb60fc40 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19640047a0 con 0x55e6fb599800 2024-04-23T18:35:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1964021820 con 0x55e6fb599800 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 10 client.16785 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 10 client.16785 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 20 client.16785 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb610250 0x55e6fb612640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 4 client.16785 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.981+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6fb4fb200 con 0x55e6fb610250 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.982+0000 7f1970e26640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb610250 0x55e6fb612640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.982+0000 7f194bfff640 10 client.16785 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:21.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.982+0000 7f1970e26640 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb610250 0x55e6fb612640 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca2b85ecb6201bba server_cookie=2c08e9e14a426e81 in_seq=0 out_seq=0 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.989+0000 7f194bfff640 1 -- 172.21.15.33:0/1446419783 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6fb4fb200 con 0x55e6fb610250 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.989+0000 7f194bfff640 10 client.16785 handle_command_reply: tid=0 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.989+0000 7f1975d9e740 1 client.16785 shutdown 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 msgr2=0x55e6fb59f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1964014400 tx=0x7f1964014430 comp rx=0 tx=0).stop 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 shutdown_connections 2024-04-23T18:35:21.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e6fb610250 0x55e6fb612640 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6fb599800 0x55e6fb59f350 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f194bfff640 0 client.16785 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e6fb58ef40 0x55e6fb59ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 --2- 172.21.15.33:0/1446419783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e6fb58e620 0x55e6fb5a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 >> 172.21.15.33:0/1446419783 conn(0x55e6fb58ab30 msgr2=0x55e6fb594bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 shutdown_connections 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 1 -- 172.21.15.33:0/1446419783 wait complete. 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f1975d9e740 20 client.16785 trim_cache size 0 max 16384 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:35:21.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:35:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:35:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 msgr2=0x7f19580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 0x7f19580a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1964005d80 tx=0x7f196400ca70 comp rx=0 tx=0).stop 2024-04-23T18:35:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 msgr2=0x7f196c198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.990+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f195c006010 tx=0x7f195c0319f0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 shutdown_connections 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f19580a3dc0 0x7f19580a6280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f196c19cfa0 0x7f196c19d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f196c106170 0x7f196c198c70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 --2- 172.21.15.33:0/2949648603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f196c1010f0 0x7f196c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 >> 172.21.15.33:0/2949648603 conn(0x7f196c077fd0 msgr2=0x7f196c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 shutdown_connections 2024-04-23T18:35:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:21.991+0000 7f197308a640 1 -- 172.21.15.33:0/2949648603 wait complete. 2024-04-23T18:35:23.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:22 smithi106 ceph-mon[32716]: pgmap v278: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:23.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:22 smithi122 ceph-mon[32546]: pgmap v278: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:23.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:22 smithi033 ceph-mon[29274]: pgmap v278: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:25.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:24 smithi106 ceph-mon[32716]: pgmap v279: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:25.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:24 smithi122 ceph-mon[32546]: pgmap v279: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:25.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:24 smithi033 ceph-mon[29274]: pgmap v279: 97 pgs: 97 active+clean; 6.3 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:27.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:26 smithi106 ceph-mon[32716]: pgmap v280: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:35:27.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:26 smithi122 ceph-mon[32546]: pgmap v280: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:35:27.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:26 smithi033 ceph-mon[29274]: pgmap v280: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:35:29.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:28 smithi106 ceph-mon[32716]: pgmap v281: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:29.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:28 smithi033 ceph-mon[29274]: pgmap v281: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:29.397 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:28 smithi122 ceph-mon[32546]: pgmap v281: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:31.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:30 smithi106 ceph-mon[32716]: pgmap v282: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:31.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:30 smithi122 ceph-mon[32546]: pgmap v282: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:31.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:30 smithi033 ceph-mon[29274]: pgmap v282: 97 pgs: 97 active+clean; 6.4 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:33.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:32 smithi106 ceph-mon[32716]: pgmap v283: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:251/600: open fd = 3 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:252/600: open fd = 3 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:253/600: open fd = 3 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.141 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:254/600: open fd = 3 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:255/600: open fd = 3 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:256/600: open fd = 3 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:257/600: open fd = 3 2024-04-23T18:35:33.142 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:258/600: open fd = 3 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:259/600: open fd = 3 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:260/600: open fd = 3 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:261/600: open fd = 3 2024-04-23T18:35:33.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:262/600: open fd = 3 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:263/600: open fd = 3 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:264/600: open fd = 3 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.144 INFO:tasks.workunit.client.0.smithi033.stdout:265/600: open fd = 3 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:266/600: open fd = 3 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:267/600: open fd = 3 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:268/600: open fd = 3 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.145 INFO:tasks.workunit.client.0.smithi033.stdout:269/600: open fd = 3 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:270/600: open fd = 3 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:271/600: open fd = 3 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:272/600: open fd = 3 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.146 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:273/600: open fd = 3 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:274/600: open fd = 3 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:275/600: open fd = 3 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:276/600: open fd = 3 2024-04-23T18:35:33.147 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:277/600: open fd = 3 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:278/600: open fd = 3 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:279/600: open fd = 3 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.148 INFO:tasks.workunit.client.0.smithi033.stdout:280/600: open fd = 3 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:281/600: open fd = 3 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:282/600: open fd = 3 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:283/600: open fd = 3 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.149 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:284/600: open fd = 3 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:285/600: open fd = 3 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:286/600: open fd = 3 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:287/600: open fd = 3 2024-04-23T18:35:33.150 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:288/600: open fd = 3 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:289/600: open fd = 3 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:290/600: open fd = 3 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:291/600: open fd = 3 2024-04-23T18:35:33.151 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:292/600: open fd = 3 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:293/600: open fd = 3 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:294/600: open fd = 3 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:295/600: open fd = 3 2024-04-23T18:35:33.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:296/600: open fd = 3 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:297/600: open fd = 3 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:298/600: open fd = 3 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:299/600: open fd = 3 2024-04-23T18:35:33.153 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:300/600: open fd = 3 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:301/600: open fd = 3 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:302/600: open fd = 3 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.154 INFO:tasks.workunit.client.0.smithi033.stdout:303/600: open fd = 3 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:304/600: open fd = 3 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:305/600: open fd = 3 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:306/600: open fd = 3 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.155 INFO:tasks.workunit.client.0.smithi033.stdout:307/600: open fd = 3 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:308/600: open fd = 3 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:309/600: open fd = 3 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:310/600: open fd = 3 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.156 INFO:tasks.workunit.client.0.smithi033.stdout:311/600: open fd = 3 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:312/600: open fd = 3 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.157 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:35:33.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:32 smithi122 ceph-mon[32546]: pgmap v283: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-23T18:35:33.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:32 smithi033 ceph-mon[29274]: pgmap v283: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 20 op/s 2024-04-23T18:35:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 -- 172.21.15.33:0/1541031477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 msgr2=0x7f2978110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 --2- 172.21.15.33:0/1541031477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f2978110d20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2970009900 tx=0x7f297002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 -- 172.21.15.33:0/1541031477 shutdown_connections 2024-04-23T18:35:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 --2- 172.21.15.33:0/1541031477 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f2978110d20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 --2- 172.21.15.33:0/1541031477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 0x7f297810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 --2- 172.21.15.33:0/1541031477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f2978101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 -- 172.21.15.33:0/1541031477 >> 172.21.15.33:0/1541031477 conn(0x7f2978077fd0 msgr2=0x7f29780783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 -- 172.21.15.33:0/1541031477 shutdown_connections 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 -- 172.21.15.33:0/1541031477 wait complete. 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.091+0000 7f297d337640 1 Processor -- start 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 -- start start 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f29781a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 0x7f29781a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29781143b0 con 0x7f297810e930 2024-04-23T18:35:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2978114230 con 0x7f2978100d80 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f297d337640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2978114530 con 0x7f29781016a0 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58186/0 (socket says 172.21.15.33:58186) 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 -- 172.21.15.33:0/4156814555 learned_addr learned my addr 172.21.15.33:0/4156814555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f2976ffd640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f29781a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29767fc640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 0x7f29781a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 -- 172.21.15.33:0/4156814555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 msgr2=0x7f29781a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 0x7f29781a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 -- 172.21.15.33:0/4156814555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 msgr2=0x7f29781a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.092+0000 7f29777fe640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f29781a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f29777fe640 1 -- 172.21.15.33:0/4156814555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297819afd0 con 0x7f297810e930 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f2976ffd640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f29781a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f29777fe640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2970005940 tx=0x7f29700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f295ffff640 1 -- 172.21.15.33:0/4156814555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297003f030 con 0x7f297810e930 2024-04-23T18:35:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f295ffff640 1 -- 172.21.15.33:0/4156814555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2970003cb0 con 0x7f297810e930 2024-04-23T18:35:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297819b260 con 0x7f297810e930 2024-04-23T18:35:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f295ffff640 1 -- 172.21.15.33:0/4156814555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970035070 con 0x7f297810e930 2024-04-23T18:35:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.093+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29781a75b0 con 0x7f297810e930 2024-04-23T18:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.095+0000 7f295ffff640 1 -- 172.21.15.33:0/4156814555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2970042030 con 0x7f297810e930 2024-04-23T18:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 Processor -- start 2024-04-23T18:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 -- start start 2024-04-23T18:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f295ffff640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 0x7f29600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877af40 0x5593d87852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d8787bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593d86df970 con 0x5593d877a620 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593d867c4a0 con 0x5593d877af40 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f298004e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593d86db930 con 0x5593d8785800 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f29767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f2976ffd640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 0x7f29600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f29767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58190/0 (socket says 172.21.15.33:58190) 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f29767fc640 1 -- 172.21.15.33:0/3845917849 learned_addr learned my addr 172.21.15.33:0/3845917849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.096+0000 7f295ffff640 1 -- 172.21.15.33:0/4156814555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f29700f11f0 con 0x7f297810e930 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29777fe640 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d8787bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877af40 0x5593d87852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 -- 172.21.15.33:0/3845917849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 msgr2=0x5593d8787bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d8787bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 -- 172.21.15.33:0/3845917849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877af40 msgr2=0x5593d87852c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877af40 0x5593d87852c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 -- 172.21.15.33:0/3845917849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d83f2210 con 0x5593d877a620 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f2976ffd640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 0x7f29600a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968009520 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f29767fc640 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f296c0134d0 tx=0x7f296c00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67993e315d544b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f295d7fa640 1 -- 172.21.15.33:0/3845917849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c019030 con 0x5593d877a620 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f295d7fa640 1 -- 172.21.15.33:0/3845917849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f296c005bf0 con 0x5593d877a620 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.097+0000 7f295d7fa640 1 -- 172.21.15.33:0/3845917849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c011440 con 0x5593d877a620 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- 172.21.15.33:0/3845917849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 msgr2=0x5593d877aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f296c0134d0 tx=0x7f296c00b5c0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- 172.21.15.33:0/3845917849 shutdown_connections 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d8787bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877af40 0x5593d87852c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 --2- 172.21.15.33:0/3845917849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d877aa00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- 172.21.15.33:0/3845917849 >> 172.21.15.33:0/3845917849 conn(0x5593d8776b30 msgr2=0x5593d8776f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- 172.21.15.33:0/3845917849 shutdown_connections 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- 172.21.15.33:0/3845917849 wait complete. 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 -- start start 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.098+0000 7f298004e740 1 Processor -- start 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 0x5593d87850b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d87828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593d8788430 con 0x5593d877af40 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593d867a0b0 con 0x5593d877a620 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593d87885b0 con 0x5593d8785800 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f298004e740 10 client.0 _ll_register_callbacks cb 0x5593d8667d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58192/0 (socket says 172.21.15.33:58192) 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 -- 172.21.15.33:0/2351911884 learned_addr learned my addr 172.21.15.33:0/2351911884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f2976ffd640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 0x5593d87850b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29777fe640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d87828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 -- 172.21.15.33:0/2351911884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 msgr2=0x5593d87828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d87828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 -- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 msgr2=0x5593d87850b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 0x5593d87850b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 -- 172.21.15.33:0/2351911884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d84be620 con 0x5593d877af40 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f2976ffd640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 0x5593d87850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f29767fc640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f296c004bf0 tx=0x7f296c00bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d4568a1faa3faf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f295aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.099+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c019050 con 0x5593d877af40 2024-04-23T18:35:35.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f296c005430 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593d86ed390 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f296c004e50 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 10 client.16797 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5593d87942a0 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5593d86ed390 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f296c026020 con 0x5593d877af40 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 10 client.16797 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 20 client.16797 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87956a0 0x5593d8797a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 4 client.16797 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593d86e7200 con 0x5593d87956a0 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.100+0000 7f2976ffd640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87956a0 0x5593d8797a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.101+0000 7f295aff5640 10 client.16797 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.101+0000 7f2976ffd640 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87956a0 0x5593d8797a90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f183e42a4e012fb server_cookie=a285f82b88bab2f9 in_seq=0 out_seq=0 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.101+0000 7f295aff5640 1 -- 172.21.15.33:0/2351911884 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593d86e7200 con 0x5593d87956a0 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.101+0000 7f295aff5640 10 client.16797 handle_command_reply: tid=0 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 client.16797 shutdown 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 msgr2=0x5593d87823a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f296c004bf0 tx=0x7f296c00bdc0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 shutdown_connections 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87956a0 0x5593d8797a90 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8785800 0x5593d87828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877af40 0x5593d87823a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 --2- 172.21.15.33:0/2351911884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d877a620 0x5593d87850b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f295aff5640 0 client.16797 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 >> 172.21.15.33:0/2351911884 conn(0x5593d8776b30 msgr2=0x5593d877c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 shutdown_connections 2024-04-23T18:35:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 1 -- 172.21.15.33:0/2351911884 wait complete. 2024-04-23T18:35:35.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.102+0000 7f298004e740 20 client.16797 trim_cache size 0 max 16384 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 Processor -- start 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 -- start start 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d877d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d8784be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29781afce0 con 0x5593d877a620 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b1820 con 0x5593d8785800 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f298004e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b03b0 con 0x5593d877af40 2024-04-23T18:35:35.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58208/0 (socket says 172.21.15.33:58208) 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 -- 172.21.15.33:0/3473193274 learned_addr learned my addr 172.21.15.33:0/3473193274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f29767fc640 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d877d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 -- 172.21.15.33:0/3473193274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 msgr2=0x5593d877d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d877d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 -- 172.21.15.33:0/3473193274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 msgr2=0x5593d8784be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d8784be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f2976ffd640 1 -- 172.21.15.33:0/3473193274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d87808f0 con 0x5593d877a620 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.117+0000 7f29767fc640 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d877d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x5593d8798150 tx=0x7f2968016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6cf1492cf6b988 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f295cff9640 1 -- 172.21.15.33:0/3473193274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968027d50 con 0x5593d877a620 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f295cff9640 1 -- 172.21.15.33:0/3473193274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2968003e20 con 0x5593d877a620 2024-04-23T18:35:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f295cff9640 1 -- 172.21.15.33:0/3473193274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680274b0 con 0x5593d877a620 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 -- 172.21.15.33:0/3473193274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 msgr2=0x5593d87912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x5593d8798150 tx=0x7f2968016000 comp rx=0 tx=0).stop 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 -- 172.21.15.33:0/3473193274 shutdown_connections 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d8784be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d877d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 --2- 172.21.15.33:0/3473193274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d87912f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 -- 172.21.15.33:0/3473193274 >> 172.21.15.33:0/3473193274 conn(0x5593d8776b30 msgr2=0x5593d877d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 -- 172.21.15.33:0/3473193274 shutdown_connections 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.118+0000 7f298004e740 1 -- 172.21.15.33:0/3473193274 wait complete. 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 -- start start 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 Processor -- start 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d878dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d877e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b1150 con 0x5593d877a620 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29781b0a80 con 0x5593d8785800 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29781adce0 con 0x5593d877af40 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f298004e740 10 client.0 _ll_register_callbacks cb 0x7f296c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58214/0 (socket says 172.21.15.33:58214) 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 -- 172.21.15.33:0/3820649268 learned_addr learned my addr 172.21.15.33:0/3820649268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f29767fc640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d878dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f29777fe640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d877e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 -- 172.21.15.33:0/3820649268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 msgr2=0x5593d878dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d878dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 -- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 msgr2=0x5593d877e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d877e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.119+0000 7f2976ffd640 1 -- 172.21.15.33:0/3820649268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d877eff0 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f2976ffd640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f29680128b0 tx=0x7f2968020880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1063469509d0f888 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968003b80 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2968003e20 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2968003fd0 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f29680041d0 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 10 client.16803 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5593d87fb4f0 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2968003e20 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f296800f040 con 0x5593d877a620 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 10 client.16803 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 10 client.16803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 20 client.16803 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87fbbd0 0x5593d87fdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 4 client.16803 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.120+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593d8782c70 con 0x5593d87fbbd0 2024-04-23T18:35:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.121+0000 7f29767fc640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87fbbd0 0x5593d87fdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.121+0000 7f29767fc640 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87fbbd0 0x5593d87fdff0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598a2f639834480 server_cookie=bb117b63053420b1 in_seq=0 out_seq=0 2024-04-23T18:35:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.121+0000 7f295aff5640 10 client.16803 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:35.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:34 smithi106 ceph-mon[32716]: pgmap v284: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f295aff5640 1 -- 172.21.15.33:0/3820649268 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593d8782c70 con 0x5593d87fbbd0 2024-04-23T18:35:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f295aff5640 10 client.16803 handle_command_reply: tid=0 2024-04-23T18:35:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 client.16803 shutdown 2024-04-23T18:35:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 msgr2=0x5593d878d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f29680128b0 tx=0x7f2968020880 comp rx=0 tx=0).stop 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 shutdown_connections 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593d87fbbd0 0x5593d87fdff0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593d8785800 0x5593d877e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f295aff5640 0 client.16803 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d877af40 0x5593d878dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 --2- 172.21.15.33:0/3820649268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593d877a620 0x5593d878d800 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 >> 172.21.15.33:0/3820649268 conn(0x5593d8776b30 msgr2=0x5593d877c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 shutdown_connections 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 1 -- 172.21.15.33:0/3820649268 wait complete. 2024-04-23T18:35:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.129+0000 7f298004e740 20 client.16803 trim_cache size 0 max 16384 2024-04-23T18:35:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:35:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:35:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:35:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:35:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 msgr2=0x7f29600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 0x7f29600a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2968005fd0 tx=0x7f2968009520 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 msgr2=0x7f297819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f2970005940 tx=0x7f29700042f0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 shutdown_connections 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29600a3dc0 0x7f29600a6280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f297810e930 0x7f297819a710 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29781016a0 0x7f29781a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 --2- 172.21.15.33:0/4156814555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2978100d80 0x7f29781a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 >> 172.21.15.33:0/4156814555 conn(0x7f2978077fd0 msgr2=0x7f297810fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 shutdown_connections 2024-04-23T18:35:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:35.130+0000 7f297d337640 1 -- 172.21.15.33:0/4156814555 wait complete. 2024-04-23T18:35:35.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:34 smithi122 ceph-mon[32546]: pgmap v284: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:35.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:34 smithi033 ceph-mon[29274]: pgmap v284: 97 pgs: 97 active+clean; 6.5 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:37.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:36 smithi106 ceph-mon[32716]: pgmap v285: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 22 op/s 2024-04-23T18:35:37.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:36 smithi122 ceph-mon[32546]: pgmap v285: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 22 op/s 2024-04-23T18:35:37.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:36 smithi033 ceph-mon[29274]: pgmap v285: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 KiB/s wr, 22 op/s 2024-04-23T18:35:39.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:38 smithi106 ceph-mon[32716]: pgmap v286: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:39.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:38 smithi122 ceph-mon[32546]: pgmap v286: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:39.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:38 smithi033 ceph-mon[29274]: pgmap v286: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:41.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:40 smithi106 ceph-mon[32716]: pgmap v287: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:41.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:40 smithi122 ceph-mon[32546]: pgmap v287: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:41.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:40 smithi033 ceph-mon[29274]: pgmap v287: 97 pgs: 97 active+clean; 6.6 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:43.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:42 smithi106 ceph-mon[32716]: pgmap v288: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 21 op/s 2024-04-23T18:35:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:43.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:35:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:42 smithi122 ceph-mon[32546]: pgmap v288: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 21 op/s 2024-04-23T18:35:43.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:43.198 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:35:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:42 smithi033 ceph-mon[29274]: pgmap v288: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 KiB/s wr, 21 op/s 2024-04-23T18:35:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:35:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:35:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:35:43.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:35:45.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:44 smithi106 ceph-mon[32716]: pgmap v289: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:45.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:44 smithi122 ceph-mon[32546]: pgmap v289: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:45.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:44 smithi033 ceph-mon[29274]: pgmap v289: 97 pgs: 97 active+clean; 6.7 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:35:47.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:46 smithi106 ceph-mon[32716]: pgmap v290: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-04-23T18:35:47.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:46 smithi122 ceph-mon[32546]: pgmap v290: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-04-23T18:35:47.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:46 smithi033 ceph-mon[29274]: pgmap v290: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 21 op/s 2024-04-23T18:35:48.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 -- 172.21.15.33:0/780741528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c101aa0 msgr2=0x7f0a9c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 --2- 172.21.15.33:0/780741528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c101aa0 0x7f0a9c105a90 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009900 tx=0x7f0a9002a410 comp rx=0 tx=0).stop 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 -- 172.21.15.33:0/780741528 shutdown_connections 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 --2- 172.21.15.33:0/780741528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 0x7f0a9c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 --2- 172.21.15.33:0/780741528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c101aa0 0x7f0a9c105a90 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 --2- 172.21.15.33:0/780741528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 -- 172.21.15.33:0/780741528 >> 172.21.15.33:0/780741528 conn(0x7f0a9c077fd0 msgr2=0x7f0a9c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 -- 172.21.15.33:0/780741528 shutdown_connections 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.230+0000 7f0aa1522640 1 -- 172.21.15.33:0/780741528 wait complete. 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 Processor -- start 2024-04-23T18:35:48.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 -- start start 2024-04-23T18:35:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 0x7f0a9c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1110b0 con 0x7f0a9c19cf50 2024-04-23T18:35:48.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c110f30 con 0x7f0a9c106170 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0aa1522640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c111230 con 0x7f0a9c1010f0 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0a9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37470/0 (socket says 172.21.15.33:37470) 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0a9b7fe640 1 -- 172.21.15.33:0/3325736190 learned_addr learned my addr 172.21.15.33:0/3325736190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 0x7f0a9c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.231+0000 7f0a9affd640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 -- 172.21.15.33:0/3325736190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 msgr2=0x7f0a9c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 -- 172.21.15.33:0/3325736190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 msgr2=0x7f0a9c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 0x7f0a9c198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 -- 172.21.15.33:0/3325736190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c19da40 con 0x7f0a9c19cf50 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9affd640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a9b7fe640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cce0 tx=0x7f0a9400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a83fff640 1 -- 172.21.15.33:0/3325736190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9400f450 con 0x7f0a9c19cf50 2024-04-23T18:35:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a83fff640 1 -- 172.21.15.33:0/3325736190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9400faa0 con 0x7f0a9c19cf50 2024-04-23T18:35:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c19dd30 con 0x7f0a9c19cf50 2024-04-23T18:35:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0a83fff640 1 -- 172.21.15.33:0/3325736190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94018b70 con 0x7f0a9c19cf50 2024-04-23T18:35:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.232+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c1a54a0 con 0x7f0a9c19cf50 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.234+0000 7f0a83fff640 1 -- 172.21.15.33:0/3325736190 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0a94018d10 con 0x7f0a9c19cf50 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.234+0000 7f0aa4230740 1 Processor -- start 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 -- start start 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9f40 0x55a7df0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a7df00e970 con 0x55a7df0a9620 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7defab4a0 con 0x55a7df0a9f40 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0aa4230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a7df00a930 con 0x55a7df0b4800 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a83fff640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 0x7f0a840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37474/0 (socket says 172.21.15.33:37474) 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 -- 172.21.15.33:0/144584887 learned_addr learned my addr 172.21.15.33:0/144584887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9affd640 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9f40 0x55a7df0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9b7fe640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 0x7f0a840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a83fff640 1 -- 172.21.15.33:0/3325736190 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0a940d0ca0 con 0x7f0a9c19cf50 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 -- 172.21.15.33:0/144584887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 msgr2=0x55a7df0b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 -- 172.21.15.33:0/144584887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9f40 msgr2=0x55a7df0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9f40 0x55a7df0b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.235+0000 7f0a9a7fc640 1 -- 172.21.15.33:0/144584887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7ded21210 con 0x55a7df0a9620 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.236+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0a90005940 tx=0x7f0a900058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8081a3dd7c55c43d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.236+0000 7f0a9b7fe640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 0x7f0a840a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a940ce9e0 tx=0x7f0a940920c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.236+0000 7f0a817fa640 1 -- 172.21.15.33:0/144584887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90043020 con 0x55a7df0a9620 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.236+0000 7f0a817fa640 1 -- 172.21.15.33:0/144584887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a900044d0 con 0x55a7df0a9620 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.236+0000 7f0a817fa640 1 -- 172.21.15.33:0/144584887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90035050 con 0x55a7df0a9620 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- 172.21.15.33:0/144584887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 msgr2=0x55a7df0a9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0a90005940 tx=0x7f0a900058b0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- 172.21.15.33:0/144584887 shutdown_connections 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9f40 0x55a7df0b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- 172.21.15.33:0/144584887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9620 0x55a7df0a9a00 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- 172.21.15.33:0/144584887 >> 172.21.15.33:0/144584887 conn(0x55a7df0a5b30 msgr2=0x55a7df0a5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- 172.21.15.33:0/144584887 shutdown_connections 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- 172.21.15.33:0/144584887 wait complete. 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- start start 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 Processor -- start 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a7df0b73d0 con 0x55a7df0a9f40 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a7defa90b0 con 0x55a7df0a9620 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a7df0b7550 con 0x55a7df0b4800 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0aa4230740 10 client.0 _ll_register_callbacks cb 0x55a7def96d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.237+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37484/0 (socket says 172.21.15.33:37484) 2024-04-23T18:35:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 -- 172.21.15.33:0/651328104 learned_addr learned my addr 172.21.15.33:0/651328104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 -- 172.21.15.33:0/651328104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 msgr2=0x55a7df0b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 -- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 msgr2=0x55a7df0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 -- 172.21.15.33:0/651328104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7deded620 con 0x55a7df0a9f40 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a9affd640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0113e0 tx=0x7f0a8c0118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1589215765b3c189 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c012980 con 0x55a7df0a9f40 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0a8c012b20 con 0x55a7df0a9f40 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a900044d0 con 0x55a7df0a9f40 2024-04-23T18:35:48.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c012d20 con 0x55a7df0a9f40 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0aa4230740 10 client.16815 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.238+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7df0c3360 con 0x55a7df0a9f40 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a900044d0 con 0x55a7df0a9f40 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0a8c00f040 con 0x55a7df0a9f40 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 10 client.16815 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 20 client.16815 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df0c48e0 0x55a7df0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 4 client.16815 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7df01c390 con 0x55a7df0c48e0 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df0c48e0 0x55a7df0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0a7eff5640 10 client.16815 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.239+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df0c48e0 0x55a7df0c6cd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe93e10e42ed50f6 server_cookie=34b55a7709b7c4c3 in_seq=0 out_seq=0 2024-04-23T18:35:48.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0a7eff5640 1 -- 172.21.15.33:0/651328104 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7df01c390 con 0x55a7df0c48e0 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0a7eff5640 10 client.16815 handle_command_reply: tid=0 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 client.16815 shutdown 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 msgr2=0x55a7df0b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0113e0 tx=0x7f0a8c0118a0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 shutdown_connections 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df0c48e0 0x55a7df0c6cd0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b1450 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0a7eff5640 0 client.16815 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 --2- 172.21.15.33:0/651328104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.240+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 >> 172.21.15.33:0/651328104 conn(0x55a7df0a5b30 msgr2=0x55a7df0ab8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.241+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 shutdown_connections 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.241+0000 7f0aa4230740 1 -- 172.21.15.33:0/651328104 wait complete. 2024-04-23T18:35:48.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.241+0000 7f0aa4230740 20 client.16815 trim_cache size 0 max 16384 2024-04-23T18:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.256+0000 7f0aa4230740 1 Processor -- start 2024-04-23T18:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.256+0000 7f0aa4230740 1 -- start start 2024-04-23T18:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.256+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0a5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0aa4230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ad9f0 con 0x55a7df0a9f40 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0aa4230740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1af530 con 0x55a7df0a9620 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0aa4230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ae0c0 con 0x55a7df0b4800 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37492/0 (socket says 172.21.15.33:37492) 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 -- 172.21.15.33:0/786815399 learned_addr learned my addr 172.21.15.33:0/786815399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 -- 172.21.15.33:0/786815399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 msgr2=0x55a7df0bc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0bc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 -- 172.21.15.33:0/786815399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 msgr2=0x55a7df0a5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 -- 172.21.15.33:0/786815399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7df0a9d70 con 0x55a7df0a9f40 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a9affd640 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006010 tx=0x7f0a8c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77457ce088febcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a80ff9640 1 -- 172.21.15.33:0/786815399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011da0 con 0x55a7df0a9f40 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a80ff9640 1 -- 172.21.15.33:0/786815399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a8c019870 con 0x55a7df0a9f40 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.257+0000 7f0a80ff9640 1 -- 172.21.15.33:0/786815399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0053d0 con 0x55a7df0a9f40 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 -- 172.21.15.33:0/786815399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 msgr2=0x55a7df0b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006010 tx=0x7f0a8c0096a0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 -- 172.21.15.33:0/786815399 shutdown_connections 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0bc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0b3c90 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 --2- 172.21.15.33:0/786815399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0a5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 -- 172.21.15.33:0/786815399 >> 172.21.15.33:0/786815399 conn(0x55a7df0a5b30 msgr2=0x55a7df0bf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 -- 172.21.15.33:0/786815399 shutdown_connections 2024-04-23T18:35:48.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.258+0000 7f0aa4230740 1 -- 172.21.15.33:0/786815399 wait complete. 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 -- start start 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 Processor -- start 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0babb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1aee60 con 0x55a7df0a9f40 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ae790 con 0x55a7df0a9620 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c1ab9f0 con 0x55a7df0b4800 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0aa4230740 10 client.0 _ll_register_callbacks cb 0x7f0a8c009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37502/0 (socket says 172.21.15.33:37502) 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:38928/0 (socket says 172.21.15.33:38928) 2024-04-23T18:35:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9affd640 1 -- 172.21.15.33:0/2676923070 learned_addr learned my addr 172.21.15.33:0/2676923070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.259+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0babb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 -- 172.21.15.33:0/2676923070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 msgr2=0x55a7df0babb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0babb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 -- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 msgr2=0x55a7df0ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 -- 172.21.15.33:0/2676923070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7df129e40 con 0x55a7df0a9f40 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a9affd640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0113e0 tx=0x7f0a8c0129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61275c1139b9b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c039060 con 0x55a7df0a9f40 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f0a8c019c30 con 0x55a7df0a9f40 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8c019870 con 0x55a7df0a9f40 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0a8c011da0 con 0x55a7df0a9f40 2024-04-23T18:35:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0aa4230740 10 client.16821 fetch_fsmap learned FSMap version 34 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7df12b080 con 0x55a7df0a9f40 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.260+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a8c019870 con 0x55a7df0a9f40 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f0a8c004a70 con 0x55a7df0a9f40 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 10 client.16821 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 10 client.16821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 20 client.16821 populate_metadata read hostname 'smithi033' 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df12b690 0x55a7df12dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 4 client.16821 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a7df016200 con 0x55a7df12b690 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df12b690 0x55a7df12dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0a9a7fc640 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df12b690 0x55a7df12dab0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c5c2f2f5ed1789 server_cookie=9a1df601e18e8c35 in_seq=0 out_seq=0 2024-04-23T18:35:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.261+0000 7f0a7eff5640 10 client.16821 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0a7eff5640 1 -- 172.21.15.33:0/2676923070 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a7df016200 con 0x55a7df12b690 2024-04-23T18:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0a7eff5640 10 client.16821 handle_command_reply: tid=0 2024-04-23T18:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 client.16821 shutdown 2024-04-23T18:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 msgr2=0x55a7df0ba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0113e0 tx=0x7f0a8c0129b0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 shutdown_connections 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a7df12b690 0x55a7df12dab0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a7df0b4800 0x55a7df0babb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0a7eff5640 0 client.16821 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a7df0a9f40 0x55a7df0ba670 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 --2- 172.21.15.33:0/2676923070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a7df0a9620 0x55a7df0ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 >> 172.21.15.33:0/2676923070 conn(0x55a7df0a5b30 msgr2=0x55a7df0bfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 shutdown_connections 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 1 -- 172.21.15.33:0/2676923070 wait complete. 2024-04-23T18:35:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.269+0000 7f0aa4230740 20 client.16821 trim_cache size 0 max 16384 2024-04-23T18:35:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:35:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:35:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:35:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 msgr2=0x7f0a840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 0x7f0a840a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0a940ce9e0 tx=0x7f0a940920c0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 msgr2=0x7f0a9c19d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400cce0 tx=0x7f0a9400a620 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 shutdown_connections 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0a840a3dc0 0x7f0a840a6280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a9c19cf50 0x7f0a9c19d400 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a9c106170 0x7f0a9c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 --2- 172.21.15.33:0/3325736190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a9c1010f0 0x7f0a9c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 >> 172.21.15.33:0/3325736190 conn(0x7f0a9c077fd0 msgr2=0x7f0a9c0699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 shutdown_connections 2024-04-23T18:35:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:35:48.270+0000 7f0aa1522640 1 -- 172.21.15.33:0/3325736190 wait complete. 2024-04-23T18:35:49.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:48 smithi106 ceph-mon[32716]: pgmap v291: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:49.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:48 smithi122 ceph-mon[32546]: pgmap v291: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:49.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:48 smithi033 ceph-mon[29274]: pgmap v291: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:51.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:50 smithi106 ceph-mon[32716]: pgmap v292: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:51.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:50 smithi122 ceph-mon[32546]: pgmap v292: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:51.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:50 smithi033 ceph-mon[29274]: pgmap v292: 97 pgs: 97 active+clean; 6.8 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-23T18:35:53.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:52 smithi106 ceph-mon[32716]: pgmap v293: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-23T18:35:53.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:52 smithi122 ceph-mon[32546]: pgmap v293: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-23T18:35:53.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:52 smithi033 ceph-mon[29274]: pgmap v293: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-23T18:35:55.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:54 smithi106 ceph-mon[32716]: pgmap v294: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:55.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:54 smithi122 ceph-mon[32546]: pgmap v294: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:55.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:54 smithi033 ceph-mon[29274]: pgmap v294: 97 pgs: 97 active+clean; 6.9 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:35:57.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:56 smithi106 ceph-mon[32716]: pgmap v295: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:35:57.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:56 smithi122 ceph-mon[32546]: pgmap v295: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:35:57.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:56 smithi033 ceph-mon[29274]: pgmap v295: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:35:59.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:35:58 smithi106 ceph-mon[32716]: pgmap v296: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:59.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:35:58 smithi122 ceph-mon[32546]: pgmap v296: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:35:59.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:35:58 smithi033 ceph-mon[29274]: pgmap v296: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:36:01.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:00 smithi106 ceph-mon[32716]: pgmap v297: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:36:01.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:00 smithi122 ceph-mon[32546]: pgmap v297: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:36:01.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:00 smithi033 ceph-mon[29274]: pgmap v297: 97 pgs: 97 active+clean; 7.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 14 op/s 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.370+0000 7f3db047c640 1 -- 172.21.15.33:0/4226548627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da81011a0 msgr2=0x7f3da8101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.370+0000 7f3db047c640 1 --2- 172.21.15.33:0/4226548627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da81011a0 0x7f3da8101620 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009900 tx=0x7f3d9802a400 comp rx=0 tx=0).stop 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 -- 172.21.15.33:0/4226548627 shutdown_connections 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 --2- 172.21.15.33:0/4226548627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8101b60 0x7f3da810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 --2- 172.21.15.33:0/4226548627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da81011a0 0x7f3da8101620 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 --2- 172.21.15.33:0/4226548627 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da8106fb0 0x7f3da8107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 -- 172.21.15.33:0/4226548627 >> 172.21.15.33:0/4226548627 conn(0x7f3da8076c50 msgr2=0x7f3da8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 -- 172.21.15.33:0/4226548627 shutdown_connections 2024-04-23T18:36:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 -- 172.21.15.33:0/4226548627 wait complete. 2024-04-23T18:36:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 Processor -- start 2024-04-23T18:36:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 -- start start 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 0x7f3da81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.371+0000 7f3db047c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3db047c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8106fb0 0x7f3da81a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3db047c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8111090 con 0x7f3da8101b60 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3db047c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8110f10 con 0x7f3da81011a0 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3db047c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3da8111210 con 0x7f3da8106fb0 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39528/0 (socket says 172.21.15.33:39528) 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 -- 172.21.15.33:0/1692374017 learned_addr learned my addr 172.21.15.33:0/1692374017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3daea19640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8106fb0 0x7f3da81a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dae218640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 0x7f3da81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 -- 172.21.15.33:0/1692374017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8106fb0 msgr2=0x7f3da81a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8106fb0 0x7f3da81a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 -- 172.21.15.33:0/1692374017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 msgr2=0x7f3da81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 0x7f3da81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 -- 172.21.15.33:0/1692374017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da81a4990 con 0x7f3da8101b60 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dae218640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 0x7f3da81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3dada17640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005940 tx=0x7f3d980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3d9f7fe640 1 -- 172.21.15.33:0/1692374017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9803f030 con 0x7f3da8101b60 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3d9f7fe640 1 -- 172.21.15.33:0/1692374017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d98003cb0 con 0x7f3da8101b60 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.372+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da81a4b60 con 0x7f3da8101b60 2024-04-23T18:36:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.373+0000 7f3d9f7fe640 1 -- 172.21.15.33:0/1692374017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98035070 con 0x7f3da8101b60 2024-04-23T18:36:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.373+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da81a5020 con 0x7f3da8101b60 2024-04-23T18:36:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.375+0000 7f3d9f7fe640 1 -- 172.21.15.33:0/1692374017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d98042030 con 0x7f3da8101b60 2024-04-23T18:36:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.375+0000 7f3db3192740 1 Processor -- start 2024-04-23T18:36:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 -- start start 2024-04-23T18:36:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2aba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2ac9c10 0x559cf2acc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559cf2a1f970 con 0x559cf2abae40 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559cf29bc4a0 con 0x559cf2aba520 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3db3192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cf2a1b930 con 0x559cf2ac9c10 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3d9f7fe640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 0x7f3d940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3dae218640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3daea19640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 0x7f3d940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3dae218640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39536/0 (socket says 172.21.15.33:39536) 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3dae218640 1 -- 172.21.15.33:0/3022778183 learned_addr learned my addr 172.21.15.33:0/3022778183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3d9f7fe640 1 -- 172.21.15.33:0/1692374017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f3d980f0fd0 con 0x7f3da8101b60 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3daea19640 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2aba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.376+0000 7f3dada17640 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2ac9c10 0x559cf2acc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 -- 172.21.15.33:0/3022778183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2ac9c10 msgr2=0x559cf2acc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2ac9c10 0x559cf2acc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 -- 172.21.15.33:0/3022778183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 msgr2=0x559cf2aba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2aba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 -- 172.21.15.33:0/3022778183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf2732210 con 0x559cf2abae40 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3daea19640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 0x7f3d940a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009fd0 tx=0x7f3da0009bc0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3dae218640 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3da4008be0 tx=0x7f3da400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2443830f0b43e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3d9cbfe640 1 -- 172.21.15.33:0/3022778183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4014070 con 0x559cf2abae40 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3d9cbfe640 1 -- 172.21.15.33:0/3022778183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da40047a0 con 0x559cf2abae40 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.377+0000 7f3d9cbfe640 1 -- 172.21.15.33:0/3022778183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4010690 con 0x559cf2abae40 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- 172.21.15.33:0/3022778183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 msgr2=0x559cf2ac96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3da4008be0 tx=0x7f3da400e720 comp rx=0 tx=0).stop 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- 172.21.15.33:0/3022778183 shutdown_connections 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2ac9c10 0x559cf2acc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2abae40 0x559cf2ac96d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 --2- 172.21.15.33:0/3022778183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2aba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- 172.21.15.33:0/3022778183 >> 172.21.15.33:0/3022778183 conn(0x559cf2ab6b30 msgr2=0x559cf2ab6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- 172.21.15.33:0/3022778183 shutdown_connections 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- 172.21.15.33:0/3022778183 wait complete. 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 -- start start 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.378+0000 7f3db3192740 1 Processor -- start 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2ac9c10 0x559cf2ac6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559cf2acc840 con 0x559cf2aba520 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559cf29ba0b0 con 0x559cf2ac9c10 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cf2acc9c0 con 0x559cf2abae40 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3db3192740 10 client.0 _ll_register_callbacks cb 0x559cf29a7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:01.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dada17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39544/0 (socket says 172.21.15.33:39544) 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 -- 172.21.15.33:0/1337916993 learned_addr learned my addr 172.21.15.33:0/1337916993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3daea19640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2ac9c10 0x559cf2ac6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 -- 172.21.15.33:0/1337916993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 msgr2=0x559cf2ac6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 -- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2ac9c10 msgr2=0x559cf2ac6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2ac9c10 0x559cf2ac6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 -- 172.21.15.33:0/1337916993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf27fe620 con 0x559cf2aba520 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3dae218640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005110 tx=0x7f3da4005140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e5e17ebed9a587 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:36:01.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.379+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da4014030 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f3da40054f0 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da40047a0 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3da4004050 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 10 client.16833 fetch_fsmap learned FSMap version 34 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559cf2ad86b0 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da40047a0 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3da4003be0 con 0x559cf2aba520 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 10 client.16833 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 20 client.16833 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2ad9ab0 0x559cf2adbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 4 client.16833 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cf2a2d390 con 0x559cf2ad9ab0 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.380+0000 7f3dada17640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2ad9ab0 0x559cf2adbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.381+0000 7f3d91ffb640 10 client.16833 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.381+0000 7f3dada17640 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2ad9ab0 0x559cf2adbea0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4443fd7e86604eb0 server_cookie=cd3e2463a098d247 in_seq=0 out_seq=0 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3d91ffb640 1 -- 172.21.15.33:0/1337916993 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cf2a2d390 con 0x559cf2ad9ab0 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3d91ffb640 10 client.16833 handle_command_reply: tid=0 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 client.16833 shutdown 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 msgr2=0x559cf2ac9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005110 tx=0x7f3da4005140 comp rx=0 tx=0).stop 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 shutdown_connections 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2ad9ab0 0x559cf2adbea0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2ac9c10 0x559cf2ac6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3d91ffb640 0 client.16833 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 --2- 172.21.15.33:0/1337916993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2aba520 0x559cf2ac9450 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 >> 172.21.15.33:0/1337916993 conn(0x559cf2ab6b30 msgr2=0x559cf2abc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 shutdown_connections 2024-04-23T18:36:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 1 -- 172.21.15.33:0/1337916993 wait complete. 2024-04-23T18:36:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.382+0000 7f3db3192740 20 client.16833 trim_cache size 0 max 16384 2024-04-23T18:36:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 Processor -- start 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 -- start start 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81ab8b0 con 0x559cf2ac9c10 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81ad3f0 con 0x559cf2aba520 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3db3192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81abf80 con 0x559cf2abae40 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3daea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3daea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39550/0 (socket says 172.21.15.33:39550) 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3daea19640 1 -- 172.21.15.33:0/2837754603 learned_addr learned my addr 172.21.15.33:0/2837754603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.397+0000 7f3dae218640 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3dada17640 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 -- 172.21.15.33:0/2837754603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 msgr2=0x559cf2ac1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 -- 172.21.15.33:0/2837754603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 msgr2=0x559cf2ad5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 -- 172.21.15.33:0/2837754603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf2ab6960 con 0x559cf2ac9c10 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3daea19640 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012ee0 tx=0x7f3da0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b1d1e0dbd09818 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3d93fff640 1 -- 172.21.15.33:0/2837754603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da002d020 con 0x559cf2ac9c10 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3d93fff640 1 -- 172.21.15.33:0/2837754603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da0003e20 con 0x559cf2ac9c10 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.398+0000 7f3d93fff640 1 -- 172.21.15.33:0/2837754603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0004020 con 0x559cf2ac9c10 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- 172.21.15.33:0/2837754603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 msgr2=0x559cf2ac8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012ee0 tx=0x7f3da0004ab0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- 172.21.15.33:0/2837754603 shutdown_connections 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac8f80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ac1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- 172.21.15.33:0/2837754603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- 172.21.15.33:0/2837754603 >> 172.21.15.33:0/2837754603 conn(0x559cf2ab6b30 msgr2=0x559cf2ac1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- 172.21.15.33:0/2837754603 shutdown_connections 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- 172.21.15.33:0/2837754603 wait complete. 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- start start 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 Processor -- start 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ad2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81acd20 con 0x559cf2ac9c10 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81ac650 con 0x559cf2aba520 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3da81a98b0 con 0x559cf2abae40 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3db3192740 10 client.0 _ll_register_callbacks cb 0x7f3da4008f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:01.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3daea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3daea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39554/0 (socket says 172.21.15.33:39554) 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.399+0000 7f3daea19640 1 -- 172.21.15.33:0/3910602837 learned_addr learned my addr 172.21.15.33:0/3910602837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3dada17640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ad2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3dae218640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 -- 172.21.15.33:0/3910602837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 msgr2=0x559cf2ad2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ad2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 -- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 msgr2=0x559cf2ad1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 -- 172.21.15.33:0/3910602837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf2ac3630 con 0x559cf2ac9c10 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3dae218640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3daea19640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3da001fcb0 tx=0x7f3da0012510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fd1c7060b8223a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:36:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da002d040 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da0003e20 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f3da0003b80 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3da0040050 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3db3192740 10 client.16839 fetch_fsmap learned FSMap version 34 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.400+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559cf2b3f9f0 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da0003e20 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f3da000f040 con 0x559cf2ac9c10 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 10 client.16839 fetch_fsmap finished waiting for FSMap version 34 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 20 client.16839 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2b40050 0x559cf2b42440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 4 client.16839 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cf2a27200 con 0x559cf2b40050 2024-04-23T18:36:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3dae218640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2b40050 0x559cf2b42440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3d91ffb640 10 client.16839 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:01.401+0000 7f3dae218640 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2b40050 0x559cf2b42440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e3b172ef9fdab7 server_cookie=8bebb1df0e28e287 in_seq=0 out_seq=0 2024-04-23T18:36:03.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:02 smithi106 ceph-mon[32716]: pgmap v298: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-04-23T18:36:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3d91ffb640 1 -- 172.21.15.33:0/3910602837 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559cf2a27200 con 0x559cf2b40050 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3d91ffb640 10 client.16839 handle_command_reply: tid=0 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 client.16839 shutdown 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 msgr2=0x559cf2ac2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3da001fcb0 tx=0x7f3da0012510 comp rx=0 tx=0).stop 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 shutdown_connections 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559cf2b40050 0x559cf2b42440 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559cf2ac9c10 0x559cf2ac2e00 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3d91ffb640 0 client.16839 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cf2abae40 0x559cf2ad2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 --2- 172.21.15.33:0/3910602837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559cf2aba520 0x559cf2ad1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 >> 172.21.15.33:0/3910602837 conn(0x559cf2ab6b30 msgr2=0x559cf2ac1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 shutdown_connections 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 1 -- 172.21.15.33:0/3910602837 wait complete. 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.173+0000 7f3db3192740 20 client.16839 trim_cache size 0 max 16384 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:36:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 msgr2=0x7f3d940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 0x7f3d940a6300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009fd0 tx=0x7f3da0009bc0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 msgr2=0x7f3da8199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005940 tx=0x7f3d980042f0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 shutdown_connections 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d940a3e40 0x7f3d940a6300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3da8106fb0 0x7f3da81a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3da8101b60 0x7f3da8199910 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 --2- 172.21.15.33:0/1692374017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3da81011a0 0x7f3da81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 >> 172.21.15.33:0/1692374017 conn(0x7f3da8076c50 msgr2=0x7f3da80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 shutdown_connections 2024-04-23T18:36:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:03.174+0000 7f3db047c640 1 -- 172.21.15.33:0/1692374017 wait complete. 2024-04-23T18:36:03.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:02 smithi122 ceph-mon[32546]: pgmap v298: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-04-23T18:36:03.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:02 smithi033 ceph-mon[29274]: pgmap v298: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 79 KiB/s wr, 38 op/s 2024-04-23T18:36:04.335 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:36:04.335 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: in thread 7ff7c8531700 thread_name:md_log_replay 2024-04-23T18:36:04.335 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:36:04.335 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7d98f2d20] 2024-04-23T18:36:04.335 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55c126a9b38f] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c126a9eb23] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 4: (MDLog::_replay_thread()+0x617) [0x55c126af9647] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c126b02d31] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff7d98e81ca] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 7: clone() 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:03.961+0000 7ff7c8531700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: in thread 7ff7c8531700 thread_name:md_log_replay 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7d98f2d20] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55c126a9b38f] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c126a9eb23] 2024-04-23T18:36:04.336 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 4: (MDLog::_replay_thread()+0x617) [0x55c126af9647] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c126b02d31] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff7d98e81ca] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 7: clone() 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 0> 2024-04-23T18:36:03.961+0000 7ff7c8531700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: in thread 7ff7c8531700 thread_name:md_log_replay 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7d98f2d20] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55c126a9b38f] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c126a9eb23] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 4: (MDLog::_replay_thread()+0x617) [0x55c126af9647] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c126b02d31] 2024-04-23T18:36:04.337 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff7d98e81ca] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 7: clone() 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 0> 2024-04-23T18:36:03.961+0000 7ff7c8531700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: in thread 7ff7c8531700 thread_name:md_log_replay 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7d98f2d20] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55c126a9b38f] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55c126a9eb23] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 4: (MDLog::_replay_thread()+0x617) [0x55c126af9647] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55c126b02d31] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff7d98e81ca] 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 7: clone() 2024-04-23T18:36:04.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:36:04.339 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68360]: 2024-04-23T18:36:04.339 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:04 smithi122 podman[68769]: 2024-04-23 18:36:04.019131303 +0000 UTC m=+0.025973709 container died 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-23T18:36:04.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:04 smithi122 ceph-mon[32546]: pgmap v299: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-04-23T18:36:04.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:04 smithi122 ceph-mon[32546]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:36:04.947 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:04 smithi122 podman[68769]: 2024-04-23 18:36:04.693954749 +0000 UTC m=+0.700797127 container remove 40e418062bfb303c4aca3236f213b3a6092f279a8368099024853a094ee9b0de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T18:36:04.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:36:04.948 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:36:05.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:04 smithi106 ceph-mon[32716]: pgmap v299: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-04-23T18:36:05.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:04 smithi106 ceph-mon[32716]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:36:05.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:04 smithi033 ceph-mon[29274]: pgmap v299: 97 pgs: 97 active+clean; 7.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 66 KiB/s wr, 29 op/s 2024-04-23T18:36:05.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:04 smithi033 ceph-mon[29274]: osdmap e98: 12 total, 12 up, 12 in 2024-04-23T18:36:07.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:06 smithi106 ceph-mon[32716]: pgmap v301: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:07.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:06 smithi122 ceph-mon[32546]: pgmap v301: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:07.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:06 smithi033 ceph-mon[29274]: pgmap v301: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:09.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:08 smithi106 ceph-mon[32716]: pgmap v302: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:09.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:08 smithi122 ceph-mon[32546]: pgmap v302: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:09.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:08 smithi033 ceph-mon[29274]: pgmap v302: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:11.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:10 smithi106 ceph-mon[32716]: pgmap v303: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:11.180 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:10 smithi106 ceph-mon[32716]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:36:11.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:10 smithi033 ceph-mon[29274]: pgmap v303: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:11.280 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:10 smithi033 ceph-mon[29274]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:36:11.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:10 smithi122 ceph-mon[32546]: pgmap v303: 97 pgs: 97 active+clean; 7.3 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 83 KiB/s wr, 38 op/s 2024-04-23T18:36:11.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:10 smithi122 ceph-mon[32546]: osdmap e99: 12 total, 12 up, 12 in 2024-04-23T18:36:13.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:12 smithi106 ceph-mon[32716]: pgmap v305: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-23T18:36:13.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:12 smithi122 ceph-mon[32546]: pgmap v305: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-23T18:36:13.248 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:12 smithi033 ceph-mon[29274]: pgmap v305: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-23T18:36:15.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:14 smithi106 ceph-mon[32716]: pgmap v306: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:15.179 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:14 smithi122 ceph-mon[32546]: pgmap v306: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:15.223 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:14 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 4. 2024-04-23T18:36:15.289 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:14 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:36:15.643 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:14 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:36:15.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:14 smithi033 ceph-mon[29274]: pgmap v306: 97 pgs: 97 active+clean; 7.4 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:15.807 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 podman[68871]: 2024-04-23 18:36:15.091943552 +0000 UTC m=+0.018651632 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:36:15.857 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 podman[68871]: 2024-04-23 18:36:15.203562764 +0000 UTC m=+0.130270809 container create 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T18:36:15.907 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 podman[68871]: 2024-04-23 18:36:15.439654926 +0000 UTC m=+0.366362980 container init 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:36:15.957 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 podman[68871]: 2024-04-23 18:36:15.442532672 +0000 UTC m=+0.369240727 container start 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T18:36:16.051 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mds[68890]: -- 172.21.15.122:0/3692106110 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559390529a40 con 0x5593904fd000 2024-04-23T18:36:16.107 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: starting mds.f at 2024-04-23T18:36:16.157 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 bash[68871]: 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e 2024-04-23T18:36:16.207 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:36:15 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:36:16.322 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:15 smithi106 ceph-mon[32716]: daemon mds.f restarted 2024-04-23T18:36:16.424 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:15 smithi106 ceph-mon[32716]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:36:16.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/2772764019,v1:172.21.15.122:6835/2772764019] up:boot 2024-04-23T18:36:16.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:15 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:16.574 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:16.632 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:15 smithi033 ceph-mon[29274]: daemon mds.f restarted 2024-04-23T18:36:16.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:15 smithi033 ceph-mon[29274]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:36:16.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/2772764019,v1:172.21.15.122:6835/2772764019] up:boot 2024-04-23T18:36:16.783 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:15 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:16.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:16.883 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mon[32546]: daemon mds.f restarted 2024-04-23T18:36:16.933 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mon[32546]: osdmap e100: 12 total, 12 up, 12 in 2024-04-23T18:36:16.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/2772764019,v1:172.21.15.122:6835/2772764019] up:boot 2024-04-23T18:36:17.074 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:36:17.124 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:36:17.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:16 smithi106 ceph-mon[32716]: pgmap v307: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:36:17.227 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:16 smithi122 ceph-mon[32546]: pgmap v307: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:36:17.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 -- 172.21.15.33:0/3918209510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc101aa0 msgr2=0x7fb7bc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:17.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/3918209510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc101aa0 0x7fb7bc105a90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009900 tx=0x7fb7b002a410 comp rx=0 tx=0).stop 2024-04-23T18:36:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 -- 172.21.15.33:0/3918209510 shutdown_connections 2024-04-23T18:36:17.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/3918209510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc106170 0x7fb7bc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:17.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/3918209510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc101aa0 0x7fb7bc105a90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:17.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/3918209510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc1010f0 0x7fb7bc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 -- 172.21.15.33:0/3918209510 >> 172.21.15.33:0/3918209510 conn(0x7fb7bc077fd0 msgr2=0x7fb7bc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 -- 172.21.15.33:0/3918209510 shutdown_connections 2024-04-23T18:36:17.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.278+0000 7fb7c1c10640 1 -- 172.21.15.33:0/3918209510 wait complete. 2024-04-23T18:36:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 Processor -- start 2024-04-23T18:36:17.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 -- start start 2024-04-23T18:36:17.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc1010f0 0x7fb7bc19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:17.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc106170 0x7fb7bc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:17.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:18.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1110b0 con 0x7fb7bc1985a0 2024-04-23T18:36:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc110f30 con 0x7fb7bc106170 2024-04-23T18:36:18.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7c1c10640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc111230 con 0x7fb7bc1010f0 2024-04-23T18:36:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48944/0 (socket says 172.21.15.33:48944) 2024-04-23T18:36:18.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7bbfff640 1 -- 172.21.15.33:0/1388321382 learned_addr learned my addr 172.21.15.33:0/1388321382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.279+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc1010f0 0x7fb7bc19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7baffd640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc106170 0x7fb7bc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:18.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 -- 172.21.15.33:0/1388321382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc1010f0 msgr2=0x7fb7bc19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc1010f0 0x7fb7bc19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 -- 172.21.15.33:0/1388321382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc106170 msgr2=0x7fb7bc19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc106170 0x7fb7bc19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 -- 172.21.15.33:0/1388321382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc1990c0 con 0x7fb7bc1985a0 2024-04-23T18:36:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7bbfff640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400eb10 tx=0x7fb7b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:18.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7b8ff9640 1 -- 172.21.15.33:0/1388321382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4004b20 con 0x7fb7bc1985a0 2024-04-23T18:36:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7b8ff9640 1 -- 172.21.15.33:0/1388321382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b4004cc0 con 0x7fb7bc1985a0 2024-04-23T18:36:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc1993b0 con 0x7fb7bc1985a0 2024-04-23T18:36:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7b8ff9640 1 -- 172.21.15.33:0/1388321382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4010670 con 0x7fb7bc1985a0 2024-04-23T18:36:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.280+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1a5650 con 0x7fb7bc1985a0 2024-04-23T18:36:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.282+0000 7fb7b8ff9640 1 -- 172.21.15.33:0/1388321382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb7b4005020 con 0x7fb7bc1985a0 2024-04-23T18:36:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 Processor -- start 2024-04-23T18:36:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 -- start start 2024-04-23T18:36:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18520 0x562b8db18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db27c10 0x562b8db2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b8da7d970 con 0x562b8db18e40 2024-04-23T18:36:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b8da1a4a0 con 0x562b8db27c10 2024-04-23T18:36:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b8da79930 con 0x562b8db18520 2024-04-23T18:36:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48954/0 (socket says 172.21.15.33:48954) 2024-04-23T18:36:19.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/3307247958 learned_addr learned my addr 172.21.15.33:0/3307247958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:19.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18520 0x562b8db18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db27c10 0x562b8db2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7b8ff9640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 0x7fb7a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/3307247958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18520 msgr2=0x562b8db18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18520 0x562b8db18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/3307247958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db27c10 msgr2=0x562b8db2a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db27c10 0x562b8db2a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/3307247958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8d790210 con 0x562b8db18e40 2024-04-23T18:36:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7bbfff640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 0x7fb7a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.283+0000 7fb7b8ff9640 1 -- 172.21.15.33:0/1388321382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fb7b40d0dd0 con 0x7fb7bc1985a0 2024-04-23T18:36:19.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009c80 tx=0x7fb7ac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284413a979be1819 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7a1ffb640 1 -- 172.21.15.33:0/3307247958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac012450 con 0x562b8db18e40 2024-04-23T18:36:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7a1ffb640 1 -- 172.21.15.33:0/3307247958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac012aa0 con 0x562b8db18e40 2024-04-23T18:36:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7a1ffb640 1 -- 172.21.15.33:0/3307247958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac00fa90 con 0x562b8db18e40 2024-04-23T18:36:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3307247958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 msgr2=0x562b8db276d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7bbfff640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 0x7fb7a40a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4099180 tx=0x7fb7b406e760 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:20.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.284+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009c80 tx=0x7fb7ac0117b0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3307247958 shutdown_connections 2024-04-23T18:36:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db27c10 0x562b8db2a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db276d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3307247958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18520 0x562b8db18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3307247958 >> 172.21.15.33:0/3307247958 conn(0x562b8db14b30 msgr2=0x562b8db14f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3307247958 shutdown_connections 2024-04-23T18:36:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3307247958 wait complete. 2024-04-23T18:36:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- start start 2024-04-23T18:36:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 Processor -- start 2024-04-23T18:36:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:20.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db27c10 0x562b8db24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b8db2a840 con 0x562b8db18e40 2024-04-23T18:36:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b8da180b0 con 0x562b8db18520 2024-04-23T18:36:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.285+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b8db2a9c0 con 0x562b8db27c10 2024-04-23T18:36:20.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7c49fe740 10 client.0 _ll_register_callbacks cb 0x562b8da05d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:21.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48968/0 (socket says 172.21.15.33:48968) 2024-04-23T18:36:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/2898724520 learned_addr learned my addr 172.21.15.33:0/2898724520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7baffd640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db27c10 0x562b8db24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7baffd640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/2898724520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db27c10 msgr2=0x562b8db24d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db27c10 0x562b8db24d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 msgr2=0x562b8db274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 -- 172.21.15.33:0/2898724520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8d85c620 con 0x562b8db18e40 2024-04-23T18:36:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009c50 tx=0x7fb7ac009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9216cbcce4e331a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb79affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:36:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac028020 con 0x562b8db18e40 2024-04-23T18:36:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fb7ac004240 con 0x562b8db18e40 2024-04-23T18:36:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7ac012aa0 con 0x562b8db18e40 2024-04-23T18:36:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.286+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb7ac033050 con 0x562b8db18e40 2024-04-23T18:36:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 10 client.16857 fetch_fsmap learned FSMap version 35 2024-04-23T18:36:21.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b8db36750 con 0x562b8db18e40 2024-04-23T18:36:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7ac012aa0 con 0x562b8db18e40 2024-04-23T18:36:21.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb7ac00fb30 con 0x562b8db18e40 2024-04-23T18:36:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 10 client.16857 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:36:21.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 20 client.16857 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8db39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:22.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 4 client.16857 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8da8b390 con 0x562b8db37bb0 2024-04-23T18:36:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.287+0000 7fb7baffd640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8db39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:22.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.288+0000 7fb7baffd640 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8db39fa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=924f9b7c0795c777 server_cookie=78f1044fcae1f36d in_seq=0 out_seq=0 2024-04-23T18:36:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.288+0000 7fb79affd640 10 client.16857 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.288+0000 7fb79affd640 1 -- 172.21.15.33:0/2898724520 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b8da8b390 con 0x562b8db37bb0 2024-04-23T18:36:22.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.288+0000 7fb79affd640 10 client.16857 handle_command_reply: tid=0 2024-04-23T18:36:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.288+0000 7fb7c49fe740 1 client.16857 shutdown 2024-04-23T18:36:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 msgr2=0x562b8db247c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009c50 tx=0x7fb7ac009fa0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 shutdown_connections 2024-04-23T18:36:22.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8db39fa0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db27c10 0x562b8db24d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db18e40 0x562b8db247c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb79affd640 0 client.16857 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/2898724520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 >> 172.21.15.33:0/2898724520 conn(0x562b8db14b30 msgr2=0x562b8db1a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 shutdown_connections 2024-04-23T18:36:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 1 -- 172.21.15.33:0/2898724520 wait complete. 2024-04-23T18:36:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.289+0000 7fb7c49fe740 20 client.16857 trim_cache size 0 max 16384 2024-04-23T18:36:22.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 Processor -- start 2024-04-23T18:36:22.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 -- start start 2024-04-23T18:36:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db33700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db1fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ab9c0 con 0x562b8db27c10 2024-04-23T18:36:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ad500 con 0x562b8db18520 2024-04-23T18:36:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ac090 con 0x562b8db18e40 2024-04-23T18:36:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48982/0 (socket says 172.21.15.33:48982) 2024-04-23T18:36:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7baffd640 1 -- 172.21.15.33:0/3339687558 learned_addr learned my addr 172.21.15.33:0/3339687558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:23.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db1fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.304+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db33700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 -- 172.21.15.33:0/3339687558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 msgr2=0x562b8db1fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:23.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db1fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 -- 172.21.15.33:0/3339687558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 msgr2=0x562b8db33700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db33700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 -- 172.21.15.33:0/3339687558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8db19380 con 0x562b8db27c10 2024-04-23T18:36:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7baffd640 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc06b730 tx=0x7fb7b0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa24e6b130dcace server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7a17fa640 1 -- 172.21.15.33:0/3339687558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0035040 con 0x562b8db27c10 2024-04-23T18:36:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7a17fa640 1 -- 172.21.15.33:0/3339687558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b0054070 con 0x562b8db27c10 2024-04-23T18:36:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.305+0000 7fb7a17fa640 1 -- 172.21.15.33:0/3339687558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00052b0 con 0x562b8db27c10 2024-04-23T18:36:23.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3339687558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 msgr2=0x562b8db27000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc06b730 tx=0x7fb7b0031400 comp rx=0 tx=0).stop 2024-04-23T18:36:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3339687558 shutdown_connections 2024-04-23T18:36:23.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db27000 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db1fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/3339687558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db33700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:23.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3339687558 >> 172.21.15.33:0/3339687558 conn(0x562b8db14b30 msgr2=0x562b8db1f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:23.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3339687558 shutdown_connections 2024-04-23T18:36:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- 172.21.15.33:0/3339687558 wait complete. 2024-04-23T18:36:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 -- start start 2024-04-23T18:36:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 Processor -- start 2024-04-23T18:36:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.306+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db2fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db2b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:24.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ace30 con 0x562b8db27c10 2024-04-23T18:36:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1ac760 con 0x562b8db18520 2024-04-23T18:36:24.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1a99a0 con 0x562b8db18e40 2024-04-23T18:36:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 10 client.0 _ll_register_callbacks cb 0x7fb7ac009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2024-04-23T18:36:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 -- 172.21.15.33:0/8797092 learned_addr learned my addr 172.21.15.33:0/8797092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db2b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db2fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 -- 172.21.15.33:0/8797092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 msgr2=0x562b8db2b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:24.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db2b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:24.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 -- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 msgr2=0x562b8db2fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:24.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db2fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 -- 172.21.15.33:0/8797092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8db14960 con 0x562b8db27c10 2024-04-23T18:36:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7baffd640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006010 tx=0x7fb7b0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a1919356355abd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb79affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:36:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b005c020 con 0x562b8db27c10 2024-04-23T18:36:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fb7b004d5c0 con 0x562b8db27c10 2024-04-23T18:36:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b0054070 con 0x562b8db27c10 2024-04-23T18:36:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb7b0035070 con 0x562b8db27c10 2024-04-23T18:36:25.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 10 client.16863 fetch_fsmap learned FSMap version 35 2024-04-23T18:36:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.307+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562b8db9d9f0 con 0x562b8db27c10 2024-04-23T18:36:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7b0054070 con 0x562b8db27c10 2024-04-23T18:36:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b003a040 con 0x562b8db27c10 2024-04-23T18:36:25.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 10 client.16863 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:36:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 20 client.16863 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8dba0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 4 client.16863 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b8db25090 con 0x562b8db37bb0 2024-04-23T18:36:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.308+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8dba0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.309+0000 7fb79affd640 10 client.16863 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.309+0000 7fb7bb7fe640 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8dba0060 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6afe53e29272dcf7 server_cookie=158fe7115ed71754 in_seq=0 out_seq=0 2024-04-23T18:36:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.316+0000 7fb79affd640 1 -- 172.21.15.33:0/8797092 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b8db25090 con 0x562b8db37bb0 2024-04-23T18:36:26.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.316+0000 7fb79affd640 10 client.16863 handle_command_reply: tid=0 2024-04-23T18:36:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.316+0000 7fb7c49fe740 1 client.16863 shutdown 2024-04-23T18:36:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 msgr2=0x562b8db2b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0006010 tx=0x7fb7b0005be0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 shutdown_connections 2024-04-23T18:36:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8db37bb0 0x562b8dba0060 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8db27c10 0x562b8db2b710 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8db18e40 0x562b8db2b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb79affd640 0 client.16863 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 --2- 172.21.15.33:0/8797092 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8db18520 0x562b8db2fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:26.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 >> 172.21.15.33:0/8797092 conn(0x562b8db14b30 msgr2=0x562b8db1a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:26.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 shutdown_connections 2024-04-23T18:36:26.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 1 -- 172.21.15.33:0/8797092 wait complete. 2024-04-23T18:36:26.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c49fe740 20 client.16863 trim_cache size 0 max 16384 2024-04-23T18:36:26.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:36:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:36:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:36:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:36:27.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 msgr2=0x7fb7a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 0x7fb7a40a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4099180 tx=0x7fb7b406e760 comp rx=0 tx=0).stop 2024-04-23T18:36:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 msgr2=0x7fb7bc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400eb10 tx=0x7fb7b400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 shutdown_connections 2024-04-23T18:36:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb7a40a3e40 0x7fb7a40a6300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7bc1985a0 0x7fb7bc198a50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb7bc106170 0x7fb7bc19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 --2- 172.21.15.33:0/1388321382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7bc1010f0 0x7fb7bc19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.317+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 >> 172.21.15.33:0/1388321382 conn(0x7fb7bc077fd0 msgr2=0x7fb7bc069a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:27.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.318+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 shutdown_connections 2024-04-23T18:36:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:16.318+0000 7fb7c1c10640 1 -- 172.21.15.33:0/1388321382 wait complete. 2024-04-23T18:36:27.599 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:18 smithi106 ceph-mon[32716]: pgmap v309: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:27.656 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:20 smithi106 ceph-mon[32716]: pgmap v310: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-04-23T18:36:27.674 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:22 smithi106 ceph-mon[32716]: pgmap v311: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:27.777 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:24 smithi106 ceph-mon[32716]: pgmap v312: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:27.799 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:18 smithi122 ceph-mon[32546]: pgmap v309: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:27.882 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:20 smithi122 ceph-mon[32546]: pgmap v310: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-04-23T18:36:27.907 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:22 smithi122 ceph-mon[32546]: pgmap v311: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:27.932 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:24 smithi122 ceph-mon[32546]: pgmap v312: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:27.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:16 smithi033 ceph-mon[29274]: pgmap v307: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-23T18:36:27.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:18 smithi033 ceph-mon[29274]: pgmap v309: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 16 op/s 2024-04-23T18:36:28.024 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:20 smithi033 ceph-mon[29274]: pgmap v310: 97 pgs: 97 active+clean; 7.5 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-04-23T18:36:28.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:22 smithi033 ceph-mon[29274]: pgmap v311: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:28.124 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:24 smithi033 ceph-mon[29274]: pgmap v312: 97 pgs: 97 active+clean; 7.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-23T18:36:28.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:26 smithi033 ceph-mon[29274]: pgmap v313: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T18:36:28.192 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:26 smithi106 ceph-mon[32716]: pgmap v313: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T18:36:28.224 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:26 smithi122 ceph-mon[32546]: pgmap v313: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 18 op/s 2024-04-23T18:36:29.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:28 smithi122 ceph-mon[32546]: pgmap v314: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:29.267 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:28 smithi033 ceph-mon[29274]: pgmap v314: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:29.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:28 smithi106 ceph-mon[32716]: pgmap v314: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 -- 172.21.15.33:0/1546185583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb4107270 msgr2=0x7fbeb410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/1546185583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb4107270 0x7fbeb410ee40 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009900 tx=0x7fbeac02a410 comp rx=0 tx=0).stop 2024-04-23T18:36:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 -- 172.21.15.33:0/1546185583 shutdown_connections 2024-04-23T18:36:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/1546185583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb4107270 0x7fbeb410ee40 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/1546185583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb4102b50 0x7fbeb4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/1546185583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb41021a0 0x7fbeb4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.416+0000 7fbeba1f0640 1 -- 172.21.15.33:0/1546185583 >> 172.21.15.33:0/1546185583 conn(0x7fbeb40fda00 msgr2=0x7fbeb40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:29.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.417+0000 7fbeba1f0640 1 -- 172.21.15.33:0/1546185583 shutdown_connections 2024-04-23T18:36:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.417+0000 7fbeba1f0640 1 -- 172.21.15.33:0/1546185583 wait complete. 2024-04-23T18:36:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.417+0000 7fbeba1f0640 1 Processor -- start 2024-04-23T18:36:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.417+0000 7fbeba1f0640 1 -- start start 2024-04-23T18:36:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4107270 0x7fbeb4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 0x7fbeb419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112210 con 0x7fbeb41021a0 2024-04-23T18:36:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112090 con 0x7fbeb419ceb0 2024-04-23T18:36:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeba1f0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb4112390 con 0x7fbeb4107270 2024-04-23T18:36:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:30.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58874/0 (socket says 172.21.15.33:58874) 2024-04-23T18:36:30.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 -- 172.21.15.33:0/2324432773 learned_addr learned my addr 172.21.15.33:0/2324432773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 0x7fbeb419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4107270 0x7fbeb4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:30.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 -- 172.21.15.33:0/2324432773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4107270 msgr2=0x7fbeb4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4107270 0x7fbeb4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:30.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 -- 172.21.15.33:0/2324432773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 msgr2=0x7fbeb419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 0x7fbeb419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 -- 172.21.15.33:0/2324432773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb419d9a0 con 0x7fbeb41021a0 2024-04-23T18:36:30.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 0x7fbeb419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbea400cce0 tx=0x7fbea400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:30.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.418+0000 7fbeb0ff9640 1 -- 172.21.15.33:0/2324432773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea400f650 con 0x7fbeb41021a0 2024-04-23T18:36:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.419+0000 7fbeb0ff9640 1 -- 172.21.15.33:0/2324432773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea400fca0 con 0x7fbeb41021a0 2024-04-23T18:36:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.419+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb419dc90 con 0x7fbeb41021a0 2024-04-23T18:36:30.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.419+0000 7fbeb0ff9640 1 -- 172.21.15.33:0/2324432773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4018b40 con 0x7fbeb41021a0 2024-04-23T18:36:30.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.419+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb41a5480 con 0x7fbeb41021a0 2024-04-23T18:36:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.421+0000 7fbeb0ff9640 1 -- 172.21.15.33:0/2324432773 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbea400f7f0 con 0x7fbeb41021a0 2024-04-23T18:36:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.421+0000 7fbebcfde740 1 Processor -- start 2024-04-23T18:36:31.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.421+0000 7fbebcfde740 1 -- start start 2024-04-23T18:36:31.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.421+0000 7fbeb0ff9640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 0x7fbe9c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d3a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:31.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564634c9f970 con 0x564634d49c10 2024-04-23T18:36:31.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564634c3c4a0 con 0x564634d3ae20 2024-04-23T18:36:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbebcfde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564634c9b930 con 0x564634d3a500 2024-04-23T18:36:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 0x7fbe9c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb0ff9640 1 -- 172.21.15.33:0/2324432773 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fbea40d0de0 con 0x7fbeb41021a0 2024-04-23T18:36:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d3a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58884/0 (socket says 172.21.15.33:58884) 2024-04-23T18:36:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb3fff640 1 -- 172.21.15.33:0/3998765607 learned_addr learned my addr 172.21.15.33:0/3998765607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 -- 172.21.15.33:0/3998765607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 msgr2=0x564634d496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 -- 172.21.15.33:0/3998765607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 msgr2=0x564634d4c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:31.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 -- 172.21.15.33:0/3998765607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646349b2210 con 0x564634d3a500 2024-04-23T18:36:31.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 0x7fbe9c0a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbeac005c40 tx=0x7fbeac004190 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.422+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d3a8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbea40ce4e0 tx=0x7fbea4099100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf193846c519fe72 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbe99ffb640 1 -- 172.21.15.33:0/3998765607 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea407a170 con 0x564634d3a500 2024-04-23T18:36:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbe99ffb640 1 -- 172.21.15.33:0/3998765607 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea400fae0 con 0x564634d3a500 2024-04-23T18:36:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbe99ffb640 1 -- 172.21.15.33:0/3998765607 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea40a59a0 con 0x564634d3a500 2024-04-23T18:36:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 -- 172.21.15.33:0/3998765607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 msgr2=0x564634d3a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d3a8e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbea40ce4e0 tx=0x7fbea4099100 comp rx=0 tx=0).stop 2024-04-23T18:36:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 -- 172.21.15.33:0/3998765607 shutdown_connections 2024-04-23T18:36:32.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3998765607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d3a8e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 -- 172.21.15.33:0/3998765607 >> 172.21.15.33:0/3998765607 conn(0x564634d36b30 msgr2=0x564634d36f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 -- 172.21.15.33:0/3998765607 shutdown_connections 2024-04-23T18:36:32.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.423+0000 7fbebcfde740 1 -- 172.21.15.33:0/3998765607 wait complete. 2024-04-23T18:36:32.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 -- start start 2024-04-23T18:36:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 Processor -- start 2024-04-23T18:36:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d46dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564634d4c7e0 con 0x564634d49c10 2024-04-23T18:36:32.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564634c3a0b0 con 0x564634d3a500 2024-04-23T18:36:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564634d4c960 con 0x564634d3ae20 2024-04-23T18:36:32.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbebcfde740 10 client.0 _ll_register_callbacks cb 0x564634c27d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d46dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d46dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58898/0 (socket says 172.21.15.33:58898) 2024-04-23T18:36:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbeb3fff640 1 -- 172.21.15.33:0/3897274888 learned_addr learned my addr 172.21.15.33:0/3897274888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.424+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 -- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 msgr2=0x564634d49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 -- 172.21.15.33:0/3897274888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 msgr2=0x564634d46dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d46dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 -- 172.21.15.33:0/3897274888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564634a7e620 con 0x564634d3ae20 2024-04-23T18:36:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d46860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4069280 tx=0x7fbea8012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ee2b5df90b50e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:33.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbe92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:36:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea8004230 con 0x564634d3ae20 2024-04-23T18:36:33.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fbea80043d0 con 0x564634d3ae20 2024-04-23T18:36:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564634cad390 con 0x564634d3ae20 2024-04-23T18:36:33.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbea80035f0 con 0x564634d3ae20 2024-04-23T18:36:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbebcfde740 10 client.26326 fetch_fsmap learned FSMap version 35 2024-04-23T18:36:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.425+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564634d587f0 con 0x564634d3ae20 2024-04-23T18:36:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564634cad390 con 0x564634d3ae20 2024-04-23T18:36:33.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbea8010a90 con 0x564634d3ae20 2024-04-23T18:36:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 10 client.26326 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:36:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 10 client.26326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 20 client.26326 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634d59c60 0x564634d5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 4 client.26326 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:36:33.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564634ca7200 con 0x564634d59c60 2024-04-23T18:36:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634d59c60 0x564634d5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbe92ffd640 10 client.26326 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:34.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.426+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634d59c60 0x564634d5c090 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c54ed778626e58 server_cookie=f463d4623a8d98cb in_seq=0 out_seq=0 2024-04-23T18:36:34.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbe92ffd640 1 -- 172.21.15.33:0/3897274888 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564634ca7200 con 0x564634d59c60 2024-04-23T18:36:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbe92ffd640 10 client.26326 handle_command_reply: tid=0 2024-04-23T18:36:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 client.26326 shutdown 2024-04-23T18:36:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 msgr2=0x564634d46860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d46860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbeb4069280 tx=0x7fbea8012820 comp rx=0 tx=0).stop 2024-04-23T18:36:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 shutdown_connections 2024-04-23T18:36:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634d59c60 0x564634d5c090 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d46dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d46860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbe92ffd640 0 client.26326 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 --2- 172.21.15.33:0/3897274888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:34.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.427+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 >> 172.21.15.33:0/3897274888 conn(0x564634d36b30 msgr2=0x564634d3c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.428+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 shutdown_connections 2024-04-23T18:36:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.428+0000 7fbebcfde740 1 -- 172.21.15.33:0/3897274888 wait complete. 2024-04-23T18:36:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.428+0000 7fbebcfde740 20 client.26326 trim_cache size 0 max 16384 2024-04-23T18:36:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 Processor -- start 2024-04-23T18:36:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 -- start start 2024-04-23T18:36:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:34.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41ab8c0 con 0x564634d49c10 2024-04-23T18:36:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41ad400 con 0x564634d3a500 2024-04-23T18:36:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbebcfde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41abf90 con 0x564634d3ae20 2024-04-23T18:36:35.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:35.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58914/0 (socket says 172.21.15.33:58914) 2024-04-23T18:36:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 -- 172.21.15.33:0/1876468668 learned_addr learned my addr 172.21.15.33:0/1876468668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:35.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 -- 172.21.15.33:0/1876468668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 msgr2=0x564634d490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d490a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 -- 172.21.15.33:0/1876468668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 msgr2=0x564634d556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:35.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 -- 172.21.15.33:0/1876468668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564634d3baa0 con 0x564634d49c10 2024-04-23T18:36:35.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:36:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.443+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02aa00 tx=0x7fbeac039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4623129c38e2ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbe997fa640 1 -- 172.21.15.33:0/1876468668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac035040 con 0x564634d49c10 2024-04-23T18:36:35.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbe997fa640 1 -- 172.21.15.33:0/1876468668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeac050070 con 0x564634d49c10 2024-04-23T18:36:35.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbe997fa640 1 -- 172.21.15.33:0/1876468668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004e40 con 0x564634d49c10 2024-04-23T18:36:35.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 -- 172.21.15.33:0/1876468668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 msgr2=0x564634d619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbeac02aa00 tx=0x7fbeac039000 comp rx=0 tx=0).stop 2024-04-23T18:36:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 -- 172.21.15.33:0/1876468668 shutdown_connections 2024-04-23T18:36:36.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d619a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3ae20 0x564634d490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 --2- 172.21.15.33:0/1876468668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3a500 0x564634d556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 -- 172.21.15.33:0/1876468668 >> 172.21.15.33:0/1876468668 conn(0x564634d36b30 msgr2=0x564634d41f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:36.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 -- 172.21.15.33:0/1876468668 shutdown_connections 2024-04-23T18:36:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.444+0000 7fbebcfde740 1 -- 172.21.15.33:0/1876468668 wait complete. 2024-04-23T18:36:36.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 -- start start 2024-04-23T18:36:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 Processor -- start 2024-04-23T18:36:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d51bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d52130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41acd30 con 0x564634d49c10 2024-04-23T18:36:36.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41ac660 con 0x564634d3ae20 2024-04-23T18:36:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb41a98a0 con 0x564634d3a500 2024-04-23T18:36:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbebcfde740 10 client.0 _ll_register_callbacks cb 0x7fbea800b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:36:36.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58928/0 (socket says 172.21.15.33:58928) 2024-04-23T18:36:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbeb3fff640 1 -- 172.21.15.33:0/394680833 learned_addr learned my addr 172.21.15.33:0/394680833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:36:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbeb2ffd640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d52130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.445+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d51bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 -- 172.21.15.33:0/394680833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 msgr2=0x564634d51bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:36.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d51bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:36.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 -- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 msgr2=0x564634d52130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:36:37.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d52130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 -- 172.21.15.33:0/394680833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564634d4f200 con 0x564634d49c10 2024-04-23T18:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbeb3fff640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0092a0 tx=0x7fbeac0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b428706ac58b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:36:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:36:37.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac035030 con 0x564634d49c10 2024-04-23T18:36:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fbeac005030 con 0x564634d49c10 2024-04-23T18:36:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbeac050070 con 0x564634d49c10 2024-04-23T18:36:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbeac004800 con 0x564634d49c10 2024-04-23T18:36:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbebcfde740 10 client.16881 fetch_fsmap learned FSMap version 35 2024-04-23T18:36:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564634dc0b70 con 0x564634d49c10 2024-04-23T18:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeac050070 con 0x564634d49c10 2024-04-23T18:36:37.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.446+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbeac03a040 con 0x564634d49c10 2024-04-23T18:36:37.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 10 client.16881 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:36:37.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:36:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 20 client.16881 populate_metadata read hostname 'smithi033' 2024-04-23T18:36:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634dc1260 0x564634dc3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 4 client.16881 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:36:37.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564634c235f0 con 0x564634dc1260 2024-04-23T18:36:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634dc1260 0x564634dc3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:37.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbeb37fe640 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634dc1260 0x564634dc3680 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c2180dc46b80b7 server_cookie=8a567c313e8f529b in_seq=0 out_seq=0 2024-04-23T18:36:38.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.447+0000 7fbe92ffd640 10 client.16881 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbe92ffd640 1 -- 172.21.15.33:0/394680833 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564634c235f0 con 0x564634dc1260 2024-04-23T18:36:38.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbe92ffd640 10 client.16881 handle_command_reply: tid=0 2024-04-23T18:36:38.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 client.16881 shutdown 2024-04-23T18:36:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 msgr2=0x564634d4ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:38.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0092a0 tx=0x7fbeac0045f0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 shutdown_connections 2024-04-23T18:36:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x564634dc1260 0x564634dc3680 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564634d49c10 0x564634d4ebc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564634d3ae20 0x564634d52130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbe92ffd640 0 client.16881 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:36:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 --2- 172.21.15.33:0/394680833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564634d3a500 0x564634d51bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.455+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 >> 172.21.15.33:0/394680833 conn(0x564634d36b30 msgr2=0x564634d35420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:38.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 shutdown_connections 2024-04-23T18:36:38.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbebcfde740 1 -- 172.21.15.33:0/394680833 wait complete. 2024-04-23T18:36:38.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbebcfde740 20 client.16881 trim_cache size 0 max 16384 2024-04-23T18:36:38.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:36:38.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:36:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:36:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:36:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 msgr2=0x7fbe9c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 0x7fbe9c0a6280 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbeac005c40 tx=0x7fbeac004190 comp rx=0 tx=0).stop 2024-04-23T18:36:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 msgr2=0x7fbeb41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbea400cce0 tx=0x7fbea400a620 comp rx=0 tx=0).stop 2024-04-23T18:36:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 shutdown_connections 2024-04-23T18:36:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe9c0a3dc0 0x7fbe9c0a6280 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbeb419ceb0 0x7fbeb419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbeb4107270 0x7fbeb4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 --2- 172.21.15.33:0/2324432773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeb41021a0 0x7fbeb41985f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 >> 172.21.15.33:0/2324432773 conn(0x7fbeb40fda00 msgr2=0x7fbeb40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 shutdown_connections 2024-04-23T18:36:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:29.456+0000 7fbeba1f0640 1 -- 172.21.15.33:0/2324432773 wait complete. 2024-04-23T18:36:39.227 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.9 2024-04-23T18:36:39.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:36:39.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:30 smithi033 ceph-mon[29274]: pgmap v315: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:39.399 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:32 smithi033 ceph-mon[29274]: pgmap v316: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-23T18:36:39.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:34 smithi033 ceph-mon[29274]: pgmap v317: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:39.467 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:39.499 INFO:tasks.workunit.client.0.smithi033.stdout:313/600: open fd = 3 2024-04-23T18:36:39.532 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.566 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.599 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:39.633 INFO:tasks.workunit.client.0.smithi033.stdout:314/600: open fd = 3 2024-04-23T18:36:39.666 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.699 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.733 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:39.766 INFO:tasks.workunit.client.0.smithi033.stdout:315/600: open fd = 3 2024-04-23T18:36:39.799 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.833 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:39.866 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:39.899 INFO:tasks.workunit.client.0.smithi033.stdout:316/600: open fd = 3 2024-04-23T18:36:39.933 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.066 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.102 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:40.133 INFO:tasks.workunit.client.0.smithi033.stdout:317/600: open fd = 3 2024-04-23T18:36:40.183 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.216 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.250 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:40.283 INFO:tasks.workunit.client.0.smithi033.stdout:318/600: open fd = 3 2024-04-23T18:36:40.316 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.413 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.443 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:40.475 INFO:tasks.workunit.client.0.smithi033.stdout:319/600: open fd = 3 2024-04-23T18:36:40.508 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.542 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.575 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:40.700 INFO:tasks.workunit.client.0.smithi033.stdout:320/600: open fd = 3 2024-04-23T18:36:40.759 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.792 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:40.825 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:40.859 INFO:tasks.workunit.client.0.smithi033.stdout:321/600: open fd = 3 2024-04-23T18:36:40.961 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.234 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.267 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:41.300 INFO:tasks.workunit.client.0.smithi033.stdout:322/600: open fd = 3 2024-04-23T18:36:41.351 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.442 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.476 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:41.509 INFO:tasks.workunit.client.0.smithi033.stdout:323/600: open fd = 3 2024-04-23T18:36:41.542 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.576 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.609 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:41.688 INFO:tasks.workunit.client.0.smithi033.stdout:324/600: open fd = 3 2024-04-23T18:36:41.779 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.801 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.829 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:41.929 INFO:tasks.workunit.client.0.smithi033.stdout:325/600: open fd = 3 2024-04-23T18:36:41.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:41.996 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.029 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:42.062 INFO:tasks.workunit.client.0.smithi033.stdout:326/600: open fd = 3 2024-04-23T18:36:42.095 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.129 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.179 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:42.241 INFO:tasks.workunit.client.0.smithi033.stdout:327/600: open fd = 3 2024-04-23T18:36:42.279 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.329 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.446 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:42.496 INFO:tasks.workunit.client.0.smithi033.stdout:328/600: open fd = 3 2024-04-23T18:36:42.546 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.596 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.646 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:42.696 INFO:tasks.workunit.client.0.smithi033.stdout:329/600: open fd = 3 2024-04-23T18:36:42.746 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.800 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:42.863 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:42.896 INFO:tasks.workunit.client.0.smithi033.stdout:330/600: open fd = 3 2024-04-23T18:36:42.946 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.080 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.130 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:43.180 INFO:tasks.workunit.client.0.smithi033.stdout:331/600: open fd = 3 2024-04-23T18:36:43.230 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.280 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.330 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:43.380 INFO:tasks.workunit.client.0.smithi033.stdout:332/600: open fd = 3 2024-04-23T18:36:43.455 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.555 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:43.605 INFO:tasks.workunit.client.0.smithi033.stdout:333/600: open fd = 3 2024-04-23T18:36:43.722 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.772 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.822 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:43.872 INFO:tasks.workunit.client.0.smithi033.stdout:334/600: open fd = 3 2024-04-23T18:36:43.930 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:43.980 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.030 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:44.080 INFO:tasks.workunit.client.0.smithi033.stdout:335/600: open fd = 3 2024-04-23T18:36:44.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.180 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.230 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:44.364 INFO:tasks.workunit.client.0.smithi033.stdout:336/600: open fd = 3 2024-04-23T18:36:44.414 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.464 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.514 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:44.564 INFO:tasks.workunit.client.0.smithi033.stdout:337/600: open fd = 3 2024-04-23T18:36:44.614 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.664 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.714 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:44.786 INFO:tasks.workunit.client.0.smithi033.stdout:338/600: open fd = 3 2024-04-23T18:36:44.839 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.889 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:44.997 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:45.047 INFO:tasks.workunit.client.0.smithi033.stdout:339/600: open fd = 3 2024-04-23T18:36:45.097 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.148 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.197 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:45.247 INFO:tasks.workunit.client.0.smithi033.stdout:340/600: open fd = 3 2024-04-23T18:36:45.298 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.348 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.427 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:45.473 INFO:tasks.workunit.client.0.smithi033.stdout:341/600: open fd = 3 2024-04-23T18:36:45.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.631 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.681 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:45.723 INFO:tasks.workunit.client.0.smithi033.stdout:342/600: open fd = 3 2024-04-23T18:36:45.764 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.798 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:45.831 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:45.873 INFO:tasks.workunit.client.0.smithi033.stdout:343/600: open fd = 3 2024-04-23T18:36:45.945 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.026 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.065 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:46.098 INFO:tasks.workunit.client.0.smithi033.stdout:344/600: open fd = 3 2024-04-23T18:36:46.190 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.223 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.257 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:46.290 INFO:tasks.workunit.client.0.smithi033.stdout:345/600: open fd = 3 2024-04-23T18:36:46.323 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.357 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.398 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:46.446 INFO:tasks.workunit.client.0.smithi033.stdout:346/600: open fd = 3 2024-04-23T18:36:46.536 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.565 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.599 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:46.715 INFO:tasks.workunit.client.0.smithi033.stdout:347/600: open fd = 3 2024-04-23T18:36:46.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.799 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.832 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:46.865 INFO:tasks.workunit.client.0.smithi033.stdout:348/600: open fd = 3 2024-04-23T18:36:46.899 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.932 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:46.974 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.001 INFO:tasks.workunit.client.0.smithi033.stdout:349/600: open fd = 3 2024-04-23T18:36:47.054 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.082 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.191 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.224 INFO:tasks.workunit.client.0.smithi033.stdout:350/600: open fd = 3 2024-04-23T18:36:47.257 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.291 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.324 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.366 INFO:tasks.workunit.client.0.smithi033.stdout:351/600: open fd = 3 2024-04-23T18:36:47.399 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.441 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.466 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.543 INFO:tasks.workunit.client.0.smithi033.stdout:352/600: open fd = 3 2024-04-23T18:36:47.574 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.683 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.716 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.749 INFO:tasks.workunit.client.0.smithi033.stdout:353/600: open fd = 3 2024-04-23T18:36:47.799 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.850 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:47.900 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:47.950 INFO:tasks.workunit.client.0.smithi033.stdout:354/600: open fd = 3 2024-04-23T18:36:48.020 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.075 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.175 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:48.225 INFO:tasks.workunit.client.0.smithi033.stdout:355/600: open fd = 3 2024-04-23T18:36:48.358 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.408 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.458 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:48.508 INFO:tasks.workunit.client.0.smithi033.stdout:356/600: open fd = 3 2024-04-23T18:36:48.558 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.608 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.658 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:48.708 INFO:tasks.workunit.client.0.smithi033.stdout:357/600: open fd = 3 2024-04-23T18:36:48.771 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.843 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:48.967 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:49.017 INFO:tasks.workunit.client.0.smithi033.stdout:358/600: open fd = 3 2024-04-23T18:36:49.067 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.117 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.167 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:49.217 INFO:tasks.workunit.client.0.smithi033.stdout:359/600: open fd = 3 2024-04-23T18:36:49.267 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.325 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.375 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:49.426 INFO:tasks.workunit.client.0.smithi033.stdout:360/600: open fd = 3 2024-04-23T18:36:49.475 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.542 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.592 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:49.643 INFO:tasks.workunit.client.0.smithi033.stdout:361/600: open fd = 3 2024-04-23T18:36:49.693 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.743 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:49.793 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:49.843 INFO:tasks.workunit.client.0.smithi033.stdout:362/600: open fd = 3 2024-04-23T18:36:50.001 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.051 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.101 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:50.151 INFO:tasks.workunit.client.0.smithi033.stdout:363/600: open fd = 3 2024-04-23T18:36:50.201 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.251 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.301 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:50.351 INFO:tasks.workunit.client.0.smithi033.stdout:364/600: open fd = 3 2024-04-23T18:36:50.401 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.451 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.501 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:50.551 INFO:tasks.workunit.client.0.smithi033.stdout:365/600: open fd = 3 2024-04-23T18:36:50.601 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.651 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:50.752 INFO:tasks.workunit.client.0.smithi033.stdout:366/600: open fd = 3 2024-04-23T18:36:50.885 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.927 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:50.960 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:50.993 INFO:tasks.workunit.client.0.smithi033.stdout:367/600: open fd = 3 2024-04-23T18:36:51.027 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.060 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.094 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:51.130 INFO:tasks.workunit.client.0.smithi033.stdout:368/600: open fd = 3 2024-04-23T18:36:51.544 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.665 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.702 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:51.780 INFO:tasks.workunit.client.0.smithi033.stdout:369/600: open fd = 3 2024-04-23T18:36:51.811 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.894 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:51.927 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:51.961 INFO:tasks.workunit.client.0.smithi033.stdout:370/600: open fd = 3 2024-04-23T18:36:51.994 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.027 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.061 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:52.094 INFO:tasks.workunit.client.0.smithi033.stdout:371/600: open fd = 3 2024-04-23T18:36:52.152 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.178 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.228 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:52.261 INFO:tasks.workunit.client.0.smithi033.stdout:372/600: open fd = 3 2024-04-23T18:36:52.353 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.386 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.419 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:52.453 INFO:tasks.workunit.client.0.smithi033.stdout:373/600: open fd = 3 2024-04-23T18:36:52.486 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.520 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.570 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:52.630 INFO:tasks.workunit.client.0.smithi033.stdout:374/600: open fd = 3 2024-04-23T18:36:52.670 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.728 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:36:52.762 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:36:52.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:36 smithi033 ceph-mon[29274]: pgmap v318: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:36:52.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:38 smithi033 ceph-mon[29274]: pgmap v319: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:52.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:40 smithi033 ceph-mon[29274]: pgmap v320: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:52.946 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:30 smithi106 ceph-mon[32716]: pgmap v315: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:52.978 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:32 smithi106 ceph-mon[32716]: pgmap v316: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-23T18:36:53.012 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:34 smithi106 ceph-mon[32716]: pgmap v317: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:53.045 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:36 smithi106 ceph-mon[32716]: pgmap v318: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:36:53.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:38 smithi106 ceph-mon[32716]: pgmap v319: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:53.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:40 smithi106 ceph-mon[32716]: pgmap v320: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:53.221 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:42 smithi106 ceph-mon[32716]: pgmap v321: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 20 op/s 2024-04-23T18:36:53.262 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:42 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:53.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: pgmap v322: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:36:53.412 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.462 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.512 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.562 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.662 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:44 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:53.755 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:53.795 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:53.845 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:53.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.012 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:36:54.062 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:36:54.112 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.162 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.212 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.262 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.312 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:36:54.418 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:36:54.463 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.529 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.605 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.638 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:54.788 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:54.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:54.938 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:46 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:36:54.988 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:46 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:36:55.038 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:46 smithi106 ceph-mon[32716]: pgmap v323: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-23T18:36:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:48 smithi106 ceph-mon[32716]: pgmap v324: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:55.166 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:50 smithi106 ceph-mon[32716]: pgmap v325: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:55.213 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:30 smithi122 ceph-mon[32546]: pgmap v315: 97 pgs: 97 active+clean; 7.8 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-23T18:36:55.271 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:32 smithi122 ceph-mon[32546]: pgmap v316: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 KiB/s wr, 22 op/s 2024-04-23T18:36:55.313 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:34 smithi122 ceph-mon[32546]: pgmap v317: 97 pgs: 97 active+clean; 7.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:55.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:36 smithi122 ceph-mon[32546]: pgmap v318: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:36:55.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:38 smithi122 ceph-mon[32546]: pgmap v319: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:55.505 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:40 smithi122 ceph-mon[32546]: pgmap v320: 97 pgs: 97 active+clean; 8.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-23T18:36:55.630 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:42 smithi122 ceph-mon[32546]: pgmap v321: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 20 op/s 2024-04-23T18:36:55.688 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:42 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:36:55.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: pgmap v322: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:36:55.788 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:55.839 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:55.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:55.938 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:55.989 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:56.155 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:44 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:56.205 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.247 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.280 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.364 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:36:56.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:36:56.507 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.572 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.639 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.764 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:36:56.831 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:36:56.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:56.964 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:57.098 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:57.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:36:57.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:36:57.398 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:36:57.464 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:36:57.531 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:46 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:36:57.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:46 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:36:57.681 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:46 smithi122 ceph-mon[32546]: pgmap v323: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-23T18:36:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:48 smithi122 ceph-mon[32546]: pgmap v324: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:57.814 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:50 smithi122 ceph-mon[32546]: pgmap v325: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:57.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:52 smithi122 ceph-mon[32546]: pgmap v326: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:36:57.948 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:53 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/266741717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:58.014 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:54 smithi122 ceph-mon[32546]: pgmap v327: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:58.081 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:56 smithi122 ceph-mon[32546]: pgmap v328: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:36:58.148 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:52 smithi106 ceph-mon[32716]: pgmap v326: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:36:58.223 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:53 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/266741717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:36:58.290 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:54 smithi106 ceph-mon[32716]: pgmap v327: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:36:58.431 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:56 smithi106 ceph-mon[32716]: pgmap v328: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:36:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 -- 172.21.15.33:0/3591869300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103b20 msgr2=0x7f296010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:36:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 --2- 172.21.15.33:0/3591869300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103b20 0x7f296010c000 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f295402a480 comp rx=0 tx=0).stop 2024-04-23T18:36:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 -- 172.21.15.33:0/3591869300 shutdown_connections 2024-04-23T18:36:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 --2- 172.21.15.33:0/3591869300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f296010c5d0 0x7f296010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 --2- 172.21.15.33:0/3591869300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103b20 0x7f296010c000 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 --2- 172.21.15.33:0/3591869300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2960103200 0x7f29601035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:36:58.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 -- 172.21.15.33:0/3591869300 >> 172.21.15.33:0/3591869300 conn(0x7f29600771d0 msgr2=0x7f29600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:36:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 -- 172.21.15.33:0/3591869300 shutdown_connections 2024-04-23T18:36:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.555+0000 7f29662cb640 1 -- 172.21.15.33:0/3591869300 wait complete. 2024-04-23T18:36:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 Processor -- start 2024-04-23T18:36:59.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 -- start start 2024-04-23T18:36:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2960103b20 0x7f2960198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296010c5d0 0x7f296019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:36:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2960112210 con 0x7f2960103200 2024-04-23T18:36:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2960112090 con 0x7f2960103b20 2024-04-23T18:36:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.556+0000 7f29662cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2960112390 con 0x7f296010c5d0 2024-04-23T18:36:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:36:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49346/0 (socket says 172.21.15.33:49346) 2024-04-23T18:36:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 -- 172.21.15.33:0/2676070554 learned_addr learned my addr 172.21.15.33:0/2676070554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f2964868640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296010c5d0 0x7f296019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295f7fe640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2960103b20 0x7f2960198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:00.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 -- 172.21.15.33:0/2676070554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296010c5d0 msgr2=0x7f296019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296010c5d0 0x7f296019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 -- 172.21.15.33:0/2676070554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2960103b20 msgr2=0x7f2960198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2960103b20 0x7f2960198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:00.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 -- 172.21.15.33:0/2676070554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296019d700 con 0x7f2960103200 2024-04-23T18:37:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295ffff640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f295000e9f0 tx=0x7f295000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:00.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295d7fa640 1 -- 172.21.15.33:0/2676070554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295000cd80 con 0x7f2960103200 2024-04-23T18:37:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295d7fa640 1 -- 172.21.15.33:0/2676070554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29500054d0 con 0x7f2960103200 2024-04-23T18:37:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296019d9f0 con 0x7f2960103200 2024-04-23T18:37:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f295d7fa640 1 -- 172.21.15.33:0/2676070554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2950010690 con 0x7f2960103200 2024-04-23T18:37:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.557+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601a5280 con 0x7f2960103200 2024-04-23T18:37:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f295d7fa640 1 -- 172.21.15.33:0/2676070554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2950004e90 con 0x7f2960103200 2024-04-23T18:37:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 Processor -- start 2024-04-23T18:37:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 -- start start 2024-04-23T18:37:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dc8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f295d7fa640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 0x7f29480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fa2d2d970 con 0x559fa2dc8f40 2024-04-23T18:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559fa2cca4a0 con 0x559fa2dc8620 2024-04-23T18:37:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.560+0000 7f2968fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559fa2d29930 con 0x559fa2dd3800 2024-04-23T18:37:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295ffff640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 0x7f29480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49350/0 (socket says 172.21.15.33:49350) 2024-04-23T18:37:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 -- 172.21.15.33:0/2980419915 learned_addr learned my addr 172.21.15.33:0/2980419915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:02.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295d7fa640 1 -- 172.21.15.33:0/2676070554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f29500d0c90 con 0x7f2960103200 2024-04-23T18:37:02.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f2964868640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f2964868640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dc8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:03.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 -- 172.21.15.33:0/2980419915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 msgr2=0x559fa2dd5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:03.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 -- 172.21.15.33:0/2980419915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 msgr2=0x559fa2dc8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:03.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dc8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 -- 172.21.15.33:0/2980419915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa2a40210 con 0x559fa2dc8f40 2024-04-23T18:37:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f2964868640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295ffff640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 0x7f29480a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f295009cbd0 tx=0x7f295006e740 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f295f7fe640 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f2954032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698473b9c7831499 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:03.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f29467fc640 1 -- 172.21.15.33:0/2980419915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295403f070 con 0x559fa2dc8f40 2024-04-23T18:37:03.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f29467fc640 1 -- 172.21.15.33:0/2980419915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2954032e40 con 0x559fa2dc8f40 2024-04-23T18:37:04.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.561+0000 7f29467fc640 1 -- 172.21.15.33:0/2980419915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954035070 con 0x559fa2dc8f40 2024-04-23T18:37:04.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- 172.21.15.33:0/2980419915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 msgr2=0x559fa2dd32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:04.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2954009900 tx=0x7f2954032b90 comp rx=0 tx=0).stop 2024-04-23T18:37:04.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- 172.21.15.33:0/2980419915 shutdown_connections 2024-04-23T18:37:04.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:04.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8f40 0x559fa2dd32c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 --2- 172.21.15.33:0/2980419915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dc8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- 172.21.15.33:0/2980419915 >> 172.21.15.33:0/2980419915 conn(0x559fa2dc4b30 msgr2=0x559fa2dc4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:04.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- 172.21.15.33:0/2980419915 shutdown_connections 2024-04-23T18:37:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- 172.21.15.33:0/2980419915 wait complete. 2024-04-23T18:37:04.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.562+0000 7f2968fd6740 1 -- start start 2024-04-23T18:37:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 Processor -- start 2024-04-23T18:37:05.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dd3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8f40 0x559fa2dd0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:05.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559fa2dd63d0 con 0x559fa2dd3800 2024-04-23T18:37:05.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559fa2cc80b0 con 0x559fa2dc8620 2024-04-23T18:37:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559fa2dd6550 con 0x559fa2dc8f40 2024-04-23T18:37:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2968fd6740 10 client.0 _ll_register_callbacks cb 0x559fa2cb5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49360/0 (socket says 172.21.15.33:49360) 2024-04-23T18:37:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2964868640 1 -- 172.21.15.33:0/752472053 learned_addr learned my addr 172.21.15.33:0/752472053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f295f7fe640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8f40 0x559fa2dd0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:06.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.563+0000 7f2964868640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dd3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 -- 172.21.15.33:0/752472053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8f40 msgr2=0x559fa2dd0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8f40 0x559fa2dd0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 -- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 msgr2=0x559fa2dd3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dd3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 -- 172.21.15.33:0/752472053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa2b0c620 con 0x559fa2dd3800 2024-04-23T18:37:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2964868640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2958011e40 tx=0x7f2958009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e953ed31689882 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f293f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:37:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958011110 con 0x559fa2dd3800 2024-04-23T18:37:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f2958009040 con 0x559fa2dd3800 2024-04-23T18:37:07.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fa2d3b390 con 0x559fa2dd3800 2024-04-23T18:37:07.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f29580112b0 con 0x559fa2dd3800 2024-04-23T18:37:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2968fd6740 10 client.16893 fetch_fsmap learned FSMap version 35 2024-04-23T18:37:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fa2de23a0 con 0x559fa2dd3800 2024-04-23T18:37:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.564+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559fa2d3b390 con 0x559fa2dd3800 2024-04-23T18:37:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f295801d030 con 0x559fa2dd3800 2024-04-23T18:37:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 10 client.16893 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:37:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 20 client.16893 populate_metadata read hostname 'smithi033' 2024-04-23T18:37:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2de3810 0x559fa2de5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 4 client.16893 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa2d35200 con 0x559fa2de3810 2024-04-23T18:37:08.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f295f7fe640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2de3810 0x559fa2de5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f295f7fe640 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2de3810 0x559fa2de5c00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=578b22be8100e6f6 server_cookie=d16e1ce7e2ac3a in_seq=0 out_seq=0 2024-04-23T18:37:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.565+0000 7f293f7fe640 10 client.16893 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f293f7fe640 1 -- 172.21.15.33:0/752472053 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559fa2d35200 con 0x559fa2de3810 2024-04-23T18:37:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f293f7fe640 10 client.16893 handle_command_reply: tid=0 2024-04-23T18:37:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 client.16893 shutdown 2024-04-23T18:37:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 msgr2=0x559fa2dd09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2958011e40 tx=0x7f2958009df0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 shutdown_connections 2024-04-23T18:37:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2de3810 0x559fa2de5c00 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2dd09c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8f40 0x559fa2dd0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f293f7fe640 0 client.16893 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 --2- 172.21.15.33:0/752472053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8620 0x559fa2dd3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 >> 172.21.15.33:0/752472053 conn(0x559fa2dc4b30 msgr2=0x559fa2dca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.566+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 shutdown_connections 2024-04-23T18:37:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.567+0000 7f2968fd6740 1 -- 172.21.15.33:0/752472053 wait complete. 2024-04-23T18:37:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.567+0000 7f2968fd6740 20 client.16893 trim_cache size 0 max 16384 2024-04-23T18:37:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 Processor -- start 2024-04-23T18:37:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 -- start start 2024-04-23T18:37:10.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dc3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29601adb00 con 0x559fa2dc8620 2024-04-23T18:37:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29601af640 con 0x559fa2dc8f40 2024-04-23T18:37:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2968fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29601ae1d0 con 0x559fa2dd3800 2024-04-23T18:37:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49362/0 (socket says 172.21.15.33:49362) 2024-04-23T18:37:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 -- 172.21.15.33:0/4076926520 learned_addr learned my addr 172.21.15.33:0/4076926520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:10.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 -- 172.21.15.33:0/4076926520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 msgr2=0x559fa2dd2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:10.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 -- 172.21.15.33:0/4076926520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 msgr2=0x559fa2dc3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f295f7fe640 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dc3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dc3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:11.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 -- 172.21.15.33:0/4076926520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa2dc8d70 con 0x559fa2dc8620 2024-04-23T18:37:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2964868640 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f29580116b0 tx=0x7f295800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918abc14105b3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.582+0000 7f2945ffb640 1 -- 172.21.15.33:0/4076926520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295802a980 con 0x559fa2dc8620 2024-04-23T18:37:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2945ffb640 1 -- 172.21.15.33:0/4076926520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f295802f070 con 0x559fa2dc8620 2024-04-23T18:37:11.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2945ffb640 1 -- 172.21.15.33:0/4076926520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29580056d0 con 0x559fa2dc8620 2024-04-23T18:37:11.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- 172.21.15.33:0/4076926520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 msgr2=0x559fa2ddf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:12.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f29580116b0 tx=0x7f295800fc20 comp rx=0 tx=0).stop 2024-04-23T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- 172.21.15.33:0/4076926520 shutdown_connections 2024-04-23T18:37:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dd3800 0x559fa2dd2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dc3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 --2- 172.21.15.33:0/4076926520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dc8620 0x559fa2ddf290 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- 172.21.15.33:0/4076926520 >> 172.21.15.33:0/4076926520 conn(0x559fa2dc4b30 msgr2=0x559fa2dc3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- 172.21.15.33:0/4076926520 shutdown_connections 2024-04-23T18:37:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- 172.21.15.33:0/4076926520 wait complete. 2024-04-23T18:37:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.583+0000 7f2968fd6740 1 -- start start 2024-04-23T18:37:12.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 Processor -- start 2024-04-23T18:37:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8620 0x559fa2dd6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dd6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f29601aef70 con 0x559fa2dd3800 2024-04-23T18:37:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f29601ae8a0 con 0x559fa2dc8f40 2024-04-23T18:37:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29601abae0 con 0x559fa2dc8620 2024-04-23T18:37:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2968fd6740 10 client.0 _ll_register_callbacks cb 0x7f2958011b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:13.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2964868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49372/0 (socket says 172.21.15.33:49372) 2024-04-23T18:37:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2964868640 1 -- 172.21.15.33:0/855594076 learned_addr learned my addr 172.21.15.33:0/855594076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f295f7fe640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dd6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:13.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.584+0000 7f2964868640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8620 0x559fa2dd6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 -- 172.21.15.33:0/855594076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8620 msgr2=0x559fa2dd6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8620 0x559fa2dd6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 -- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 msgr2=0x559fa2dd6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dd6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 -- 172.21.15.33:0/855594076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa2e4c750 con 0x559fa2dd3800 2024-04-23T18:37:14.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2964868640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2958005370 tx=0x7f2958011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100fd871c0573915 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f293f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:37:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295802a980 con 0x559fa2dd3800 2024-04-23T18:37:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f2958005470 con 0x559fa2dd3800 2024-04-23T18:37:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f295802f070 con 0x559fa2dd3800 2024-04-23T18:37:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2958005670 con 0x559fa2dd3800 2024-04-23T18:37:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2968fd6740 10 client.16899 fetch_fsmap learned FSMap version 35 2024-04-23T18:37:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559fa2e48400 con 0x559fa2dd3800 2024-04-23T18:37:15.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.585+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f295802f070 con 0x559fa2dd3800 2024-04-23T18:37:15.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f295801d070 con 0x559fa2dd3800 2024-04-23T18:37:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 10 client.16899 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:37:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 10 client.16899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 20 client.16899 populate_metadata read hostname 'smithi033' 2024-04-23T18:37:15.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2e48b50 0x559fa2e4af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 4 client.16899 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fa2cb15f0 con 0x559fa2e48b50 2024-04-23T18:37:15.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f295f7fe640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2e48b50 0x559fa2e4af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:16.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f295f7fe640 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2e48b50 0x559fa2e4af70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f9332137350bd0 server_cookie=7edb11ad4560af05 in_seq=0 out_seq=0 2024-04-23T18:37:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.586+0000 7f293f7fe640 10 client.16899 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f293f7fe640 1 -- 172.21.15.33:0/855594076 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fa2cb15f0 con 0x559fa2e48b50 2024-04-23T18:37:16.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f293f7fe640 10 client.16899 handle_command_reply: tid=0 2024-04-23T18:37:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 client.16899 shutdown 2024-04-23T18:37:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 msgr2=0x559fa2e47d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:16.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2958005370 tx=0x7f2958011a80 comp rx=0 tx=0).stop 2024-04-23T18:37:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 shutdown_connections 2024-04-23T18:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559fa2e48b50 0x559fa2e4af70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559fa2dd3800 0x559fa2e47d50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f293f7fe640 0 client.16899 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559fa2dc8f40 0x559fa2dd6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:17.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 --2- 172.21.15.33:0/855594076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559fa2dc8620 0x559fa2dd6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 >> 172.21.15.33:0/855594076 conn(0x559fa2dc4b30 msgr2=0x559fa2dce940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 shutdown_connections 2024-04-23T18:37:17.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 1 -- 172.21.15.33:0/855594076 wait complete. 2024-04-23T18:37:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.594+0000 7f2968fd6740 20 client.16899 trim_cache size 0 max 16384 2024-04-23T18:37:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:37:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:37:17.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:37:18.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:37:18.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 msgr2=0x7f29480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:18.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 0x7f29480a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f295009cbd0 tx=0x7f295006e740 comp rx=0 tx=0).stop 2024-04-23T18:37:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 msgr2=0x7f2960198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f295000e9f0 tx=0x7f295000edf0 comp rx=0 tx=0).stop 2024-04-23T18:37:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 shutdown_connections 2024-04-23T18:37:18.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f29480a3dc0 0x7f29480a6280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296010c5d0 0x7f296019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:18.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2960103b20 0x7f2960198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 --2- 172.21.15.33:0/2676070554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2960103200 0x7f2960198710 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 >> 172.21.15.33:0/2676070554 conn(0x7f29600771d0 msgr2=0x7f29600fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 shutdown_connections 2024-04-23T18:37:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:42.595+0000 7f29662cb640 1 -- 172.21.15.33:0/2676070554 wait complete. 2024-04-23T18:37:19.213 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:42 smithi033 ceph-mon[29274]: pgmap v321: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 20 op/s 2024-04-23T18:37:19.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:42 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:19.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: pgmap v322: 97 pgs: 97 active+clean; 8.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:19.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.729 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.887 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:44 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:19.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.037 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:37:20.412 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:37:20.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.612 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.729 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:20.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:37:21.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:37:21.171 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:21.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:21.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:21.523 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:37:21.621 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:21.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:21.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:21.896 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:46 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:37:23.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:46 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:37:25.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:46 smithi033 ceph-mon[29274]: pgmap v323: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 20 op/s 2024-04-23T18:37:25.414 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:48 smithi033 ceph-mon[29274]: pgmap v324: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:37:25.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:50 smithi033 ceph-mon[29274]: pgmap v325: 97 pgs: 97 active+clean; 8.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:37:25.553 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:36:58 smithi106 ceph-mon[32716]: pgmap v329: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:25.638 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:00 smithi106 ceph-mon[32716]: pgmap v330: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:25.705 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:02 smithi106 ceph-mon[32716]: pgmap v331: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:37:25.805 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:04 smithi106 ceph-mon[32716]: pgmap v332: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-04-23T18:37:25.863 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:06 smithi106 ceph-mon[32716]: pgmap v333: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:37:25.947 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:08 smithi106 ceph-mon[32716]: pgmap v334: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 71 KiB/s wr, 31 op/s 2024-04-23T18:37:25.980 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:09 smithi106 ceph-mon[32716]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:37:26.072 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:10 smithi106 ceph-mon[32716]: pgmap v336: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 38 op/s 2024-04-23T18:37:26.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:12 smithi106 ceph-mon[32716]: pgmap v337: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:37:26.313 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:14 smithi106 ceph-mon[32716]: pgmap v338: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:37:26.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:16 smithi106 ceph-mon[32716]: pgmap v339: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:37:26.422 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:16 smithi106 ceph-mon[32716]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:37:26.472 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:18 smithi106 ceph-mon[32716]: pgmap v341: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:26.530 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:20 smithi106 ceph-mon[32716]: pgmap v342: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:37:26.580 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:22 smithi106 ceph-mon[32716]: pgmap v343: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:37:26.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:24 smithi106 ceph-mon[32716]: pgmap v344: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:37:26.681 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:36:58 smithi122 ceph-mon[32546]: pgmap v329: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:26.731 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:00 smithi122 ceph-mon[32546]: pgmap v330: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:26.797 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:02 smithi122 ceph-mon[32546]: pgmap v331: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:37:26.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:04 smithi122 ceph-mon[32546]: pgmap v332: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-04-23T18:37:26.997 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:06 smithi122 ceph-mon[32546]: pgmap v333: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:37:27.064 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:08 smithi122 ceph-mon[32546]: pgmap v334: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 71 KiB/s wr, 31 op/s 2024-04-23T18:37:27.123 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:09 smithi122 ceph-mon[32546]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:37:27.164 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:10 smithi122 ceph-mon[32546]: pgmap v336: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 38 op/s 2024-04-23T18:37:27.214 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:12 smithi122 ceph-mon[32546]: pgmap v337: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:37:27.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:14 smithi122 ceph-mon[32546]: pgmap v338: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:37:27.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:16 smithi122 ceph-mon[32546]: pgmap v339: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:37:27.364 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:16 smithi122 ceph-mon[32546]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:37:27.414 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:18 smithi122 ceph-mon[32546]: pgmap v341: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:27.473 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:20 smithi122 ceph-mon[32546]: pgmap v342: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:37:27.564 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:22 smithi122 ceph-mon[32546]: pgmap v343: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:37:27.616 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:24 smithi122 ceph-mon[32546]: pgmap v344: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:37:27.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:26 smithi106 ceph-mon[32716]: pgmap v345: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-23T18:37:27.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:26 smithi122 ceph-mon[32546]: pgmap v345: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-23T18:37:27.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 -- 172.21.15.33:0/2568690502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4106190 msgr2=0x7fc2a410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 --2- 172.21.15.33:0/2568690502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4106190 0x7fc2a410dd60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009ca0 tx=0x7fc29c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:37:27.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 -- 172.21.15.33:0/2568690502 shutdown_connections 2024-04-23T18:37:27.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 --2- 172.21.15.33:0/2568690502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4106190 0x7fc2a410dd60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 --2- 172.21.15.33:0/2568690502 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a4101ac0 0x7fc2a4105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 --2- 172.21.15.33:0/2568690502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4101110 0x7fc2a41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 -- 172.21.15.33:0/2568690502 >> 172.21.15.33:0/2568690502 conn(0x7fc2a4077fb0 msgr2=0x7fc2a40783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:28.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 -- 172.21.15.33:0/2568690502 shutdown_connections 2024-04-23T18:37:28.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.970+0000 7fc2a8819640 1 -- 172.21.15.33:0/2568690502 wait complete. 2024-04-23T18:37:28.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 Processor -- start 2024-04-23T18:37:28.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 -- start start 2024-04-23T18:37:28.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:28.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4106190 0x7fc2a410ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:28.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a410f170 0x7fc2a41a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:28.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a4112240 con 0x7fc2a4101110 2024-04-23T18:37:28.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41120c0 con 0x7fc2a410f170 2024-04-23T18:37:28.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a8819640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a41123c0 con 0x7fc2a4106190 2024-04-23T18:37:28.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:29.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53158/0 (socket says 172.21.15.33:53158) 2024-04-23T18:37:29.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.971+0000 7fc2a259c640 1 -- 172.21.15.33:0/266741717 learned_addr learned my addr 172.21.15.33:0/266741717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a2d9d640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a410f170 0x7fc2a41a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc29bfff640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4106190 0x7fc2a410ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:29.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 -- 172.21.15.33:0/266741717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4106190 msgr2=0x7fc2a410ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:29.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4106190 0x7fc2a410ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:29.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 -- 172.21.15.33:0/266741717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a410f170 msgr2=0x7fc2a41a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a410f170 0x7fc2a41a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:29.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 -- 172.21.15.33:0/266741717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a41a6f90 con 0x7fc2a4101110 2024-04-23T18:37:29.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a259c640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc29400d950 tx=0x7fc29400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294017030 con 0x7fc2a4101110 2024-04-23T18:37:29.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc294004020 con 0x7fc2a4101110 2024-04-23T18:37:29.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a41a7280 con 0x7fc2a4101110 2024-04-23T18:37:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2940047a0 con 0x7fc2a4101110 2024-04-23T18:37:30.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.972+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a41a77d0 con 0x7fc2a4101110 2024-04-23T18:37:30.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.973+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a40691c0 con 0x7fc2a4101110 2024-04-23T18:37:30.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.974+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc294009a30 con 0x7fc2a4101110 2024-04-23T18:37:30.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.975+0000 7fc29b7fe640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 0x7fc2900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:30.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.975+0000 7fc29bfff640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 0x7fc2900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.975+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fc2940d01c0 con 0x7fc2a4101110 2024-04-23T18:37:30.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.976+0000 7fc29bfff640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 0x7fc2900a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4110130 tx=0x7fc28c00b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:30.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:52.980+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc29409c870 con 0x7fc2a4101110 2024-04-23T18:37:30.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.205+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2a4109420 con 0x7fc2a4101110 2024-04-23T18:37:30.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.205+0000 7fc29b7fe640 1 -- 172.21.15.33:0/266741717 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11929 (secure 0 0 0) 0x7fc29409c170 con 0x7fc2a4101110 2024-04-23T18:37:30.849 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:37:30.899 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/3122213420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":3122213420},{"type":"v1","addr":"172.21.15.33:6837","nonce":3122213420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:36:15.776069+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26299":{"gid":26299,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6835/2772764019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2772764019},{"type":"v1","addr":"172.21.15.122:6835","nonce":2772764019}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:37:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-23T18:37:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 msgr2=0x7fc2900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 0x7fc2900a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4110130 tx=0x7fc28c00b390 comp rx=0 tx=0).stop 2024-04-23T18:37:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 msgr2=0x7fc2a410e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:31.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc29400d950 tx=0x7fc29400de10 comp rx=0 tx=0).stop 2024-04-23T18:37:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 shutdown_connections 2024-04-23T18:37:31.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc2900a3d20 0x7fc2900a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:31.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2a410f170 0x7fc2a41a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2a4106190 0x7fc2a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:31.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 --2- 172.21.15.33:0/266741717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a4101110 0x7fc2a410e6f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:31.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 >> 172.21.15.33:0/266741717 conn(0x7fc2a4077fb0 msgr2=0x7fc2a40feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:31.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 shutdown_connections 2024-04-23T18:37:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:36:53.207+0000 7fc2a8819640 1 -- 172.21.15.33:0/266741717 wait complete. 2024-04-23T18:37:31.866 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:52 smithi033 ceph-mon[29274]: pgmap v326: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 20 op/s 2024-04-23T18:37:31.910 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0' 2024-04-23T18:37:31.966 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db / --timeout 60 --expiration 82.9 --await-for 42.0 2024-04-23T18:37:32.034 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:28 smithi106 ceph-mon[32716]: pgmap v346: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:32.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:30 smithi106 ceph-mon[32716]: pgmap v347: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:32.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:28 smithi122 ceph-mon[32546]: pgmap v346: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:32.274 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:30 smithi122 ceph-mon[32546]: pgmap v347: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:32.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:53 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/266741717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:37:32.375 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:54 smithi033 ceph-mon[29274]: pgmap v327: 97 pgs: 97 active+clean; 8.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 14 op/s 2024-04-23T18:37:32.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 -- 172.21.15.33:0/3792262701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c1021d0 msgr2=0x7f278c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:32.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 --2- 172.21.15.33:0/3792262701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c1021d0 0x7f278c1025b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f277c009900 tx=0x7f277c02a410 comp rx=0 tx=0).stop 2024-04-23T18:37:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 -- 172.21.15.33:0/3792262701 shutdown_connections 2024-04-23T18:37:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 --2- 172.21.15.33:0/3792262701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1072a0 0x7f278c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 --2- 172.21.15.33:0/3792262701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c102b80 0x7f278c106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 --2- 172.21.15.33:0/3792262701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c1021d0 0x7f278c1025b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:32.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 -- 172.21.15.33:0/3792262701 >> 172.21.15.33:0/3792262701 conn(0x7f278c0793e0 msgr2=0x7f278c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 -- 172.21.15.33:0/3792262701 shutdown_connections 2024-04-23T18:37:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.694+0000 7f279094a640 1 -- 172.21.15.33:0/3792262701 wait complete. 2024-04-23T18:37:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 Processor -- start 2024-04-23T18:37:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 -- start start 2024-04-23T18:37:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1021d0 0x7f278c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c1072a0 0x7f278c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f278c112250 con 0x7f278c19cf80 2024-04-23T18:37:33.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1120d0 con 0x7f278c1021d0 2024-04-23T18:37:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.695+0000 7f279094a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1123d0 con 0x7f278c1072a0 2024-04-23T18:37:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53178/0 (socket says 172.21.15.33:53178) 2024-04-23T18:37:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 -- 172.21.15.33:0/3753961845 learned_addr learned my addr 172.21.15.33:0/3753961845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f2783fff640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c1072a0 0x7f278c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278a59c640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1021d0 0x7f278c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 -- 172.21.15.33:0/3753961845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c1072a0 msgr2=0x7f278c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c1072a0 0x7f278c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 -- 172.21.15.33:0/3753961845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1021d0 msgr2=0x7f278c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1021d0 0x7f278c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 -- 172.21.15.33:0/3753961845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c19da70 con 0x7f278c19cf80 2024-04-23T18:37:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f278ad9d640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f278400e960 tx=0x7f278400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f27837fe640 1 -- 172.21.15.33:0/3753961845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784004d20 con 0x7f278c19cf80 2024-04-23T18:37:34.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f27837fe640 1 -- 172.21.15.33:0/3753961845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2784005020 con 0x7f278c19cf80 2024-04-23T18:37:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c19dd60 con 0x7f278c19cf80 2024-04-23T18:37:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.696+0000 7f27837fe640 1 -- 172.21.15.33:0/3753961845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27840116d0 con 0x7f278c19cf80 2024-04-23T18:37:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.697+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1a5500 con 0x7f278c19cf80 2024-04-23T18:37:34.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.699+0000 7f27837fe640 1 -- 172.21.15.33:0/3753961845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27840051d0 con 0x7f278c19cf80 2024-04-23T18:37:34.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.699+0000 7f2793659740 1 Processor -- start 2024-04-23T18:37:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.699+0000 7f2793659740 1 -- start start 2024-04-23T18:37:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd756652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75665800 0x55cd75667bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd755bf970 con 0x55cd7565a620 2024-04-23T18:37:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cd7555c4a0 con 0x55cd75665800 2024-04-23T18:37:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2793659740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd755bb930 con 0x55cd7565af40 2024-04-23T18:37:34.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f27837fe640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 0x7f27780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53188/0 (socket says 172.21.15.33:53188) 2024-04-23T18:37:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 -- 172.21.15.33:0/3812914957 learned_addr learned my addr 172.21.15.33:0/3812914957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f278ad9d640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 0x7f27780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f278a59c640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd756652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75665800 0x55cd75667bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:35.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f27837fe640 1 -- 172.21.15.33:0/3753961845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f27840d0850 con 0x7f278c19cf80 2024-04-23T18:37:35.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 -- 172.21.15.33:0/3812914957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 msgr2=0x55cd756652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd756652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 -- 172.21.15.33:0/3812914957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75665800 msgr2=0x55cd75667bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75665800 0x55cd75667bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f2783fff640 1 -- 172.21.15.33:0/3812914957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd752d2210 con 0x55cd7565a620 2024-04-23T18:37:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.700+0000 7f278a59c640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd756652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:37:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2783fff640 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f277400bdd0 tx=0x7f2774004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feb08ae3f6292ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f278ad9d640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 0x7f27780a6280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27840ce060 tx=0x7f278406c190 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2780ff9640 1 -- 172.21.15.33:0/3812914957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774003e80 con 0x55cd7565a620 2024-04-23T18:37:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2780ff9640 1 -- 172.21.15.33:0/3812914957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27740054d0 con 0x55cd7565a620 2024-04-23T18:37:36.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2780ff9640 1 -- 172.21.15.33:0/3812914957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774004e40 con 0x55cd7565a620 2024-04-23T18:37:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 -- 172.21.15.33:0/3812914957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 msgr2=0x55cd7565aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f277400bdd0 tx=0x7f2774004610 comp rx=0 tx=0).stop 2024-04-23T18:37:36.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 -- 172.21.15.33:0/3812914957 shutdown_connections 2024-04-23T18:37:36.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75665800 0x55cd75667bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd756652c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 --2- 172.21.15.33:0/3812914957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7565aa00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.701+0000 7f2793659740 1 -- 172.21.15.33:0/3812914957 >> 172.21.15.33:0/3812914957 conn(0x55cd75656b30 msgr2=0x55cd75656f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- 172.21.15.33:0/3812914957 shutdown_connections 2024-04-23T18:37:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- 172.21.15.33:0/3812914957 wait complete. 2024-04-23T18:37:37.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- start start 2024-04-23T18:37:37.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 Processor -- start 2024-04-23T18:37:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565a620 0x55cd756650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd756623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:37.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cd756683d0 con 0x55cd75665800 2024-04-23T18:37:37.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cd7555a0b0 con 0x55cd7565af40 2024-04-23T18:37:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd75668550 con 0x55cd7565a620 2024-04-23T18:37:37.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2793659740 10 client.0 _ll_register_callbacks cb 0x55cd75547d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.702+0000 7f2783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53190/0 (socket says 172.21.15.33:53190) 2024-04-23T18:37:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 -- 172.21.15.33:0/4148565972 learned_addr learned my addr 172.21.15.33:0/4148565972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:37.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565a620 0x55cd756650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f278a59c640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd756623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:37.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 -- 172.21.15.33:0/4148565972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565a620 msgr2=0x55cd756650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:38.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565a620 0x55cd756650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 -- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 msgr2=0x55cd756623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd756623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 -- 172.21.15.33:0/4148565972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7539e620 con 0x55cd75665800 2024-04-23T18:37:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2783fff640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f277400b3f0 tx=0x7f277400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401d580e8c8c280e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740048d0 con 0x55cd75665800 2024-04-23T18:37:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f2774004fc0 con 0x55cd75665800 2024-04-23T18:37:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27740054d0 con 0x55cd75665800 2024-04-23T18:37:38.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f27740051c0 con 0x55cd75665800 2024-04-23T18:37:38.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2793659740 10 client.16917 fetch_fsmap learned FSMap version 35 2024-04-23T18:37:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.703+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd75674260 con 0x55cd75665800 2024-04-23T18:37:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27740054d0 con 0x55cd75665800 2024-04-23T18:37:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f2774010c90 con 0x55cd75665800 2024-04-23T18:37:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 10 client.16917 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:37:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 10 client.16917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 20 client.16917 populate_metadata read hostname 'smithi033' 2024-04-23T18:37:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd756755b0 0x55cd756779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 4 client.16917 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:37:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd755cd390 con 0x55cd756755b0 2024-04-23T18:37:39.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.704+0000 7f278a59c640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd756755b0 0x55cd756779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f278a59c640 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd756755b0 0x55cd756779a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc14387d36c35ae4 server_cookie=fc3e93edefcd61ac in_seq=0 out_seq=0 2024-04-23T18:37:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2771ffb640 10 client.16917 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2771ffb640 1 -- 172.21.15.33:0/4148565972 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd755cd390 con 0x55cd756755b0 2024-04-23T18:37:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2771ffb640 10 client.16917 handle_command_reply: tid=0 2024-04-23T18:37:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2793659740 1 client.16917 shutdown 2024-04-23T18:37:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 msgr2=0x55cd756628f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f277400b3f0 tx=0x7f277400b5b0 comp rx=0 tx=0).stop 2024-04-23T18:37:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 shutdown_connections 2024-04-23T18:37:39.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.705+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd756755b0 0x55cd756779a0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:39.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd75665800 0x55cd756628f0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:39.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd756623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2771ffb640 0 client.16917 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 --2- 172.21.15.33:0/4148565972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565a620 0x55cd756650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:40.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 >> 172.21.15.33:0/4148565972 conn(0x55cd75656b30 msgr2=0x55cd7565c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 shutdown_connections 2024-04-23T18:37:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 1 -- 172.21.15.33:0/4148565972 wait complete. 2024-04-23T18:37:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.706+0000 7f2793659740 20 client.16917 trim_cache size 0 max 16384 2024-04-23T18:37:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 Processor -- start 2024-04-23T18:37:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 -- start start 2024-04-23T18:37:40.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd75656b30 0x55cd75661cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd7567d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1adaf0 con 0x55cd7565a620 2024-04-23T18:37:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1af630 con 0x55cd7565af40 2024-04-23T18:37:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2793659740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1ae1c0 con 0x55cd75656b30 2024-04-23T18:37:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f278a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f278a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53204/0 (socket says 172.21.15.33:53204) 2024-04-23T18:37:41.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f278a59c640 1 -- 172.21.15.33:0/1403585067 learned_addr learned my addr 172.21.15.33:0/1403585067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.721+0000 7f2783fff640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd75656b30 0x55cd75661cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f2783fff640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd7567d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 -- 172.21.15.33:0/1403585067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd75656b30 msgr2=0x55cd75661cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd75656b30 0x55cd75661cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 -- 172.21.15.33:0/1403585067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 msgr2=0x55cd7567d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:41.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd7567d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:41.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 -- 172.21.15.33:0/1403585067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd7565ad70 con 0x55cd7565a620 2024-04-23T18:37:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f2783fff640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd7567d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:37:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f278a59c640 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f277c005ab0 tx=0x7f277c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e21eaf0f1089368 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f2773fff640 1 -- 172.21.15.33:0/1403585067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c035040 con 0x55cd7565a620 2024-04-23T18:37:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f2773fff640 1 -- 172.21.15.33:0/1403585067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c046bc0 con 0x55cd7565a620 2024-04-23T18:37:42.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.722+0000 7f2773fff640 1 -- 172.21.15.33:0/1403585067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0468e0 con 0x55cd7565a620 2024-04-23T18:37:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- 172.21.15.33:0/1403585067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 msgr2=0x55cd7567d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f277c005ab0 tx=0x7f277c005a20 comp rx=0 tx=0).stop 2024-04-23T18:37:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- 172.21.15.33:0/1403585067 shutdown_connections 2024-04-23T18:37:42.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd7565af40 0x55cd7567d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7567d180 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 --2- 172.21.15.33:0/1403585067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd75656b30 0x55cd75661cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- 172.21.15.33:0/1403585067 >> 172.21.15.33:0/1403585067 conn(0x55cd75662a50 msgr2=0x55cd75670280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- 172.21.15.33:0/1403585067 shutdown_connections 2024-04-23T18:37:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- 172.21.15.33:0/1403585067 wait complete. 2024-04-23T18:37:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 -- start start 2024-04-23T18:37:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.723+0000 7f2793659740 1 Processor -- start 2024-04-23T18:37:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75656b30 0x55cd7566d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd7566a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1aef60 con 0x55cd7565a620 2024-04-23T18:37:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1ae890 con 0x55cd75656b30 2024-04-23T18:37:43.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1abad0 con 0x55cd7565af40 2024-04-23T18:37:43.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 10 client.0 _ll_register_callbacks cb 0x7f2774004e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:37:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:44.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2783fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75656b30 0x55cd7566d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53206/0 (socket says 172.21.15.33:53206) 2024-04-23T18:37:44.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 -- 172.21.15.33:0/1386614 learned_addr learned my addr 172.21.15.33:0/1386614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2783fff640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd7566a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:44.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 -- 172.21.15.33:0/1386614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 msgr2=0x55cd7566a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:44.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd7566a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:44.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 -- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75656b30 msgr2=0x55cd7566d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75656b30 0x55cd7566d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 -- 172.21.15.33:0/1386614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd7566b010 con 0x55cd7565a620 2024-04-23T18:37:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2783fff640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd7566a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f278a59c640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f277c031440 tx=0x7f277c004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9360165b4539abaa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:45.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:37:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c053070 con 0x55cd7565a620 2024-04-23T18:37:45.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7f277c046860 con 0x55cd7565a620 2024-04-23T18:37:45.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.724+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c046bc0 con 0x55cd7565a620 2024-04-23T18:37:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f277c035040 con 0x55cd7565a620 2024-04-23T18:37:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 10 client.16923 fetch_fsmap learned FSMap version 35 2024-04-23T18:37:45.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55cd756db3b0 con 0x55cd7565a620 2024-04-23T18:37:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c046bc0 con 0x55cd7565a620 2024-04-23T18:37:46.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f277c0035f0 con 0x55cd7565a620 2024-04-23T18:37:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 10 client.16923 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:37:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:37:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 20 client.16923 populate_metadata read hostname 'smithi033' 2024-04-23T18:37:46.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd75665800 0x55cd756ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 4 client.16923 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:37:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd755c7200 con 0x55cd75665800 2024-04-23T18:37:47.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.725+0000 7f2783fff640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd75665800 0x55cd756ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:47.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.726+0000 7f2771ffb640 10 client.16923 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.726+0000 7f2783fff640 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd75665800 0x55cd756ddb50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9101a58bc25f3bf server_cookie=ee1c686f5dab051c in_seq=0 out_seq=0 2024-04-23T18:37:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2771ffb640 1 -- 172.21.15.33:0/1386614 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd755c7200 con 0x55cd75665800 2024-04-23T18:37:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2771ffb640 10 client.16923 handle_command_reply: tid=0 2024-04-23T18:37:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 client.16923 shutdown 2024-04-23T18:37:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 msgr2=0x55cd7566d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f277c031440 tx=0x7f277c004240 comp rx=0 tx=0).stop 2024-04-23T18:37:47.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 shutdown_connections 2024-04-23T18:37:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cd75665800 0x55cd756ddb50 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd7565af40 0x55cd7566a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2771ffb640 0 client.16923 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:37:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cd7565a620 0x55cd7566d910 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 --2- 172.21.15.33:0/1386614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cd75656b30 0x55cd7566d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 >> 172.21.15.33:0/1386614 conn(0x55cd75662a50 msgr2=0x55cd75661530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 shutdown_connections 2024-04-23T18:37:48.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.733+0000 7f2793659740 1 -- 172.21.15.33:0/1386614 wait complete. 2024-04-23T18:37:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f2793659740 20 client.16923 trim_cache size 0 max 16384 2024-04-23T18:37:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:37:49.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:37:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:37:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:37:49.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 msgr2=0x7f27780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:49.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 0x7f27780a6280 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27840ce060 tx=0x7f278406c190 comp rx=0 tx=0).stop 2024-04-23T18:37:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 msgr2=0x7f278c19d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:49.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f278400e960 tx=0x7f278400ee20 comp rx=0 tx=0).stop 2024-04-23T18:37:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 shutdown_connections 2024-04-23T18:37:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f27780a3dc0 0x7f27780a6280 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c19cf80 0x7f278c19d430 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f278c1072a0 0x7f278c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 --2- 172.21.15.33:0/3753961845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f278c1021d0 0x7f278c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 >> 172.21.15.33:0/3753961845 conn(0x7f278c0793e0 msgr2=0x7f278c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 shutdown_connections 2024-04-23T18:37:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:36:55.734+0000 7f279094a640 1 -- 172.21.15.33:0/3753961845 wait complete. 2024-04-23T18:37:50.633 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:56 smithi033 ceph-mon[29274]: pgmap v328: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 21 op/s 2024-04-23T18:37:50.753 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:36:58 smithi033 ceph-mon[29274]: pgmap v329: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:50.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:00 smithi033 ceph-mon[29274]: pgmap v330: 97 pgs: 97 active+clean; 8.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 12 op/s 2024-04-23T18:37:50.897 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:02 smithi033 ceph-mon[29274]: pgmap v331: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-23T18:37:50.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:04 smithi033 ceph-mon[29274]: pgmap v332: 97 pgs: 97 active+clean; 8.6 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 69 KiB/s wr, 31 op/s 2024-04-23T18:37:51.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:06 smithi033 ceph-mon[29274]: pgmap v333: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 82 KiB/s wr, 38 op/s 2024-04-23T18:37:51.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:32 smithi122 ceph-mon[32546]: pgmap v348: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 20 op/s 2024-04-23T18:37:51.264 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:34 smithi122 ceph-mon[32546]: pgmap v349: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:51.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:36 smithi122 ceph-mon[32546]: pgmap v350: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:37:51.406 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:38 smithi122 ceph-mon[32546]: pgmap v351: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:51.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:40 smithi122 ceph-mon[32546]: pgmap v352: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:51.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:42 smithi122 ceph-mon[32546]: pgmap v353: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:51.647 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:44 smithi122 ceph-mon[32546]: pgmap v354: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:51.706 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:45 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:51.773 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: pgmap v355: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T18:37:51.840 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:51.906 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:51.990 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:52.107 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:52.190 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:46 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:52.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:48 smithi122 ceph-mon[32546]: pgmap v356: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:37:52.324 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:32 smithi106 ceph-mon[32716]: pgmap v348: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 20 op/s 2024-04-23T18:37:52.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:34 smithi106 ceph-mon[32716]: pgmap v349: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:37:52.465 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:36 smithi106 ceph-mon[32716]: pgmap v350: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:37:52.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:38 smithi106 ceph-mon[32716]: pgmap v351: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:52.607 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:40 smithi106 ceph-mon[32716]: pgmap v352: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:52.674 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:42 smithi106 ceph-mon[32716]: pgmap v353: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:52.782 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:44 smithi106 ceph-mon[32716]: pgmap v354: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:37:52.849 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:45 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:37:53.055 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: pgmap v355: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T18:37:53.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:53.641 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:53.782 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:37:53.865 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:37:53.953 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:46 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:37:54.032 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:48 smithi106 ceph-mon[32716]: pgmap v356: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:37:54.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:50 smithi106 ceph-mon[32716]: pgmap v357: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:37:54.199 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:52 smithi106 ceph-mon[32716]: pgmap v358: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:54.299 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:50 smithi122 ceph-mon[32546]: pgmap v357: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:37:54.424 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:52 smithi122 ceph-mon[32546]: pgmap v358: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:37:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 -- 172.21.15.33:0/894571960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c102b70 msgr2=0x7fe93c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 --2- 172.21.15.33:0/894571960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c102b70 0x7fe93c106bb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe92802a410 comp rx=0 tx=0).stop 2024-04-23T18:37:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 -- 172.21.15.33:0/894571960 shutdown_connections 2024-04-23T18:37:54.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 --2- 172.21.15.33:0/894571960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c107290 0x7fe93c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:54.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 --2- 172.21.15.33:0/894571960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c102b70 0x7fe93c106bb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 --2- 172.21.15.33:0/894571960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c1021c0 0x7fe93c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 -- 172.21.15.33:0/894571960 >> 172.21.15.33:0/894571960 conn(0x7fe93c0fda20 msgr2=0x7fe93c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:37:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 -- 172.21.15.33:0/894571960 shutdown_connections 2024-04-23T18:37:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.834+0000 7fe942d15640 1 -- 172.21.15.33:0/894571960 wait complete. 2024-04-23T18:37:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 Processor -- start 2024-04-23T18:37:55.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 -- start start 2024-04-23T18:37:55.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c1021c0 0x7fe93c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 0x7fe93c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1121b0 con 0x7fe93c19cf70 2024-04-23T18:37:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c112030 con 0x7fe93c1021c0 2024-04-23T18:37:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe942d15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c112330 con 0x7fe93c107290 2024-04-23T18:37:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:56.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37374/0 (socket says 172.21.15.33:37374) 2024-04-23T18:37:56.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe9412b2640 1 -- 172.21.15.33:0/1135611785 learned_addr learned my addr 172.21.15.33:0/1135611785 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 0x7fe93c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe940ab1640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c1021c0 0x7fe93c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe9412b2640 1 -- 172.21.15.33:0/1135611785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 msgr2=0x7fe93c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.835+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 0x7fe93c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe9412b2640 1 -- 172.21.15.33:0/1135611785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c1021c0 msgr2=0x7fe93c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:37:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c1021c0 0x7fe93c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe9412b2640 1 -- 172.21.15.33:0/1135611785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c19da60 con 0x7fe93c19cf70 2024-04-23T18:37:57.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 0x7fe93c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:57.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe93400eaa0 tx=0x7fe93400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:37:57.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe939ffb640 1 -- 172.21.15.33:0/1135611785 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93400cdb0 con 0x7fe93c19cf70 2024-04-23T18:37:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe939ffb640 1 -- 172.21.15.33:0/1135611785 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe934005020 con 0x7fe93c19cf70 2024-04-23T18:37:57.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe939ffb640 1 -- 172.21.15.33:0/1135611785 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9340116d0 con 0x7fe93c19cf70 2024-04-23T18:37:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c19dd50 con 0x7fe93c19cf70 2024-04-23T18:37:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.836+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1a5510 con 0x7fe93c19cf70 2024-04-23T18:37:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.838+0000 7fe939ffb640 1 -- 172.21.15.33:0/1135611785 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe9340051d0 con 0x7fe93c19cf70 2024-04-23T18:37:57.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.839+0000 7fe945a20740 1 Processor -- start 2024-04-23T18:37:57.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.839+0000 7fe945a20740 1 -- start start 2024-04-23T18:37:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.839+0000 7fe939ffb640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 0x7fe92c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.839+0000 7fe945a20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 0x7fe92c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe945a20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe945a20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b42bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:37:58.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe945a20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f618a9a970 con 0x55f618b35620 2024-04-23T18:37:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe945a20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f618a374a0 con 0x55f618b40800 2024-04-23T18:37:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe945a20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f618a96930 con 0x55f618b35f40 2024-04-23T18:37:58.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe939ffb640 1 -- 172.21.15.33:0/1135611785 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fe9340d0a00 con 0x7fe93c19cf70 2024-04-23T18:37:58.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:58.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37380/0 (socket says 172.21.15.33:37380) 2024-04-23T18:37:58.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 -- 172.21.15.33:0/2278072401 learned_addr learned my addr 172.21.15.33:0/2278072401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:37:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe93bfff640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe9412b2640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b42bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:37:59.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 -- 172.21.15.33:0/2278072401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 msgr2=0x55f618b402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:59.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 -- 172.21.15.33:0/2278072401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 msgr2=0x55f618b42bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:37:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b42bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:37:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 -- 172.21.15.33:0/2278072401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6187ad210 con 0x55f618b35620 2024-04-23T18:37:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe93bfff640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe9412b2640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:37:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 0x7fe92c0a6300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe928000c00 tx=0x7fe928032f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:00.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe93000cbf0 tx=0x7fe93000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e437df3a93568b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.840+0000 7fe926ffd640 1 -- 172.21.15.33:0/2278072401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930010630 con 0x55f618b35620 2024-04-23T18:38:00.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe926ffd640 1 -- 172.21.15.33:0/2278072401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930010c80 con 0x55f618b35620 2024-04-23T18:38:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe926ffd640 1 -- 172.21.15.33:0/2278072401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe930019b30 con 0x55f618b35620 2024-04-23T18:38:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 -- 172.21.15.33:0/2278072401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 msgr2=0x55f618b35a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:00.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe93000cbf0 tx=0x7fe93000e720 comp rx=0 tx=0).stop 2024-04-23T18:38:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 -- 172.21.15.33:0/2278072401 shutdown_connections 2024-04-23T18:38:00.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b42bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 --2- 172.21.15.33:0/2278072401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b35a00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 -- 172.21.15.33:0/2278072401 >> 172.21.15.33:0/2278072401 conn(0x55f618b31b30 msgr2=0x55f618b31f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:01.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 -- 172.21.15.33:0/2278072401 shutdown_connections 2024-04-23T18:38:01.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.841+0000 7fe945a20740 1 -- 172.21.15.33:0/2278072401 wait complete. 2024-04-23T18:38:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 -- start start 2024-04-23T18:38:01.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 Processor -- start 2024-04-23T18:38:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b3d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f618b43430 con 0x55f618b35620 2024-04-23T18:38:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f618a350b0 con 0x55f618b40800 2024-04-23T18:38:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f618b435b0 con 0x55f618b35f40 2024-04-23T18:38:01.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe945a20740 10 client.0 _ll_register_callbacks cb 0x55f618a22d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe940ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:02.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe940ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37392/0 (socket says 172.21.15.33:37392) 2024-04-23T18:38:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe940ab1640 1 -- 172.21.15.33:0/2378071103 learned_addr learned my addr 172.21.15.33:0/2378071103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:02.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.842+0000 7fe93bfff640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b3d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe9412b2640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:02.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 -- 172.21.15.33:0/2378071103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 msgr2=0x55f618b3d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b3d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 -- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 msgr2=0x55f618b3d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b3d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 -- 172.21.15.33:0/2378071103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f618879620 con 0x55f618b35620 2024-04-23T18:38:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe9412b2640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b3d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:38:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe940ab1640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe930019750 tx=0x7fe930005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc1ed3bc275ac10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:02.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe91bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93002a060 con 0x55f618b35620 2024-04-23T18:38:03.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fe930005250 con 0x55f618b35620 2024-04-23T18:38:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe930010c80 con 0x55f618b35620 2024-04-23T18:38:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe930019e40 con 0x55f618b35620 2024-04-23T18:38:03.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe945a20740 10 client.16935 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.843+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f618b4f2c0 con 0x55f618b35620 2024-04-23T18:38:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe930010c80 con 0x55f618b35620 2024-04-23T18:38:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe9300046c0 con 0x55f618b35620 2024-04-23T18:38:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 10 client.16935 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:04.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 20 client.16935 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:04.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618b506c0 0x55f618b52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:04.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 4 client.16935 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:04.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f618aa8390 con 0x55f618b506c0 2024-04-23T18:38:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe93bfff640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618b506c0 0x55f618b52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe91bfff640 10 client.16935 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.844+0000 7fe93bfff640 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618b506c0 0x55f618b52ab0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b16ffc8998c35bb server_cookie=6bd123aa2fb603fa in_seq=0 out_seq=0 2024-04-23T18:38:05.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe91bfff640 1 -- 172.21.15.33:0/2378071103 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f618aa8390 con 0x55f618b506c0 2024-04-23T18:38:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe91bfff640 10 client.16935 handle_command_reply: tid=0 2024-04-23T18:38:05.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 client.16935 shutdown 2024-04-23T18:38:05.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 msgr2=0x55f618b400c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe930019750 tx=0x7fe930005d40 comp rx=0 tx=0).stop 2024-04-23T18:38:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 shutdown_connections 2024-04-23T18:38:05.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618b506c0 0x55f618b52ab0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:05.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b40800 0x55f618b3d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe91bfff640 0 client.16935 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35f40 0x55f618b3d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 --2- 172.21.15.33:0/2378071103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b35620 0x55f618b400c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 >> 172.21.15.33:0/2378071103 conn(0x55f618b31b30 msgr2=0x55f618b37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.845+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 shutdown_connections 2024-04-23T18:38:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.846+0000 7fe945a20740 1 -- 172.21.15.33:0/2378071103 wait complete. 2024-04-23T18:38:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.846+0000 7fe945a20740 20 client.16935 trim_cache size 0 max 16384 2024-04-23T18:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 Processor -- start 2024-04-23T18:38:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 -- start start 2024-04-23T18:38:06.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b4c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b3fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1adad0 con 0x55f618b40800 2024-04-23T18:38:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1af610 con 0x55f618b35f40 2024-04-23T18:38:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe945a20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1ae1a0 con 0x55f618b35620 2024-04-23T18:38:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:06.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37402/0 (socket says 172.21.15.33:37402) 2024-04-23T18:38:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 -- 172.21.15.33:0/3971740202 learned_addr learned my addr 172.21.15.33:0/3971740202 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe93bfff640 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b3fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe940ab1640 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b4c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 -- 172.21.15.33:0/3971740202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 msgr2=0x55f618b4c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b4c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 -- 172.21.15.33:0/3971740202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 msgr2=0x55f618b3fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b3fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.861+0000 7fe9412b2640 1 -- 172.21.15.33:0/3971740202 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f618b369f0 con 0x55f618b40800 2024-04-23T18:38:07.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe9412b2640 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe934098db0 tx=0x7fe93409d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339dcc362b1d61da server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe9267fc640 1 -- 172.21.15.33:0/3971740202 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93401f040 con 0x55f618b40800 2024-04-23T18:38:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe9267fc640 1 -- 172.21.15.33:0/3971740202 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9340a0d00 con 0x55f618b40800 2024-04-23T18:38:07.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe9267fc640 1 -- 172.21.15.33:0/3971740202 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9340870b0 con 0x55f618b40800 2024-04-23T18:38:07.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 -- 172.21.15.33:0/3971740202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 msgr2=0x55f618b58050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe934098db0 tx=0x7fe93409d050 comp rx=0 tx=0).stop 2024-04-23T18:38:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 -- 172.21.15.33:0/3971740202 shutdown_connections 2024-04-23T18:38:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b58050 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b3fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 --2- 172.21.15.33:0/3971740202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b4c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 -- 172.21.15.33:0/3971740202 >> 172.21.15.33:0/3971740202 conn(0x55f618b31b30 msgr2=0x55f618b37860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:07.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 -- 172.21.15.33:0/3971740202 shutdown_connections 2024-04-23T18:38:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.862+0000 7fe945a20740 1 -- 172.21.15.33:0/3971740202 wait complete. 2024-04-23T18:38:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 -- start start 2024-04-23T18:38:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 Processor -- start 2024-04-23T18:38:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b44200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b44740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1aef40 con 0x55f618b40800 2024-04-23T18:38:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1ae870 con 0x55f618b35f40 2024-04-23T18:38:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1abab0 con 0x55f618b35620 2024-04-23T18:38:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe945a20740 10 client.0 _ll_register_callbacks cb 0x7fe930019d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe9412b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37412/0 (socket says 172.21.15.33:37412) 2024-04-23T18:38:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe9412b2640 1 -- 172.21.15.33:0/1445463772 learned_addr learned my addr 172.21.15.33:0/1445463772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.863+0000 7fe940ab1640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b44200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b44740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 -- 172.21.15.33:0/1445463772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 msgr2=0x55f618b44200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b44200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 -- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 msgr2=0x55f618b44740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:08.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b44740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:08.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 -- 172.21.15.33:0/1445463772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f618bb58c0 con 0x55f618b40800 2024-04-23T18:38:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe93bfff640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b44740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:38:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe9412b2640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe9340bd140 tx=0x7fe934005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc5a9408cc04277 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:08.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9340b8070 con 0x55f618b40800 2024-04-23T18:38:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8498+0+0 (secure 0 0 0) 0x7fe9340872c0 con 0x55f618b40800 2024-04-23T18:38:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9340a0d00 con 0x55f618b40800 2024-04-23T18:38:09.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe93401f050 con 0x55f618b40800 2024-04-23T18:38:09.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe945a20740 10 client.16941 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f618bb6ba0 con 0x55f618b40800 2024-04-23T18:38:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9340a0d00 con 0x55f618b40800 2024-04-23T18:38:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fe93400f040 con 0x55f618b40800 2024-04-23T18:38:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe945a20740 10 client.16941 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.864+0000 7fe945a20740 10 client.16941 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe945a20740 20 client.16941 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618bb7210 0x55f618bb9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe945a20740 4 client.16941 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f618aa2200 con 0x55f618bb7210 2024-04-23T18:38:10.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe940ab1640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618bb7210 0x55f618bb9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe940ab1640 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618bb7210 0x55f618bb9600 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f6edea2f1fb7eaa server_cookie=2a06349252b2969b in_seq=0 out_seq=0 2024-04-23T18:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.865+0000 7fe91bfff640 10 client.16941 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:10.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe91bfff640 1 -- 172.21.15.33:0/1445463772 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f618aa2200 con 0x55f618bb7210 2024-04-23T18:38:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe91bfff640 10 client.16941 handle_command_reply: tid=0 2024-04-23T18:38:10.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 client.16941 shutdown 2024-04-23T18:38:10.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 msgr2=0x55f618b44c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe9340bd140 tx=0x7fe934005720 comp rx=0 tx=0).stop 2024-04-23T18:38:10.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 shutdown_connections 2024-04-23T18:38:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f618bb7210 0x55f618bb9600 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:10.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f618b40800 0x55f618b44c80 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f618b35f40 0x55f618b44740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe91bfff640 0 client.16941 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:10.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 --2- 172.21.15.33:0/1445463772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f618b35620 0x55f618b44200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 >> 172.21.15.33:0/1445463772 conn(0x55f618b31b30 msgr2=0x55f618b3cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:11.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 shutdown_connections 2024-04-23T18:38:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 1 -- 172.21.15.33:0/1445463772 wait complete. 2024-04-23T18:38:11.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe945a20740 20 client.16941 trim_cache size 0 max 16384 2024-04-23T18:38:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:38:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:38:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:38:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:38:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 msgr2=0x7fe92c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 0x7fe92c0a6300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe928000c00 tx=0x7fe928032f90 comp rx=0 tx=0).stop 2024-04-23T18:38:11.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 msgr2=0x7fe93c19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:11.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe93400eaa0 tx=0x7fe93400ef60 comp rx=0 tx=0).stop 2024-04-23T18:38:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 shutdown_connections 2024-04-23T18:38:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe92c0a3e40 0x7fe92c0a6300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe93c19cf70 0x7fe93c19d420 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe93c107290 0x7fe93c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 --2- 172.21.15.33:0/1135611785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe93c1021c0 0x7fe93c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 >> 172.21.15.33:0/1135611785 conn(0x7fe93c0fda20 msgr2=0x7fe93c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:12.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 shutdown_connections 2024-04-23T18:38:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:08.875+0000 7fe942d15640 1 -- 172.21.15.33:0/1135611785 wait complete. 2024-04-23T18:38:12.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:08 smithi033 ceph-mon[29274]: pgmap v334: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 71 KiB/s wr, 31 op/s 2024-04-23T18:38:12.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:09 smithi033 ceph-mon[29274]: osdmap e101: 12 total, 12 up, 12 in 2024-04-23T18:38:12.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:10 smithi033 ceph-mon[29274]: pgmap v336: 97 pgs: 97 active+clean; 8.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 85 KiB/s wr, 38 op/s 2024-04-23T18:38:12.370 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:12 smithi033 ceph-mon[29274]: pgmap v337: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:38:12.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:14 smithi033 ceph-mon[29274]: pgmap v338: 97 pgs: 97 active+clean; 8.9 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:38:12.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:16 smithi033 ceph-mon[29274]: pgmap v339: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:38:12.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:16 smithi033 ceph-mon[29274]: osdmap e102: 12 total, 12 up, 12 in 2024-04-23T18:38:12.570 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:18 smithi033 ceph-mon[29274]: pgmap v341: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:38:12.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:20 smithi033 ceph-mon[29274]: pgmap v342: 97 pgs: 97 active+clean; 9.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-23T18:38:12.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:54 smithi122 ceph-mon[32546]: pgmap v359: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:38:12.839 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:56 smithi122 ceph-mon[32546]: pgmap v360: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 21 op/s 2024-04-23T18:38:12.879 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:37:58 smithi122 ceph-mon[32546]: pgmap v361: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:12.912 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:00 smithi122 ceph-mon[32546]: pgmap v362: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:12.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:02 smithi122 ceph-mon[32546]: pgmap v363: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-04-23T18:38:12.988 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:38:13.020 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: in thread 7f704c0b0700 thread_name:md_log_replay 2024-04-23T18:38:13.054 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:38:13.137 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f705d471d20] 2024-04-23T18:38:13.171 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2: /usr/bin/ceph-mds(+0x549372) [0x55938e6eb372] 2024-04-23T18:38:13.204 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55938e6eeb23] 2024-04-23T18:38:13.238 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 4: (MDLog::_replay_thread()+0x617) [0x55938e749647] 2024-04-23T18:38:13.271 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55938e752d31] 2024-04-23T18:38:13.305 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f705d4671ca] 2024-04-23T18:38:13.338 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 7: clone() 2024-04-23T18:38:13.371 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:03.989+0000 7f704c0b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:38:13.406 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: in thread 7f704c0b0700 thread_name:md_log_replay 2024-04-23T18:38:13.438 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:13.555 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:38:13.588 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f705d471d20] 2024-04-23T18:38:13.621 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2: /usr/bin/ceph-mds(+0x549372) [0x55938e6eb372] 2024-04-23T18:38:13.655 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55938e6eeb23] 2024-04-23T18:38:13.688 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 4: (MDLog::_replay_thread()+0x617) [0x55938e749647] 2024-04-23T18:38:13.722 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55938e752d31] 2024-04-23T18:38:13.755 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f705d4671ca] 2024-04-23T18:38:13.788 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 7: clone() 2024-04-23T18:38:13.847 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:38:13.880 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:13.913 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 0> 2024-04-23T18:38:03.989+0000 7f704c0b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:38:13.947 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: in thread 7f704c0b0700 thread_name:md_log_replay 2024-04-23T18:38:14.008 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:14.088 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:38:14.189 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f705d471d20] 2024-04-23T18:38:14.222 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2: /usr/bin/ceph-mds(+0x549372) [0x55938e6eb372] 2024-04-23T18:38:14.255 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55938e6eeb23] 2024-04-23T18:38:14.289 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 4: (MDLog::_replay_thread()+0x617) [0x55938e749647] 2024-04-23T18:38:14.322 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55938e752d31] 2024-04-23T18:38:14.423 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f705d4671ca] 2024-04-23T18:38:14.464 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 7: clone() 2024-04-23T18:38:14.497 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:38:14.531 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:14.572 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 0> 2024-04-23T18:38:03.989+0000 7f704c0b0700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:38:14.606 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: in thread 7f704c0b0700 thread_name:md_log_replay 2024-04-23T18:38:14.639 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:14.672 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:38:14.706 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f705d471d20] 2024-04-23T18:38:14.739 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2: /usr/bin/ceph-mds(+0x549372) [0x55938e6eb372] 2024-04-23T18:38:14.773 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55938e6eeb23] 2024-04-23T18:38:14.807 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 4: (MDLog::_replay_thread()+0x617) [0x55938e749647] 2024-04-23T18:38:14.914 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55938e752d31] 2024-04-23T18:38:14.947 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f705d4671ca] 2024-04-23T18:38:14.981 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 7: clone() 2024-04-23T18:38:15.014 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:38:15.048 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[68886]: 2024-04-23T18:38:15.081 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 podman[69664]: 2024-04-23 18:38:04.043379623 +0000 UTC m=+0.021805511 container died 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:38:15.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-mon[32546]: pgmap v364: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-23T18:38:15.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:04 smithi122 ceph-mon[32546]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:38:15.182 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:54 smithi106 ceph-mon[32716]: pgmap v359: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:38:15.214 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:56 smithi106 ceph-mon[32716]: pgmap v360: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 21 op/s 2024-04-23T18:38:15.248 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:37:58 smithi106 ceph-mon[32716]: pgmap v361: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:15.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:00 smithi106 ceph-mon[32716]: pgmap v362: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:15.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:02 smithi106 ceph-mon[32716]: pgmap v363: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-04-23T18:38:15.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:04 smithi106 ceph-mon[32716]: pgmap v364: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-23T18:38:15.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:04 smithi106 ceph-mon[32716]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:38:15.548 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:06 smithi106 ceph-mon[32716]: pgmap v366: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:15.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:08 smithi106 ceph-mon[32716]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:15.615 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:10 smithi106 ceph-mon[32716]: pgmap v368: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:15.656 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:10 smithi106 ceph-mon[32716]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:38:15.707 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:12 smithi106 ceph-mon[32716]: pgmap v370: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:38:15.757 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 podman[69664]: 2024-04-23 18:38:04.738926345 +0000 UTC m=+0.717352282 container remove 45b83cede4d21cda18182203f607111a8cf959cec96cba0c66a5d2437df5621e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-23T18:38:15.807 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:38:15.858 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:38:15.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:06 smithi122 ceph-mon[32546]: pgmap v366: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:16.005 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:08 smithi122 ceph-mon[32546]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:16.142 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:10 smithi122 ceph-mon[32546]: pgmap v368: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:38:16.190 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:10 smithi122 ceph-mon[32546]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:38:16.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:12 smithi122 ceph-mon[32546]: pgmap v370: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:38:16.290 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:14 smithi122 ceph-mon[32546]: pgmap v371: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-23T18:38:16.343 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 5. 2024-04-23T18:38:16.407 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:38:16.490 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 systemd[1]: Starting Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:38:16.541 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 podman[69786]: 2024-04-23 18:38:15.347130558 +0000 UTC m=+0.018185758 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:38:16.590 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 podman[69786]: 2024-04-23 18:38:15.465373663 +0000 UTC m=+0.136428855 container create 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T18:38:16.677 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 podman[69786]: 2024-04-23 18:38:15.694223995 +0000 UTC m=+0.365279186 container init 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-23T18:38:16.764 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 podman[69786]: 2024-04-23 18:38:15.696822967 +0000 UTC m=+0.367878165 container start 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T18:38:16.907 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mds[69806]: -- 172.21.15.122:0/206136862 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559277fefa40 con 0x559277fc3000 2024-04-23T18:38:16.957 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: starting mds.f at 2024-04-23T18:38:17.007 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 bash[69786]: 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 2024-04-23T18:38:17.058 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:38:15 smithi122 systemd[1]: Started Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:38:17.108 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:14 smithi106 ceph-mon[32716]: pgmap v371: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-23T18:38:17.160 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mon[32546]: daemon mds.f restarted 2024-04-23T18:38:17.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mon[32546]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:38:17.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/3654262177,v1:172.21.15.122:6835/3654262177] up:boot 2024-04-23T18:38:17.357 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:38:17.439 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:38:17.546 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:15 smithi106 ceph-mon[32716]: daemon mds.f restarted 2024-04-23T18:38:17.688 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:15 smithi106 ceph-mon[32716]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:38:17.741 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/3654262177,v1:172.21.15.122:6835/3654262177] up:boot 2024-04-23T18:38:17.791 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:15 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:38:17.841 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:38:17.891 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:16 smithi106 ceph-mon[32716]: pgmap v372: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:38:17.944 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:16 smithi122 ceph-mon[32546]: pgmap v372: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:38:17.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 -- 172.21.15.33:0/140508861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f581810c5d0 msgr2=0x7f581810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:18.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 --2- 172.21.15.33:0/140508861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f581810c5d0 0x7f581810ea00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f5810009ca0 tx=0x7f581002a950 comp rx=0 tx=0).stop 2024-04-23T18:38:18.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 -- 172.21.15.33:0/140508861 shutdown_connections 2024-04-23T18:38:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 --2- 172.21.15.33:0/140508861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f581810c5d0 0x7f581810ea00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 --2- 172.21.15.33:0/140508861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5818103b20 0x7f581810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 --2- 172.21.15.33:0/140508861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 0x7f58181035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 -- 172.21.15.33:0/140508861 >> 172.21.15.33:0/140508861 conn(0x7f58180771d0 msgr2=0x7f58180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 -- 172.21.15.33:0/140508861 shutdown_connections 2024-04-23T18:38:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.977+0000 7f581ee13640 1 -- 172.21.15.33:0/140508861 wait complete. 2024-04-23T18:38:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 Processor -- start 2024-04-23T18:38:18.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 -- start start 2024-04-23T18:38:18.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 0x7f5818100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:18.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:18.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f581810c5d0 0x7f58181018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58181121f0 con 0x7f5818103b20 2024-04-23T18:38:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5818112070 con 0x7f5818103200 2024-04-23T18:38:19.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.978+0000 7f581ee13640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5818112370 con 0x7f581810c5d0 2024-04-23T18:38:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:19.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57526/0 (socket says 172.21.15.33:57526) 2024-04-23T18:38:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 -- 172.21.15.33:0/782313167 learned_addr learned my addr 172.21.15.33:0/782313167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f581cbaf640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 0x7f5818100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:19.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f581d3b0640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f581810c5d0 0x7f58181018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 -- 172.21.15.33:0/782313167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f581810c5d0 msgr2=0x7f58181018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f581810c5d0 0x7f58181018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 -- 172.21.15.33:0/782313167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 msgr2=0x7f5818100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 0x7f5818100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 -- 172.21.15.33:0/782313167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181a6af0 con 0x7f5818103b20 2024-04-23T18:38:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5817fff640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f580400d770 tx=0x7f580400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:20.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5815ffb640 1 -- 172.21.15.33:0/782313167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804017070 con 0x7f5818103b20 2024-04-23T18:38:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5815ffb640 1 -- 172.21.15.33:0/782313167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58040054d0 con 0x7f5818103b20 2024-04-23T18:38:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181a6de0 con 0x7f5818103b20 2024-04-23T18:38:20.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.979+0000 7f5815ffb640 1 -- 172.21.15.33:0/782313167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804004c30 con 0x7f5818103b20 2024-04-23T18:38:20.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.980+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a7250 con 0x7f5818103b20 2024-04-23T18:38:20.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.982+0000 7f5815ffb640 1 -- 172.21.15.33:0/782313167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5804004dd0 con 0x7f5818103b20 2024-04-23T18:38:20.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.982+0000 7f5821c0a740 1 Processor -- start 2024-04-23T18:38:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.982+0000 7f5821c0a740 1 -- start start 2024-04-23T18:38:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5815ffb640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 0x7f58080a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:21.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbe5800 0x55867bbe7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:21.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55867bb3f970 con 0x55867bbda620 2024-04-23T18:38:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55867badc4a0 con 0x55867bbe5800 2024-04-23T18:38:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5821c0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55867bb3b930 con 0x55867bbdaf40 2024-04-23T18:38:21.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581cbaf640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 0x7f58080a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:21.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57528/0 (socket says 172.21.15.33:57528) 2024-04-23T18:38:21.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 -- 172.21.15.33:0/1646133830 learned_addr learned my addr 172.21.15.33:0/1646133830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5815ffb640 1 -- 172.21.15.33:0/782313167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f58040d0140 con 0x7f5818103b20 2024-04-23T18:38:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f5817fff640 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbe5800 0x55867bbe7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 -- 172.21.15.33:0/1646133830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 msgr2=0x55867bbe52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 -- 172.21.15.33:0/1646133830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbe5800 msgr2=0x55867bbe7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbe5800 0x55867bbe7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 -- 172.21.15.33:0/1646133830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867b852210 con 0x55867bbda620 2024-04-23T18:38:21.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581cbaf640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 0x7f58080a62b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f580c00bbe0 tx=0x7f580c00b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.983+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5810009430 tx=0x7f5810004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a64ece0869fd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5802ffd640 1 -- 172.21.15.33:0/1646133830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810034040 con 0x55867bbda620 2024-04-23T18:38:21.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5802ffd640 1 -- 172.21.15.33:0/1646133830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810044040 con 0x55867bbda620 2024-04-23T18:38:22.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5802ffd640 1 -- 172.21.15.33:0/1646133830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810004450 con 0x55867bbda620 2024-04-23T18:38:22.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 -- 172.21.15.33:0/1646133830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 msgr2=0x55867bbdaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:22.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f5810009430 tx=0x7f5810004a60 comp rx=0 tx=0).stop 2024-04-23T18:38:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 -- 172.21.15.33:0/1646133830 shutdown_connections 2024-04-23T18:38:22.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbe5800 0x55867bbe7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1646133830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbda620 0x55867bbdaa00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 -- 172.21.15.33:0/1646133830 >> 172.21.15.33:0/1646133830 conn(0x55867bbd6b30 msgr2=0x55867bbd6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 -- 172.21.15.33:0/1646133830 shutdown_connections 2024-04-23T18:38:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.984+0000 7f5821c0a740 1 -- 172.21.15.33:0/1646133830 wait complete. 2024-04-23T18:38:22.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 -- start start 2024-04-23T18:38:22.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 Processor -- start 2024-04-23T18:38:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbe51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:22.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55867bbe83d0 con 0x55867bbe5800 2024-04-23T18:38:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55867bada0b0 con 0x55867bbda620 2024-04-23T18:38:22.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55867bbe8550 con 0x55867bbdaf40 2024-04-23T18:38:22.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5821c0a740 10 client.0 _ll_register_callbacks cb 0x55867bac7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57532/0 (socket says 172.21.15.33:57532) 2024-04-23T18:38:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f581d3b0640 1 -- 172.21.15.33:0/3659434117 learned_addr learned my addr 172.21.15.33:0/3659434117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:23.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.985+0000 7f5817fff640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581cbaf640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbe51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 -- 172.21.15.33:0/3659434117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 msgr2=0x55867bbe2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:23.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 -- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 msgr2=0x55867bbe51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbe51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:23.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 -- 172.21.15.33:0/3659434117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867b91e620 con 0x55867bbe5800 2024-04-23T18:38:23.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f581d3b0640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5810000f80 tx=0x7f5810003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95c1db517d6f03b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:23.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f57f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810050030 con 0x55867bbe5800 2024-04-23T18:38:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f58100041b0 con 0x55867bbe5800 2024-04-23T18:38:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5810044040 con 0x55867bbe5800 2024-04-23T18:38:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5810034050 con 0x55867bbe5800 2024-04-23T18:38:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f5821c0a740 10 client.16953 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:24.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55867bbf4340 con 0x55867bbe5800 2024-04-23T18:38:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.986+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810044040 con 0x55867bbe5800 2024-04-23T18:38:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f58100306b0 con 0x55867bbe5800 2024-04-23T18:38:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 10 client.16953 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:24.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 20 client.16953 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:24.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bbf5690 0x55867bbf7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 4 client.16953 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55867bb4d390 con 0x55867bbf5690 2024-04-23T18:38:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f581cbaf640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bbf5690 0x55867bbf7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f581cbaf640 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bbf5690 0x55867bbf7a80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=122ada832e348727 server_cookie=9f6b49a37bb462d4 in_seq=0 out_seq=0 2024-04-23T18:38:24.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.987+0000 7f57f7fff640 10 client.16953 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:24.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f57f7fff640 1 -- 172.21.15.33:0/3659434117 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55867bb4d390 con 0x55867bbf5690 2024-04-23T18:38:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f57f7fff640 10 client.16953 handle_command_reply: tid=0 2024-04-23T18:38:25.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 client.16953 shutdown 2024-04-23T18:38:25.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 msgr2=0x55867bbe2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5810000f80 tx=0x7f5810003ca0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 shutdown_connections 2024-04-23T18:38:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bbf5690 0x55867bbf7a80 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbe2a30 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f57f7fff640 0 client.16953 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbe2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 --2- 172.21.15.33:0/3659434117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbe51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 >> 172.21.15.33:0/3659434117 conn(0x55867bbd6b30 msgr2=0x55867bbdc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 shutdown_connections 2024-04-23T18:38:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.988+0000 7f5821c0a740 1 -- 172.21.15.33:0/3659434117 wait complete. 2024-04-23T18:38:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:21.989+0000 7f5821c0a740 20 client.16953 trim_cache size 0 max 16384 2024-04-23T18:38:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.003+0000 7f5821c0a740 1 Processor -- start 2024-04-23T18:38:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.003+0000 7f5821c0a740 1 -- start start 2024-04-23T18:38:25.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbfaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58181afcd0 con 0x55867bbe5800 2024-04-23T18:38:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58181b1810 con 0x55867bbda620 2024-04-23T18:38:26.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5821c0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58181b03a0 con 0x55867bbdaf40 2024-04-23T18:38:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57546/0 (socket says 172.21.15.33:57546) 2024-04-23T18:38:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 -- 172.21.15.33:0/1011955936 learned_addr learned my addr 172.21.15.33:0/1011955936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581cbaf640 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbfaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f5817fff640 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 -- 172.21.15.33:0/1011955936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 msgr2=0x55867bbfaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbfaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 -- 172.21.15.33:0/1011955936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 msgr2=0x55867bbfaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbfaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 -- 172.21.15.33:0/1011955936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867bbe3010 con 0x55867bbe5800 2024-04-23T18:38:26.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5810030a20 tx=0x7f58100093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b1fad30eccd0df server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:26.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f58027fc640 1 -- 172.21.15.33:0/1011955936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810050030 con 0x55867bbe5800 2024-04-23T18:38:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f58027fc640 1 -- 172.21.15.33:0/1011955936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810030c90 con 0x55867bbe5800 2024-04-23T18:38:26.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.004+0000 7f58027fc640 1 -- 172.21.15.33:0/1011955936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810030600 con 0x55867bbe5800 2024-04-23T18:38:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- 172.21.15.33:0/1011955936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 msgr2=0x55867bbdf6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:26.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5810030a20 tx=0x7f58100093e0 comp rx=0 tx=0).stop 2024-04-23T18:38:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- 172.21.15.33:0/1011955936 shutdown_connections 2024-04-23T18:38:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbdf6f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbdaf40 0x55867bbfaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1011955936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbda620 0x55867bbfaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- 172.21.15.33:0/1011955936 >> 172.21.15.33:0/1011955936 conn(0x55867bbd6b30 msgr2=0x55867bbf1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- 172.21.15.33:0/1011955936 shutdown_connections 2024-04-23T18:38:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- 172.21.15.33:0/1011955936 wait complete. 2024-04-23T18:38:27.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 -- start start 2024-04-23T18:38:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.005+0000 7f5821c0a740 1 Processor -- start 2024-04-23T18:38:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 0x55867bc54180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbdaf40 0x55867bc546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:27.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f58181b1140 con 0x55867bbe5800 2024-04-23T18:38:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f58181b0a70 con 0x55867bbdaf40 2024-04-23T18:38:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58181adcd0 con 0x55867bbda620 2024-04-23T18:38:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5821c0a740 10 client.0 _ll_register_callbacks cb 0x7f581002f6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581cbaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 0x55867bc54180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57562/0 (socket says 172.21.15.33:57562) 2024-04-23T18:38:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 -- 172.21.15.33:0/1052228152 learned_addr learned my addr 172.21.15.33:0/1052228152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f5817fff640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbdaf40 0x55867bc546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 -- 172.21.15.33:0/1052228152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 msgr2=0x55867bc54180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 0x55867bc54180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 -- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbdaf40 msgr2=0x55867bc546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbdaf40 0x55867bc546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:27.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581d3b0640 1 -- 172.21.15.33:0/1052228152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867bbfa9b0 con 0x55867bbe5800 2024-04-23T18:38:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.006+0000 7f581cbaf640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 0x55867bc54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:38:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f581d3b0640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5810030040 tx=0x7f581002b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a669e7004c1832 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810061070 con 0x55867bbe5800 2024-04-23T18:38:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f5810004140 con 0x55867bbe5800 2024-04-23T18:38:28.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5810030c90 con 0x55867bbe5800 2024-04-23T18:38:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f581006d050 con 0x55867bbe5800 2024-04-23T18:38:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 10 client.16959 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55867bc5b930 con 0x55867bbe5800 2024-04-23T18:38:28.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5810030c90 con 0x55867bbe5800 2024-04-23T18:38:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f58100667b0 con 0x55867bbe5800 2024-04-23T18:38:28.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 10 client.16959 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:28.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 20 client.16959 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bc5bcc0 0x55867bc5e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 4 client.16959 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.007+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55867bb47200 con 0x55867bc5bcc0 2024-04-23T18:38:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.008+0000 7f581cbaf640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bc5bcc0 0x55867bc5e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.008+0000 7f581cbaf640 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bc5bcc0 0x55867bc5e0d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96aaf19f07109b0d server_cookie=b007b2895298ba00 in_seq=0 out_seq=0 2024-04-23T18:38:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.008+0000 7f57f7fff640 10 client.16959 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f57f7fff640 1 -- 172.21.15.33:0/1052228152 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55867bb47200 con 0x55867bc5bcc0 2024-04-23T18:38:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f57f7fff640 10 client.16959 handle_command_reply: tid=0 2024-04-23T18:38:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 client.16959 shutdown 2024-04-23T18:38:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 msgr2=0x55867bbfa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5810030040 tx=0x7f581002b0d0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 shutdown_connections 2024-04-23T18:38:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55867bc5bcc0 0x55867bc5e0d0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55867bbe5800 0x55867bbfa2c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f57f7fff640 0 client.16959 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55867bbdaf40 0x55867bc546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 --2- 172.21.15.33:0/1052228152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55867bbda620 0x55867bc54180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 >> 172.21.15.33:0/1052228152 conn(0x55867bbd6b30 msgr2=0x55867bbf1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 shutdown_connections 2024-04-23T18:38:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 1 -- 172.21.15.33:0/1052228152 wait complete. 2024-04-23T18:38:29.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.016+0000 7f5821c0a740 20 client.16959 trim_cache size 0 max 16384 2024-04-23T18:38:29.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:38:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:38:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:38:29.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:38:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 msgr2=0x7f58080a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 0x7f58080a62b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f580c00bbe0 tx=0x7f580c00b390 comp rx=0 tx=0).stop 2024-04-23T18:38:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 msgr2=0x7f5818101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f580400d770 tx=0x7f580400dc30 comp rx=0 tx=0).stop 2024-04-23T18:38:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 shutdown_connections 2024-04-23T18:38:30.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f58080a3df0 0x7f58080a62b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f581810c5d0 0x7f58181018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5818103b20 0x7f5818101390 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 --2- 172.21.15.33:0/782313167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5818103200 0x7f5818100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:30.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 >> 172.21.15.33:0/782313167 conn(0x7f58180771d0 msgr2=0x7f58180fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 shutdown_connections 2024-04-23T18:38:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:22.017+0000 7f581ee13640 1 -- 172.21.15.33:0/782313167 wait complete. 2024-04-23T18:38:30.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:18 smithi122 ceph-mon[32546]: pgmap v374: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-23T18:38:30.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:20 smithi122 ceph-mon[32546]: pgmap v375: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-04-23T18:38:30.564 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:23 smithi122 ceph-mon[32546]: pgmap v376: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:38:30.595 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:25 smithi122 ceph-mon[32546]: pgmap v377: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:38:30.629 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:27 smithi122 ceph-mon[32546]: pgmap v378: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T18:38:30.662 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:29 smithi122 ceph-mon[32546]: pgmap v379: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:38:30.696 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:18 smithi106 ceph-mon[32716]: pgmap v374: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-23T18:38:30.729 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:20 smithi106 ceph-mon[32716]: pgmap v375: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-04-23T18:38:30.762 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:23 smithi106 ceph-mon[32716]: pgmap v376: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:38:30.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:25 smithi106 ceph-mon[32716]: pgmap v377: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:38:30.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:27 smithi106 ceph-mon[32716]: pgmap v378: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T18:38:30.954 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:29 smithi106 ceph-mon[32716]: pgmap v379: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:38:30.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:22 smithi033 ceph-mon[29274]: pgmap v343: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:38:31.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:24 smithi033 ceph-mon[29274]: pgmap v344: 97 pgs: 97 active+clean; 9.1 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 15 op/s 2024-04-23T18:38:31.054 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:26 smithi033 ceph-mon[29274]: pgmap v345: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-04-23T18:38:31.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:28 smithi033 ceph-mon[29274]: pgmap v346: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:31.121 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:30 smithi033 ceph-mon[29274]: pgmap v347: 97 pgs: 97 active+clean; 9.3 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:31.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:32 smithi033 ceph-mon[29274]: pgmap v348: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 20 op/s 2024-04-23T18:38:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1809007402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 msgr2=0x7f1db81025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1809007402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81025a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1da8009900 tx=0x7f1da802a410 comp rx=0 tx=0).stop 2024-04-23T18:38:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1809007402 shutdown_connections 2024-04-23T18:38:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1809007402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 0x7f1db810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:31.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1809007402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db8102b70 0x7f1db8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:31.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1809007402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81025a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1809007402 >> 172.21.15.33:0/1809007402 conn(0x7f1db80fda20 msgr2=0x7f1db80ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1809007402 shutdown_connections 2024-04-23T18:38:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.117+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1809007402 wait complete. 2024-04-23T18:38:31.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 Processor -- start 2024-04-23T18:38:31.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 -- start start 2024-04-23T18:38:31.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 0x7f1db81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db819a7b0 0x7f1db819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:31.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81121f0 con 0x7f1db81021c0 2024-04-23T18:38:31.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8112070 con 0x7f1db8107290 2024-04-23T18:38:31.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1dbdd7d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1db8112370 con 0x7f1db819a7b0 2024-04-23T18:38:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1db77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45400/0 (socket says 172.21.15.33:45400) 2024-04-23T18:38:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.118+0000 7f1db77fe640 1 -- 172.21.15.33:0/1144293280 learned_addr learned my addr 172.21.15.33:0/1144293280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db819a7b0 0x7f1db819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 0x7f1db81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 -- 172.21.15.33:0/1144293280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db819a7b0 msgr2=0x7f1db819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:32.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db819a7b0 0x7f1db819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 -- 172.21.15.33:0/1144293280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 msgr2=0x7f1db81a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 0x7f1db81a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:32.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 -- 172.21.15.33:0/1144293280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db819b340 con 0x7f1db81021c0 2024-04-23T18:38:32.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005940 tx=0x7f1da80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db4ff9640 1 -- 172.21.15.33:0/1144293280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da803f030 con 0x7f1db81021c0 2024-04-23T18:38:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db819b5d0 con 0x7f1db81021c0 2024-04-23T18:38:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db4ff9640 1 -- 172.21.15.33:0/1144293280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da8003cb0 con 0x7f1db81021c0 2024-04-23T18:38:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1db4ff9640 1 -- 172.21.15.33:0/1144293280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da8035070 con 0x7f1db81021c0 2024-04-23T18:38:32.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.119+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db81a7780 con 0x7f1db81021c0 2024-04-23T18:38:32.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1db4ff9640 1 -- 172.21.15.33:0/1144293280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1da8042030 con 0x7f1db81021c0 2024-04-23T18:38:32.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 Processor -- start 2024-04-23T18:38:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 -- start start 2024-04-23T18:38:32.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 0x55f8405802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840582bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8404da970 con 0x55f840575620 2024-04-23T18:38:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f8404774a0 con 0x55f840580800 2024-04-23T18:38:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f8404d6930 con 0x55f840575f40 2024-04-23T18:38:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.122+0000 7f1db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45402/0 (socket says 172.21.15.33:45402) 2024-04-23T18:38:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 -- 172.21.15.33:0/1567598098 learned_addr learned my addr 172.21.15.33:0/1567598098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db4ff9640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 0x7f1da00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 0x55f8405802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:33.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840582bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:33.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 -- 172.21.15.33:0/1567598098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 msgr2=0x55f8405802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:33.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 0x55f8405802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 -- 172.21.15.33:0/1567598098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 msgr2=0x55f840582bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840582bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 -- 172.21.15.33:0/1567598098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8401ed210 con 0x55f840575620 2024-04-23T18:38:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 0x55f8405802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db4ff9640 1 -- 172.21.15.33:0/1144293280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f1da80f1160 con 0x7f1db81021c0 2024-04-23T18:38:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 0x7f1da00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:33.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009fc0 tx=0x7f1db0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecae005b747eb1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1d9dffb640 1 -- 172.21.15.33:0/1567598098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db001c900 con 0x55f840575620 2024-04-23T18:38:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1d9dffb640 1 -- 172.21.15.33:0/1567598098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db001b3d0 con 0x55f840575620 2024-04-23T18:38:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.123+0000 7f1d9dffb640 1 -- 172.21.15.33:0/1567598098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db0026680 con 0x55f840575620 2024-04-23T18:38:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567598098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 msgr2=0x55f840575a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 0x7f1da00a6280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1db0020000 tx=0x7f1db001d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009fc0 tx=0x7f1db0010800 comp rx=0 tx=0).stop 2024-04-23T18:38:33.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567598098 shutdown_connections 2024-04-23T18:38:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840582bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575f40 0x55f8405802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567598098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575620 0x55f840575a00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567598098 >> 172.21.15.33:0/1567598098 conn(0x55f840571b30 msgr2=0x55f840571f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567598098 shutdown_connections 2024-04-23T18:38:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567598098 wait complete. 2024-04-23T18:38:34.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.124+0000 7f1dc0b72740 1 -- start start 2024-04-23T18:38:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 Processor -- start 2024-04-23T18:38:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840580130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f84057d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f840583430 con 0x55f840575f40 2024-04-23T18:38:34.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f8404750b0 con 0x55f840580800 2024-04-23T18:38:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f8405835b0 con 0x55f840575620 2024-04-23T18:38:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1dc0b72740 10 client.0 _ll_register_callbacks cb 0x55f840462d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:34.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45410/0 (socket says 172.21.15.33:45410) 2024-04-23T18:38:34.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1384687388 learned_addr learned my addr 172.21.15.33:0/1384687388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840580130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f84057d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:34.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.125+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1384687388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 msgr2=0x55f840580130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840580130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 msgr2=0x55f84057d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f84057d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1384687388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8402b9620 con 0x55f840575f40 2024-04-23T18:38:35.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840580130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f84057d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1db806b5f0 tx=0x7f1dac011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841198b2ae3bce1c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1d96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac01e060 con 0x55f840575f40 2024-04-23T18:38:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f1dac00fdb0 con 0x55f840575f40 2024-04-23T18:38:35.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db001b3d0 con 0x55f840575f40 2024-04-23T18:38:35.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1dac022420 con 0x55f840575f40 2024-04-23T18:38:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1dc0b72740 10 client.16971 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:35.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f84058f3a0 con 0x55f840575f40 2024-04-23T18:38:35.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.126+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db001b3d0 con 0x55f840575f40 2024-04-23T18:38:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f1dac018030 con 0x55f840575f40 2024-04-23T18:38:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 10 client.16971 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 20 client.16971 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405908b0 0x55f840592cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 4 client.16971 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:38:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8404e8390 con 0x55f8405908b0 2024-04-23T18:38:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405908b0 0x55f840592cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405908b0 0x55f840592cc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa7d3ab17d3f9ee2 server_cookie=daa64a8bcc3aa13d in_seq=0 out_seq=0 2024-04-23T18:38:36.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.127+0000 7f1d96ffd640 10 client.16971 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.128+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1384687388 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8404e8390 con 0x55f8405908b0 2024-04-23T18:38:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.128+0000 7f1d96ffd640 10 client.16971 handle_command_reply: tid=0 2024-04-23T18:38:36.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.128+0000 7f1dc0b72740 1 client.16971 shutdown 2024-04-23T18:38:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 msgr2=0x55f84057d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1db806b5f0 tx=0x7f1dac011be0 comp rx=0 tx=0).stop 2024-04-23T18:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 shutdown_connections 2024-04-23T18:38:36.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405908b0 0x55f840592cc0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f84057d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f84057d420 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1d96ffd640 0 client.16971 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1384687388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840580130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:36.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 >> 172.21.15.33:0/1384687388 conn(0x55f840571b30 msgr2=0x55f840577860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 shutdown_connections 2024-04-23T18:38:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1384687388 wait complete. 2024-04-23T18:38:37.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.129+0000 7f1dc0b72740 20 client.16971 trim_cache size 0 max 16384 2024-04-23T18:38:37.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 Processor -- start 2024-04-23T18:38:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 -- start start 2024-04-23T18:38:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f84058c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:37.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 0x55f8405985d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81afd90 con 0x55f840580800 2024-04-23T18:38:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81b18d0 con 0x55f840575f40 2024-04-23T18:38:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81b0460 con 0x55f840575620 2024-04-23T18:38:37.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:37.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1db7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45420/0 (socket says 172.21.15.33:45420) 2024-04-23T18:38:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1db7fff640 1 -- 172.21.15.33:0/795995901 learned_addr learned my addr 172.21.15.33:0/795995901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:37.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1db77fe640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f84058c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.144+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 0x55f8405985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 -- 172.21.15.33:0/795995901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 msgr2=0x55f84058c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f84058c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 -- 172.21.15.33:0/795995901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 msgr2=0x55f8405985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:37.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 0x55f8405985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:37.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 -- 172.21.15.33:0/795995901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84057c3d0 con 0x55f840580800 2024-04-23T18:38:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 0x55f8405985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db77fe640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f84058c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:38:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1db7fff640 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1db0005bd0 tx=0x7f1db0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193346ec8627a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1d9d7fa640 1 -- 172.21.15.33:0/795995901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db002d040 con 0x55f840580800 2024-04-23T18:38:38.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1d9d7fa640 1 -- 172.21.15.33:0/795995901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db00096e0 con 0x55f840580800 2024-04-23T18:38:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.145+0000 7f1d9d7fa640 1 -- 172.21.15.33:0/795995901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db001c880 con 0x55f840580800 2024-04-23T18:38:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- 172.21.15.33:0/795995901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 msgr2=0x55f840598b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:38.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1db0005bd0 tx=0x7f1db0027000 comp rx=0 tx=0).stop 2024-04-23T18:38:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- 172.21.15.33:0/795995901 shutdown_connections 2024-04-23T18:38:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840580800 0x55f840598b10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:38.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840575f40 0x55f8405985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:38.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/795995901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f84058c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- 172.21.15.33:0/795995901 >> 172.21.15.33:0/795995901 conn(0x55f840571b30 msgr2=0x55f840570cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:38.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- 172.21.15.33:0/795995901 shutdown_connections 2024-04-23T18:38:38.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- 172.21.15.33:0/795995901 wait complete. 2024-04-23T18:38:38.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- start start 2024-04-23T18:38:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 Processor -- start 2024-04-23T18:38:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840586320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840586da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.146+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81b1200 con 0x55f840575f40 2024-04-23T18:38:38.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81b0b30 con 0x55f840580800 2024-04-23T18:38:38.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1db81add90 con 0x55f840575620 2024-04-23T18:38:38.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 10 client.0 _ll_register_callbacks cb 0x7f1dac00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:38:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45422/0 (socket says 172.21.15.33:45422) 2024-04-23T18:38:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1567276802 learned_addr learned my addr 172.21.15.33:0/1567276802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:38:38.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840586320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:38.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db7fff640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840586da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1567276802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 msgr2=0x55f840586320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840586320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 msgr2=0x55f840586da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:38:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840586da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 -- 172.21.15.33:0/1567276802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8405f5f20 con 0x55f840575f40 2024-04-23T18:38:39.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1db6ffd640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55f84057e9f0 tx=0x7f1dac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22639e776ae359f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:38:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1d96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:38:39.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac000e60 con 0x55f840575f40 2024-04-23T18:38:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f1dac011030 con 0x55f840575f40 2024-04-23T18:38:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1db00096e0 con 0x55f840575f40 2024-04-23T18:38:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1dac00e3b0 con 0x55f840575f40 2024-04-23T18:38:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 10 client.16977 fetch_fsmap learned FSMap version 35 2024-04-23T18:38:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.147+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f8405f7230 con 0x55f840575f40 2024-04-23T18:38:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db00096e0 con 0x55f840575f40 2024-04-23T18:38:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f1dac018040 con 0x55f840575f40 2024-04-23T18:38:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 10 client.16977 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:38:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:38:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 20 client.16977 populate_metadata read hostname 'smithi033' 2024-04-23T18:38:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405f7890 0x55f8405f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 4 client.16977 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:38:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8404e2200 con 0x55f8405f7890 2024-04-23T18:38:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.148+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405f7890 0x55f8405f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:38:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.149+0000 7f1db77fe640 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405f7890 0x55f8405f9cb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70f883d0534b87e5 server_cookie=23ec19d6b378e3a6 in_seq=0 out_seq=0 2024-04-23T18:38:39.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.149+0000 7f1d96ffd640 10 client.16977 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:39.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.156+0000 7f1d96ffd640 1 -- 172.21.15.33:0/1567276802 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8404e2200 con 0x55f8405f7890 2024-04-23T18:38:39.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.156+0000 7f1d96ffd640 10 client.16977 handle_command_reply: tid=0 2024-04-23T18:38:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.156+0000 7f1dc0b72740 1 client.16977 shutdown 2024-04-23T18:38:39.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 msgr2=0x55f840586860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55f84057e9f0 tx=0x7f1dac00eee0 comp rx=0 tx=0).stop 2024-04-23T18:38:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 shutdown_connections 2024-04-23T18:38:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f8405f7890 0x55f8405f9cb0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f840580800 0x55f840586da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f840575f40 0x55f840586860 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 --2- 172.21.15.33:0/1567276802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f840575620 0x55f840586320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1d96ffd640 0 client.16977 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:38:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 >> 172.21.15.33:0/1567276802 conn(0x55f840571b30 msgr2=0x55f84058b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 shutdown_connections 2024-04-23T18:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 1 -- 172.21.15.33:0/1567276802 wait complete. 2024-04-23T18:38:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dc0b72740 20 client.16977 trim_cache size 0 max 16384 2024-04-23T18:38:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:38:40.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:38:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:38:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:38:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 msgr2=0x7f1da00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:41.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 0x7f1da00a6280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1db0020000 tx=0x7f1db001d000 comp rx=0 tx=0).stop 2024-04-23T18:38:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 msgr2=0x7f1db81a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:41.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1da8005940 tx=0x7f1da80042f0 comp rx=0 tx=0).stop 2024-04-23T18:38:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 shutdown_connections 2024-04-23T18:38:41.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1da00a3dc0 0x7f1da00a6280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1db819a7b0 0x7f1db819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:41.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1db8107290 0x7f1db81a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:41.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 --2- 172.21.15.33:0/1144293280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1db81021c0 0x7f1db81a0640 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:41.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.157+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 >> 172.21.15.33:0/1144293280 conn(0x7f1db80fda20 msgr2=0x7f1db80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.158+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 shutdown_connections 2024-04-23T18:38:41.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:35.158+0000 7f1dbdd7d640 1 -- 172.21.15.33:0/1144293280 wait complete. 2024-04-23T18:38:41.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:34 smithi033 ceph-mon[29274]: pgmap v349: 97 pgs: 97 active+clean; 9.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:41.662 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:36 smithi033 ceph-mon[29274]: pgmap v350: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-23T18:38:41.696 INFO:tasks.workunit.client.0.smithi033.stdout:375/600: open fd = 3 2024-04-23T18:38:41.729 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:41.788 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:41.846 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:41.901 INFO:tasks.workunit.client.0.smithi033.stdout:376/600: open fd = 3 2024-04-23T18:38:41.939 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:41.979 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.013 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:42.046 INFO:tasks.workunit.client.0.smithi033.stdout:377/600: open fd = 3 2024-04-23T18:38:42.079 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.112 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.146 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:42.179 INFO:tasks.workunit.client.0.smithi033.stdout:378/600: open fd = 3 2024-04-23T18:38:42.213 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.314 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.383 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:42.429 INFO:tasks.workunit.client.0.smithi033.stdout:379/600: open fd = 3 2024-04-23T18:38:42.463 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.530 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:42.563 INFO:tasks.workunit.client.0.smithi033.stdout:380/600: open fd = 3 2024-04-23T18:38:42.596 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.630 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.663 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:42.696 INFO:tasks.workunit.client.0.smithi033.stdout:381/600: open fd = 3 2024-04-23T18:38:42.780 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.813 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:42.978 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.005 INFO:tasks.workunit.client.0.smithi033.stdout:382/600: open fd = 3 2024-04-23T18:38:43.038 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.071 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.105 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.138 INFO:tasks.workunit.client.0.smithi033.stdout:383/600: open fd = 3 2024-04-23T18:38:43.172 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.228 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.343 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.380 INFO:tasks.workunit.client.0.smithi033.stdout:384/600: open fd = 3 2024-04-23T18:38:43.413 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.555 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.580 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.597 INFO:tasks.workunit.client.0.smithi033.stdout:385/600: open fd = 3 2024-04-23T18:38:43.614 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.630 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.655 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.672 INFO:tasks.workunit.client.0.smithi033.stdout:386/600: open fd = 3 2024-04-23T18:38:43.697 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.780 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.798 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.814 INFO:tasks.workunit.client.0.smithi033.stdout:387/600: open fd = 3 2024-04-23T18:38:43.916 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.939 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:43.956 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:43.973 INFO:tasks.workunit.client.0.smithi033.stdout:388/600: open fd = 3 2024-04-23T18:38:43.990 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.007 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.061 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:44.081 INFO:tasks.workunit.client.0.smithi033.stdout:389/600: open fd = 3 2024-04-23T18:38:44.186 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.206 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.223 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:44.358 INFO:tasks.workunit.client.0.smithi033.stdout:390/600: open fd = 3 2024-04-23T18:38:44.381 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.398 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.415 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:44.432 INFO:tasks.workunit.client.0.smithi033.stdout:391/600: open fd = 3 2024-04-23T18:38:44.459 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.510 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.560 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:44.610 INFO:tasks.workunit.client.0.smithi033.stdout:392/600: open fd = 3 2024-04-23T18:38:44.660 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.760 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.810 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:44.860 INFO:tasks.workunit.client.0.smithi033.stdout:393/600: open fd = 3 2024-04-23T18:38:44.910 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:44.960 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.076 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:45.127 INFO:tasks.workunit.client.0.smithi033.stdout:394/600: open fd = 3 2024-04-23T18:38:45.177 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.227 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.285 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:45.335 INFO:tasks.workunit.client.0.smithi033.stdout:395/600: open fd = 3 2024-04-23T18:38:45.386 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.485 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.577 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:45.710 INFO:tasks.workunit.client.0.smithi033.stdout:396/600: open fd = 3 2024-04-23T18:38:45.777 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.861 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:45.910 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:45.960 INFO:tasks.workunit.client.0.smithi033.stdout:397/600: open fd = 3 2024-04-23T18:38:46.010 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.060 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.110 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:46.185 INFO:tasks.workunit.client.0.smithi033.stdout:398/600: open fd = 3 2024-04-23T18:38:46.235 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.285 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.335 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:46.385 INFO:tasks.workunit.client.0.smithi033.stdout:399/600: open fd = 3 2024-04-23T18:38:46.435 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.486 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.536 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:46.586 INFO:tasks.workunit.client.0.smithi033.stdout:400/600: open fd = 3 2024-04-23T18:38:46.635 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.687 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.819 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:46.869 INFO:tasks.workunit.client.0.smithi033.stdout:401/600: open fd = 3 2024-04-23T18:38:46.919 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:46.969 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.019 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:47.069 INFO:tasks.workunit.client.0.smithi033.stdout:402/600: open fd = 3 2024-04-23T18:38:47.119 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.169 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.252 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:47.294 INFO:tasks.workunit.client.0.smithi033.stdout:403/600: open fd = 3 2024-04-23T18:38:47.353 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.395 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.445 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:47.495 INFO:tasks.workunit.client.0.smithi033.stdout:404/600: open fd = 3 2024-04-23T18:38:47.611 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.661 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.711 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:47.761 INFO:tasks.workunit.client.0.smithi033.stdout:405/600: open fd = 3 2024-04-23T18:38:47.811 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.862 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:47.911 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:47.990 INFO:tasks.workunit.client.0.smithi033.stdout:406/600: open fd = 3 2024-04-23T18:38:48.036 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.087 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.145 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:48.228 INFO:tasks.workunit.client.0.smithi033.stdout:407/600: open fd = 3 2024-04-23T18:38:48.278 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.328 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.378 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:48.428 INFO:tasks.workunit.client.0.smithi033.stdout:408/600: open fd = 3 2024-04-23T18:38:48.548 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.604 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.703 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:48.754 INFO:tasks.workunit.client.0.smithi033.stdout:409/600: open fd = 3 2024-04-23T18:38:48.804 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:48.854 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.087 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:49.129 INFO:tasks.workunit.client.0.smithi033.stdout:410/600: open fd = 3 2024-04-23T18:38:49.217 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.246 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.279 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:49.312 INFO:tasks.workunit.client.0.smithi033.stdout:411/600: open fd = 3 2024-04-23T18:38:49.371 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.426 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.454 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:49.487 INFO:tasks.workunit.client.0.smithi033.stdout:412/600: open fd = 3 2024-04-23T18:38:49.521 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.646 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.679 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:49.773 INFO:tasks.workunit.client.0.smithi033.stdout:413/600: open fd = 3 2024-04-23T18:38:49.813 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.846 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:49.879 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:49.913 INFO:tasks.workunit.client.0.smithi033.stdout:414/600: open fd = 3 2024-04-23T18:38:49.976 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.013 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.046 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:50.080 INFO:tasks.workunit.client.0.smithi033.stdout:415/600: open fd = 3 2024-04-23T18:38:50.163 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.196 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.271 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:50.296 INFO:tasks.workunit.client.0.smithi033.stdout:416/600: open fd = 3 2024-04-23T18:38:50.336 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.363 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.396 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:50.507 INFO:tasks.workunit.client.0.smithi033.stdout:417/600: open fd = 3 2024-04-23T18:38:50.906 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:50.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.006 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:51.132 INFO:tasks.workunit.client.0.smithi033.stdout:418/600: open fd = 3 2024-04-23T18:38:51.163 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.305 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.330 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:51.364 INFO:tasks.workunit.client.0.smithi033.stdout:419/600: open fd = 3 2024-04-23T18:38:51.397 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.431 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.519 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:51.556 INFO:tasks.workunit.client.0.smithi033.stdout:420/600: open fd = 3 2024-04-23T18:38:51.606 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.656 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.781 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:51.831 INFO:tasks.workunit.client.0.smithi033.stdout:421/600: open fd = 3 2024-04-23T18:38:51.925 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:51.972 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.022 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:52.072 INFO:tasks.workunit.client.0.smithi033.stdout:422/600: open fd = 3 2024-04-23T18:38:52.123 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.184 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.231 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:52.281 INFO:tasks.workunit.client.0.smithi033.stdout:423/600: open fd = 3 2024-04-23T18:38:52.331 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.406 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.456 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:52.506 INFO:tasks.workunit.client.0.smithi033.stdout:424/600: open fd = 3 2024-04-23T18:38:52.556 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.606 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.657 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:52.782 INFO:tasks.workunit.client.0.smithi033.stdout:425/600: open fd = 3 2024-04-23T18:38:52.832 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.882 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:52.935 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:52.982 INFO:tasks.workunit.client.0.smithi033.stdout:426/600: open fd = 3 2024-04-23T18:38:53.099 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.158 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.207 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:53.257 INFO:tasks.workunit.client.0.smithi033.stdout:427/600: open fd = 3 2024-04-23T18:38:53.307 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.359 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.407 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:53.507 INFO:tasks.workunit.client.0.smithi033.stdout:428/600: open fd = 3 2024-04-23T18:38:53.557 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.607 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.657 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:53.707 INFO:tasks.workunit.client.0.smithi033.stdout:429/600: open fd = 3 2024-04-23T18:38:53.780 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.832 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:53.882 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:53.933 INFO:tasks.workunit.client.0.smithi033.stdout:430/600: open fd = 3 2024-04-23T18:38:53.982 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.033 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.174 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:54.224 INFO:tasks.workunit.client.0.smithi033.stdout:431/600: open fd = 3 2024-04-23T18:38:54.274 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.324 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.397 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:54.466 INFO:tasks.workunit.client.0.smithi033.stdout:432/600: open fd = 3 2024-04-23T18:38:54.499 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.549 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.599 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:54.650 INFO:tasks.workunit.client.0.smithi033.stdout:433/600: open fd = 3 2024-04-23T18:38:54.700 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.833 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:54.883 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:54.933 INFO:tasks.workunit.client.0.smithi033.stdout:434/600: open fd = 3 2024-04-23T18:38:55.080 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.246 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:55.320 INFO:tasks.workunit.client.0.smithi033.stdout:435/600: open fd = 3 2024-04-23T18:38:55.358 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.442 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.492 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:55.550 INFO:tasks.workunit.client.0.smithi033.stdout:436/600: open fd = 3 2024-04-23T18:38:55.633 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.683 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:38:55.733 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:38:55.855 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:31 smithi122 ceph-mon[32546]: pgmap v380: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:38:56.143 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:33 smithi122 ceph-mon[32546]: pgmap v381: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-23T18:38:56.201 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:35 smithi122 ceph-mon[32546]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:56.283 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:37 smithi122 ceph-mon[32546]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-04-23T18:38:56.334 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:39 smithi122 ceph-mon[32546]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:38:56.384 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:41 smithi122 ceph-mon[32546]: pgmap v385: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:38:56.434 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:31 smithi106 ceph-mon[32716]: pgmap v380: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:38:56.484 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:33 smithi106 ceph-mon[32716]: pgmap v381: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-23T18:38:56.617 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:35 smithi106 ceph-mon[32716]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:38:56.667 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:37 smithi106 ceph-mon[32716]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-04-23T18:38:56.717 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:39 smithi106 ceph-mon[32716]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:38:56.816 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:41 smithi106 ceph-mon[32716]: pgmap v385: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:38:56.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:38 smithi033 ceph-mon[29274]: pgmap v351: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:38:56.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:40 smithi033 ceph-mon[29274]: pgmap v352: 97 pgs: 97 active+clean; 9.5 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:38:56.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:42 smithi033 ceph-mon[29274]: pgmap v353: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:38:57.017 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:44 smithi033 ceph-mon[29274]: pgmap v354: 97 pgs: 97 active+clean; 9.6 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:38:57.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:45 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:57.117 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: pgmap v355: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 20 op/s 2024-04-23T18:38:57.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.267 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:57.318 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:57.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:46 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:43 smithi122 ceph-mon[32546]: pgmap v386: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-23T18:38:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:45 smithi122 ceph-mon[32546]: pgmap v387: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:38:57.518 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:47 smithi122 ceph-mon[32546]: pgmap v388: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-23T18:38:57.651 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:47 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:57.701 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.762 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.873 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:57.909 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: pgmap v389: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:38:57.962 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:58.010 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:58.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:58.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:50 smithi122 ceph-mon[32546]: pgmap v390: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:38:58.160 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:52 smithi122 ceph-mon[32546]: pgmap v391: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-23T18:38:58.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:54 smithi122 ceph-mon[32546]: pgmap v392: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-23T18:38:58.301 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:56 smithi122 ceph-mon[32546]: pgmap v393: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-04-23T18:38:58.366 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:43 smithi106 ceph-mon[32716]: pgmap v386: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-23T18:38:58.495 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:45 smithi106 ceph-mon[32716]: pgmap v387: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:38:58.559 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:47 smithi106 ceph-mon[32716]: pgmap v388: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-23T18:38:58.593 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:47 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:38:58.643 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:58.693 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:58.743 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:58.793 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:58.843 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: pgmap v389: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:38:58.902 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:38:58.993 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:38:59.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:38:59.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:50 smithi106 ceph-mon[32716]: pgmap v390: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:38:59.168 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:52 smithi106 ceph-mon[32716]: pgmap v391: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-23T18:38:59.202 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:54 smithi106 ceph-mon[32716]: pgmap v392: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-23T18:38:59.245 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:56 smithi106 ceph-mon[32716]: pgmap v393: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-04-23T18:38:59.287 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:38:58 smithi122 ceph-mon[32546]: pgmap v394: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:38:59.319 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:38:58 smithi106 ceph-mon[32716]: pgmap v394: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:38:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 -- 172.21.15.33:0/960306545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c101a90 msgr2=0x7f874c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:38:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 --2- 172.21.15.33:0/960306545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c101a90 0x7f874c105ad0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f873402a410 comp rx=0 tx=0).stop 2024-04-23T18:38:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 -- 172.21.15.33:0/960306545 shutdown_connections 2024-04-23T18:38:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 --2- 172.21.15.33:0/960306545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c1061b0 0x7f874c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 --2- 172.21.15.33:0/960306545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c101a90 0x7f874c105ad0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 --2- 172.21.15.33:0/960306545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1010e0 0x7f874c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:38:59.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 -- 172.21.15.33:0/960306545 >> 172.21.15.33:0/960306545 conn(0x7f874c0771d0 msgr2=0x7f874c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:38:59.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.256+0000 7f8750d76640 1 -- 172.21.15.33:0/960306545 shutdown_connections 2024-04-23T18:38:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.257+0000 7f8750d76640 1 -- 172.21.15.33:0/960306545 wait complete. 2024-04-23T18:38:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.257+0000 7f8750d76640 1 Processor -- start 2024-04-23T18:38:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 -- start start 2024-04-23T18:38:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:38:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1061b0 0x7f874c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 0x7f874c1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1110f0 con 0x7f874c1010e0 2024-04-23T18:39:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f874c110f70 con 0x7f874c197d00 2024-04-23T18:39:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8750d76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f874c111270 con 0x7f874c1061b0 2024-04-23T18:39:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51028/0 (socket says 172.21.15.33:51028) 2024-04-23T18:39:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 -- 172.21.15.33:0/1256533182 learned_addr learned my addr 172.21.15.33:0/1256533182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f8741d9b640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1061b0 0x7f874c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 0x7f874c1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 -- 172.21.15.33:0/1256533182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1061b0 msgr2=0x7f874c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1061b0 0x7f874c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 -- 172.21.15.33:0/1256533182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 msgr2=0x7f874c1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 0x7f874c1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874a59c640 1 -- 172.21.15.33:0/1256533182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874c1a6ec0 con 0x7f874c1010e0 2024-04-23T18:39:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.258+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 0x7f874c1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f874a59c640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f873c00d7a0 tx=0x7f873c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f8743fff640 1 -- 172.21.15.33:0/1256533182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c017070 con 0x7f874c1010e0 2024-04-23T18:39:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f8743fff640 1 -- 172.21.15.33:0/1256533182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c0054d0 con 0x7f874c1010e0 2024-04-23T18:39:01.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874c1a71b0 con 0x7f874c1010e0 2024-04-23T18:39:01.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f8743fff640 1 -- 172.21.15.33:0/1256533182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c004c60 con 0x7f874c1010e0 2024-04-23T18:39:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.259+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874c1a76d0 con 0x7f874c1010e0 2024-04-23T18:39:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.261+0000 7f8743fff640 1 -- 172.21.15.33:0/1256533182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f873c004e00 con 0x7f874c1010e0 2024-04-23T18:39:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.261+0000 7f8753b6f740 1 Processor -- start 2024-04-23T18:39:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 -- start start 2024-04-23T18:39:01.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 0x5593e0c2fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593e0b94970 con 0x5593e0c3a800 2024-04-23T18:39:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593e0b314a0 con 0x5593e0c2ff40 2024-04-23T18:39:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8753b6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593e0b90930 con 0x5593e0c2f620 2024-04-23T18:39:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8743fff640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 0x7f87380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51036/0 (socket says 172.21.15.33:51036) 2024-04-23T18:39:01.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 -- 172.21.15.33:0/164680725 learned_addr learned my addr 172.21.15.33:0/164680725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f874a59c640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f874ad9d640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 0x5593e0c2fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 0x7f87380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8743fff640 1 -- 172.21.15.33:0/1256533182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f873c0d0110 con 0x7f874c1010e0 2024-04-23T18:39:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 -- 172.21.15.33:0/164680725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 msgr2=0x5593e0c2fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 0x5593e0c2fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 -- 172.21.15.33:0/164680725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 msgr2=0x5593e0c3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f8741d9b640 1 -- 172.21.15.33:0/164680725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e08a7210 con 0x5593e0c3a800 2024-04-23T18:39:02.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f874ad9d640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 0x5593e0c2fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.262+0000 7f874a59c640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.263+0000 7f8741d9b640 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f874c06a570 tx=0x7f8734032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2abbdb8c7eb0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.263+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 0x7f87380a6300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8744007fd0 tx=0x7f87440072a0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.263+0000 7f8740d99640 1 -- 172.21.15.33:0/164680725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734040070 con 0x5593e0c3a800 2024-04-23T18:39:02.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.263+0000 7f8740d99640 1 -- 172.21.15.33:0/164680725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340316a0 con 0x5593e0c3a800 2024-04-23T18:39:02.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.263+0000 7f8740d99640 1 -- 172.21.15.33:0/164680725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734035040 con 0x5593e0c3a800 2024-04-23T18:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- 172.21.15.33:0/164680725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 msgr2=0x5593e0c3cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:03.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f874c06a570 tx=0x7f8734032df0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- 172.21.15.33:0/164680725 shutdown_connections 2024-04-23T18:39:03.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c3a800 0x5593e0c3cbf0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 --2- 172.21.15.33:0/164680725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2f620 0x5593e0c2fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- 172.21.15.33:0/164680725 >> 172.21.15.33:0/164680725 conn(0x5593e0c2bb30 msgr2=0x5593e0c2bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- 172.21.15.33:0/164680725 shutdown_connections 2024-04-23T18:39:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- 172.21.15.33:0/164680725 wait complete. 2024-04-23T18:39:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 -- start start 2024-04-23T18:39:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.264+0000 7f8753b6f740 1 Processor -- start 2024-04-23T18:39:03.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:03.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:03.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593e0c3d3d0 con 0x5593e0c2f620 2024-04-23T18:39:03.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5593e0b2f0b0 con 0x5593e0c2ff40 2024-04-23T18:39:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593e0c3d550 con 0x5593e0c3a800 2024-04-23T18:39:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 10 client.0 _ll_register_callbacks cb 0x5593e0b1cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51052/0 (socket says 172.21.15.33:51052) 2024-04-23T18:39:04.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 -- 172.21.15.33:0/1635794183 learned_addr learned my addr 172.21.15.33:0/1635794183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:04.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8741d9b640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 -- 172.21.15.33:0/1635794183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 msgr2=0x5593e0c379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c379b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:04.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 -- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 msgr2=0x5593e0c37450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c37450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 -- 172.21.15.33:0/1635794183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e0973620 con 0x5593e0c2f620 2024-04-23T18:39:04.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874ad9d640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:04.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f874a59c640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f873c09c0d0 tx=0x7f873c09bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2ad6d4ce547d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:04.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c09c140 con 0x5593e0c2f620 2024-04-23T18:39:04.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f873c0cdb70 con 0x5593e0c2f620 2024-04-23T18:39:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.265+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87340316a0 con 0x5593e0c2f620 2024-04-23T18:39:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f873c0a04f0 con 0x5593e0c2f620 2024-04-23T18:39:05.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 10 client.16989 fetch_fsmap learned FSMap version 35 2024-04-23T18:39:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593e0c493b0 con 0x5593e0c2f620 2024-04-23T18:39:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87340316a0 con 0x5593e0c2f620 2024-04-23T18:39:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f873c0b0be0 con 0x5593e0c2f620 2024-04-23T18:39:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 10 client.16989 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:39:05.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:05.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 20 client.16989 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0c4a870 0x5593e0c4cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:05.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 4 client.16989 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593e0ba2390 con 0x5593e0c4a870 2024-04-23T18:39:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.266+0000 7f8741d9b640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0c4a870 0x5593e0c4cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.267+0000 7f8741d9b640 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0c4a870 0x5593e0c4cc60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc7bb3de68dedb server_cookie=b8afd116dad40048 in_seq=0 out_seq=0 2024-04-23T18:39:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.267+0000 7f8731ffb640 10 client.16989 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.267+0000 7f8731ffb640 1 -- 172.21.15.33:0/1635794183 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593e0ba2390 con 0x5593e0c4a870 2024-04-23T18:39:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.267+0000 7f8731ffb640 10 client.16989 handle_command_reply: tid=0 2024-04-23T18:39:05.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.267+0000 7f8753b6f740 1 client.16989 shutdown 2024-04-23T18:39:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 msgr2=0x5593e0c3a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f873c09c0d0 tx=0x7f873c09bba0 comp rx=0 tx=0).stop 2024-04-23T18:39:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 shutdown_connections 2024-04-23T18:39:06.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0c4a870 0x5593e0c4cc60 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c37450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8731ffb640 0 client.16989 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 --2- 172.21.15.33:0/1635794183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c3a160 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 >> 172.21.15.33:0/1635794183 conn(0x5593e0c2bb30 msgr2=0x5593e0c31860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 shutdown_connections 2024-04-23T18:39:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 1 -- 172.21.15.33:0/1635794183 wait complete. 2024-04-23T18:39:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.268+0000 7f8753b6f740 20 client.16989 trim_cache size 0 max 16384 2024-04-23T18:39:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 Processor -- start 2024-04-23T18:39:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 -- start start 2024-04-23T18:39:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:06.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c39c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1afc30 con 0x5593e0c2f620 2024-04-23T18:39:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1b1770 con 0x5593e0c2ff40 2024-04-23T18:39:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8753b6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1b0300 con 0x5593e0c3a800 2024-04-23T18:39:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51064/0 (socket says 172.21.15.33:51064) 2024-04-23T18:39:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 -- 172.21.15.33:0/844100800 learned_addr learned my addr 172.21.15.33:0/844100800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:07.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874ad9d640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8741d9b640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c39c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 -- 172.21.15.33:0/844100800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 msgr2=0x5593e0c52500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c52500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 -- 172.21.15.33:0/844100800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 msgr2=0x5593e0c39c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c39c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874a59c640 1 -- 172.21.15.33:0/844100800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e0c37d20 con 0x5593e0c2f620 2024-04-23T18:39:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f874ad9d640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c52500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.283+0000 7f8741d9b640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f874a59c640 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f873c09be40 tx=0x7f873c00deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e5f569c30e9417 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8733fff640 1 -- 172.21.15.33:0/844100800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0a4920 con 0x5593e0c2f620 2024-04-23T18:39:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8733fff640 1 -- 172.21.15.33:0/844100800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c0a4db0 con 0x5593e0c2f620 2024-04-23T18:39:07.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8733fff640 1 -- 172.21.15.33:0/844100800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0cdb70 con 0x5593e0c2f620 2024-04-23T18:39:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 -- 172.21.15.33:0/844100800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 msgr2=0x5593e0c2b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f873c09be40 tx=0x7f873c00deb0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 -- 172.21.15.33:0/844100800 shutdown_connections 2024-04-23T18:39:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c3a800 0x5593e0c52500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c2ff40 0x5593e0c39c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 --2- 172.21.15.33:0/844100800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c2b430 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 -- 172.21.15.33:0/844100800 >> 172.21.15.33:0/844100800 conn(0x5593e0c2bb30 msgr2=0x5593e0c32af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 -- 172.21.15.33:0/844100800 shutdown_connections 2024-04-23T18:39:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.284+0000 7f8753b6f740 1 -- 172.21.15.33:0/844100800 wait complete. 2024-04-23T18:39:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 -- start start 2024-04-23T18:39:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 Processor -- start 2024-04-23T18:39:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2ff40 0x5593e0c3dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c3a800 0x5593e0c3e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1b10a0 con 0x5593e0c2f620 2024-04-23T18:39:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1b09d0 con 0x5593e0c3a800 2024-04-23T18:39:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1adc30 con 0x5593e0c2ff40 2024-04-23T18:39:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8753b6f740 10 client.0 _ll_register_callbacks cb 0x7f873c09bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51078/0 (socket says 172.21.15.33:51078) 2024-04-23T18:39:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 -- 172.21.15.33:0/2991513621 learned_addr learned my addr 172.21.15.33:0/2991513621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:08.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f8741d9b640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2ff40 0x5593e0c3dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:08.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 -- 172.21.15.33:0/2991513621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2ff40 msgr2=0x5593e0c3dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2ff40 0x5593e0c3dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:08.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 -- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c3a800 msgr2=0x5593e0c3e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c3a800 0x5593e0c3e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.285+0000 7f874a59c640 1 -- 172.21.15.33:0/2991513621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e0c3e850 con 0x5593e0c2f620 2024-04-23T18:39:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f874a59c640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f873c098550 tx=0x7f873c09bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c7984a84a37f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c0987b0 con 0x5593e0c2f620 2024-04-23T18:39:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f873c0a0120 con 0x5593e0c2f620 2024-04-23T18:39:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873c0a4db0 con 0x5593e0c2f620 2024-04-23T18:39:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f873c0aa050 con 0x5593e0c2f620 2024-04-23T18:39:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 10 client.16995 fetch_fsmap learned FSMap version 35 2024-04-23T18:39:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5593e0c3e850 con 0x5593e0c2f620 2024-04-23T18:39:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873c0a4db0 con 0x5593e0c2f620 2024-04-23T18:39:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f873c0a1680 con 0x5593e0c2f620 2024-04-23T18:39:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 10 client.16995 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:39:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 10 client.16995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 20 client.16995 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0cb0da0 0x5593e0cb3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 4 client.16995 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.286+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593e0b9c200 con 0x5593e0cb0da0 2024-04-23T18:39:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.287+0000 7f8741d9b640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0cb0da0 0x5593e0cb3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.287+0000 7f8731ffb640 10 client.16995 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.287+0000 7f8741d9b640 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0cb0da0 0x5593e0cb3190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b7f26260348e407 server_cookie=13604579dd61d8cb in_seq=0 out_seq=0 2024-04-23T18:39:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.294+0000 7f8731ffb640 1 -- 172.21.15.33:0/2991513621 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593e0b9c200 con 0x5593e0cb0da0 2024-04-23T18:39:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.294+0000 7f8731ffb640 10 client.16995 handle_command_reply: tid=0 2024-04-23T18:39:09.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 client.16995 shutdown 2024-04-23T18:39:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 msgr2=0x5593e0c42960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:09.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f873c098550 tx=0x7f873c09bfe0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 shutdown_connections 2024-04-23T18:39:10.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5593e0cb0da0 0x5593e0cb3190 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5593e0c3a800 0x5593e0c3e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e0c2ff40 0x5593e0c3dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8731ffb640 0 client.16995 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 --2- 172.21.15.33:0/2991513621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593e0c2f620 0x5593e0c42960 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 >> 172.21.15.33:0/2991513621 conn(0x5593e0c2bb30 msgr2=0x5593e0c2a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 shutdown_connections 2024-04-23T18:39:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 1 -- 172.21.15.33:0/2991513621 wait complete. 2024-04-23T18:39:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8753b6f740 20 client.16995 trim_cache size 0 max 16384 2024-04-23T18:39:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:39:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:39:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:39:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:39:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 msgr2=0x7f87380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:10.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 0x7f87380a6300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8744007fd0 tx=0x7f87440072a0 comp rx=0 tx=0).stop 2024-04-23T18:39:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 msgr2=0x7f874c197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.295+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f873c00d7a0 tx=0x7f873c00dc60 comp rx=0 tx=0).stop 2024-04-23T18:39:10.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 shutdown_connections 2024-04-23T18:39:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87380a3e40 0x7f87380a6300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:11.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f874c197d00 0x7f874c1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f874c1061b0 0x7f874c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 --2- 172.21.15.33:0/1256533182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f874c1010e0 0x7f874c197280 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:11.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 >> 172.21.15.33:0/1256533182 conn(0x7f874c0771d0 msgr2=0x7f874c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 shutdown_connections 2024-04-23T18:39:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:37:48.296+0000 7f8750d76640 1 -- 172.21.15.33:0/1256533182 wait complete. 2024-04-23T18:39:11.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:48 smithi033 ceph-mon[29274]: pgmap v356: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:39:11.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:50 smithi033 ceph-mon[29274]: pgmap v357: 97 pgs: 97 active+clean; 9.8 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-23T18:39:11.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:52 smithi033 ceph-mon[29274]: pgmap v358: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-04-23T18:39:11.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:54 smithi033 ceph-mon[29274]: pgmap v359: 97 pgs: 97 active+clean; 9.9 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-23T18:39:12.035 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:56 smithi033 ceph-mon[29274]: pgmap v360: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 21 op/s 2024-04-23T18:39:12.085 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:37:58 smithi033 ceph-mon[29274]: pgmap v361: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:39:12.151 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:00 smithi033 ceph-mon[29274]: pgmap v362: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:39:12.213 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:00 smithi122 ceph-mon[32546]: pgmap v395: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:39:12.260 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:02 smithi122 ceph-mon[32546]: pgmap v396: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-23T18:39:12.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:04 smithi122 ceph-mon[32546]: pgmap v397: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:39:12.360 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:06 smithi122 ceph-mon[32546]: pgmap v398: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:39:12.411 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:08 smithi122 ceph-mon[32546]: pgmap v399: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-04-23T18:39:12.461 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:09 smithi122 ceph-mon[32546]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:39:12.510 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:10 smithi122 ceph-mon[32546]: pgmap v401: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:39:12.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:00 smithi106 ceph-mon[32716]: pgmap v395: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:39:12.691 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:02 smithi106 ceph-mon[32716]: pgmap v396: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-23T18:39:12.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:04 smithi106 ceph-mon[32716]: pgmap v397: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:39:12.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:06 smithi106 ceph-mon[32716]: pgmap v398: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:39:12.852 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:08 smithi106 ceph-mon[32716]: pgmap v399: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-04-23T18:39:12.902 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:09 smithi106 ceph-mon[32716]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:39:12.994 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:10 smithi106 ceph-mon[32716]: pgmap v401: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:39:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 -- 172.21.15.33:0/3367619190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 msgr2=0x7f45d410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 --2- 172.21.15.33:0/3367619190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d410ee40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009ca0 tx=0x7f45c402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 -- 172.21.15.33:0/3367619190 shutdown_connections 2024-04-23T18:39:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 --2- 172.21.15.33:0/3367619190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d410ee40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 --2- 172.21.15.33:0/3367619190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4102b50 0x7f45d4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 --2- 172.21.15.33:0/3367619190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 -- 172.21.15.33:0/3367619190 >> 172.21.15.33:0/3367619190 conn(0x7f45d40fda00 msgr2=0x7f45d40ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 -- 172.21.15.33:0/3367619190 shutdown_connections 2024-04-23T18:39:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 -- 172.21.15.33:0/3367619190 wait complete. 2024-04-23T18:39:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.395+0000 7f45d98d9640 1 Processor -- start 2024-04-23T18:39:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 -- start start 2024-04-23T18:39:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4074cd0 0x7f45d41a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41121c0 con 0x7f45d4107270 2024-04-23T18:39:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112040 con 0x7f45d41021a0 2024-04-23T18:39:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d98d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112340 con 0x7f45d4074cd0 2024-04-23T18:39:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53858/0 (socket says 172.21.15.33:53858) 2024-04-23T18:39:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 -- 172.21.15.33:0/4110724574 learned_addr learned my addr 172.21.15.33:0/4110724574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d37fe640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4074cd0 0x7f45d41a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 -- 172.21.15.33:0/4110724574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4074cd0 msgr2=0x7f45d41a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:14.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4074cd0 0x7f45d41a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.396+0000 7f45d27fc640 1 -- 172.21.15.33:0/4110724574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 msgr2=0x7f45d4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:14.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d27fc640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d27fc640 1 -- 172.21.15.33:0/4110724574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d41a6f30 con 0x7f45d4107270 2024-04-23T18:39:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d27fc640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f45c8005fd0 tx=0x7f45c800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:14.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d88d7640 1 -- 172.21.15.33:0/4110724574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8017070 con 0x7f45d4107270 2024-04-23T18:39:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d88d7640 1 -- 172.21.15.33:0/4110724574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c8009c30 con 0x7f45d4107270 2024-04-23T18:39:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d41a7220 con 0x7f45d4107270 2024-04-23T18:39:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d88d7640 1 -- 172.21.15.33:0/4110724574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8003d30 con 0x7f45d4107270 2024-04-23T18:39:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.397+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d41a7770 con 0x7f45d4107270 2024-04-23T18:39:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.399+0000 7f45d88d7640 1 -- 172.21.15.33:0/4110724574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f45c8003ed0 con 0x7f45d4107270 2024-04-23T18:39:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 Processor -- start 2024-04-23T18:39:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 -- start start 2024-04-23T18:39:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d88d7640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 0x7f45bc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f1aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 0x555a54f27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a54e7f970 con 0x555a54f25800 2024-04-23T18:39:15.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555a54e1c4a0 con 0x555a54f1a620 2024-04-23T18:39:15.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555a54e7b930 con 0x555a54f1af40 2024-04-23T18:39:15.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d37fe640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 0x7f45bc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d88d7640 1 -- 172.21.15.33:0/4110724574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f45c80d0300 con 0x7f45d4107270 2024-04-23T18:39:15.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 0x555a54f27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 0x555a54f27bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53870/0 (socket says 172.21.15.33:53870) 2024-04-23T18:39:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d37fe640 1 -- 172.21.15.33:0/41275088 learned_addr learned my addr 172.21.15.33:0/41275088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d27fc640 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.400+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f1aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 -- 172.21.15.33:0/41275088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 msgr2=0x555a54f1aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f1aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 -- 172.21.15.33:0/41275088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 msgr2=0x555a54f27bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 0x555a54f27bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 -- 172.21.15.33:0/41275088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a54b92210 con 0x555a54f1af40 2024-04-23T18:39:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d37fe640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 0x7f45bc0a62b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45c402f500 tx=0x7f45c402f490 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45d27fc640 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f252c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cda40 tx=0x7f45c80b5cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=291802742a32befb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:15.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45b9ffb640 1 -- 172.21.15.33:0/41275088 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80a0640 con 0x555a54f1af40 2024-04-23T18:39:15.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45b9ffb640 1 -- 172.21.15.33:0/41275088 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80a4f00 con 0x555a54f1af40 2024-04-23T18:39:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.401+0000 7f45b9ffb640 1 -- 172.21.15.33:0/41275088 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8098900 con 0x555a54f1af40 2024-04-23T18:39:15.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- 172.21.15.33:0/41275088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 msgr2=0x555a54f252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:15.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f252c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cda40 tx=0x7f45c80b5cb0 comp rx=0 tx=0).stop 2024-04-23T18:39:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- 172.21.15.33:0/41275088 shutdown_connections 2024-04-23T18:39:16.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f25800 0x555a54f27bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f252c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/41275088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f1aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:16.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- 172.21.15.33:0/41275088 >> 172.21.15.33:0/41275088 conn(0x555a54f16b30 msgr2=0x555a54f16f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:16.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- 172.21.15.33:0/41275088 shutdown_connections 2024-04-23T18:39:16.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- 172.21.15.33:0/41275088 wait complete. 2024-04-23T18:39:16.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 -- start start 2024-04-23T18:39:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.402+0000 7f45dc5e0740 1 Processor -- start 2024-04-23T18:39:16.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:16.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f25800 0x555a54f22af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555a54f283d0 con 0x555a54f1af40 2024-04-23T18:39:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555a54e1a0b0 con 0x555a54f1a620 2024-04-23T18:39:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555a54f28550 con 0x555a54f25800 2024-04-23T18:39:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45dc5e0740 10 client.0 _ll_register_callbacks cb 0x555a54e07d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53878/0 (socket says 172.21.15.33:53878) 2024-04-23T18:39:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 -- 172.21.15.33:0/786881934 learned_addr learned my addr 172.21.15.33:0/786881934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d37fe640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f25800 0x555a54f22af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:17.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 -- 172.21.15.33:0/786881934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f25800 msgr2=0x555a54f22af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f25800 0x555a54f22af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 -- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 msgr2=0x555a54f251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 -- 172.21.15.33:0/786881934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a54c5e620 con 0x555a54f1af40 2024-04-23T18:39:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45d27fc640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cdaa0 tx=0x7f45c8097ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09701475151be17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c80a0640 con 0x555a54f1af40 2024-04-23T18:39:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.403+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f45c8098900 con 0x555a54f1af40 2024-04-23T18:39:17.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c80a4f00 con 0x555a54f1af40 2024-04-23T18:39:17.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f45c80b5d20 con 0x555a54f1af40 2024-04-23T18:39:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 10 client.17007 fetch_fsmap learned FSMap version 35 2024-04-23T18:39:17.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555a54f283d0 con 0x555a54f1af40 2024-04-23T18:39:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80a4f00 con 0x555a54f1af40 2024-04-23T18:39:17.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f45c809b6e0 con 0x555a54f1af40 2024-04-23T18:39:17.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 10 client.17007 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:39:17.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:18.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 20 client.17007 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:18.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f35500 0x555a54f378f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 4 client.17007 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:18.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a54e8d390 con 0x555a54f35500 2024-04-23T18:39:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.404+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f35500 0x555a54f378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.405+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f35500 0x555a54f378f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b3a622e9f45331 server_cookie=47eb4bfa31e841e in_seq=0 out_seq=0 2024-04-23T18:39:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.405+0000 7f45b2ffd640 10 client.17007 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.405+0000 7f45b2ffd640 1 -- 172.21.15.33:0/786881934 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a54e8d390 con 0x555a54f35500 2024-04-23T18:39:18.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.405+0000 7f45b2ffd640 10 client.17007 handle_command_reply: tid=0 2024-04-23T18:39:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 client.17007 shutdown 2024-04-23T18:39:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 msgr2=0x555a54f224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:18.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f45c80cdaa0 tx=0x7f45c8097ec0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 shutdown_connections 2024-04-23T18:39:18.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f35500 0x555a54f378f0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f25800 0x555a54f22af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1af40 0x555a54f224e0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45b2ffd640 0 client.17007 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/786881934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f1a620 0x555a54f251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 >> 172.21.15.33:0/786881934 conn(0x555a54f16b30 msgr2=0x555a54f1c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:19.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 shutdown_connections 2024-04-23T18:39:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 1 -- 172.21.15.33:0/786881934 wait complete. 2024-04-23T18:39:19.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.406+0000 7f45dc5e0740 20 client.17007 trim_cache size 0 max 16384 2024-04-23T18:39:19.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.421+0000 7f45dc5e0740 1 Processor -- start 2024-04-23T18:39:19.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.421+0000 7f45dc5e0740 1 -- start start 2024-04-23T18:39:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f21780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f24d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41afdb0 con 0x555a54f1a620 2024-04-23T18:39:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41b18f0 con 0x555a54f25800 2024-04-23T18:39:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41b0480 con 0x555a54f1af40 2024-04-23T18:39:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53888/0 (socket says 172.21.15.33:53888) 2024-04-23T18:39:19.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 -- 172.21.15.33:0/259193643 learned_addr learned my addr 172.21.15.33:0/259193643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d37fe640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f24d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 -- 172.21.15.33:0/259193643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 msgr2=0x555a54f21780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d27fc640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f21780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:19.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f21780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 -- 172.21.15.33:0/259193643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 msgr2=0x555a54f24d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f24d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 -- 172.21.15.33:0/259193643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a54f16940 con 0x555a54f1a620 2024-04-23T18:39:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d37fe640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:20.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x555a54f38020 tx=0x7f45cc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234cda0187c67071 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45b97fa640 1 -- 172.21.15.33:0/259193643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004220 con 0x555a54f1a620 2024-04-23T18:39:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45b97fa640 1 -- 172.21.15.33:0/259193643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45cc025030 con 0x555a54f1a620 2024-04-23T18:39:20.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.422+0000 7f45b97fa640 1 -- 172.21.15.33:0/259193643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004e20 con 0x555a54f1a620 2024-04-23T18:39:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- 172.21.15.33:0/259193643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 msgr2=0x555a54f1dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x555a54f38020 tx=0x7f45cc005f90 comp rx=0 tx=0).stop 2024-04-23T18:39:20.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- 172.21.15.33:0/259193643 shutdown_connections 2024-04-23T18:39:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f24d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f21780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/259193643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1dd30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- 172.21.15.33:0/259193643 >> 172.21.15.33:0/259193643 conn(0x555a54f16b30 msgr2=0x555a54f15cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- 172.21.15.33:0/259193643 shutdown_connections 2024-04-23T18:39:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- 172.21.15.33:0/259193643 wait complete. 2024-04-23T18:39:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.423+0000 7f45dc5e0740 1 -- start start 2024-04-23T18:39:20.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 Processor -- start 2024-04-23T18:39:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:21.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f1f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:21.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f1fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41b1220 con 0x555a54f1a620 2024-04-23T18:39:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41b0b50 con 0x555a54f25800 2024-04-23T18:39:21.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41addb0 con 0x555a54f1af40 2024-04-23T18:39:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45dc5e0740 10 client.0 _ll_register_callbacks cb 0x7f45c809bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53904/0 (socket says 172.21.15.33:53904) 2024-04-23T18:39:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 -- 172.21.15.33:0/2806869002 learned_addr learned my addr 172.21.15.33:0/2806869002 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d37fe640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f1fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d27fc640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f1f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:21.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 -- 172.21.15.33:0/2806869002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 msgr2=0x555a54f1f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:21.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f1f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:21.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 -- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 msgr2=0x555a54f1fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f1fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.424+0000 7f45d2ffd640 1 -- 172.21.15.33:0/2806869002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a54f9a8f0 con 0x555a54f1a620 2024-04-23T18:39:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45d2ffd640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f45cc012280 tx=0x7f45cc004930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7a2691bb0d5526 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:22.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004da0 con 0x555a54f1a620 2024-04-23T18:39:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8542+0+0 (secure 0 0 0) 0x7f45cc0108b0 con 0x555a54f1a620 2024-04-23T18:39:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc025030 con 0x555a54f1a620 2024-04-23T18:39:22.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f45cc010ab0 con 0x555a54f1a620 2024-04-23T18:39:22.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 10 client.17013 fetch_fsmap learned FSMap version 35 2024-04-23T18:39:22.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555a54f9bb50 con 0x555a54f1a620 2024-04-23T18:39:22.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45cc025030 con 0x555a54f1a620 2024-04-23T18:39:22.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f45cc005040 con 0x555a54f1a620 2024-04-23T18:39:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 10 client.17013 fetch_fsmap finished waiting for FSMap version 35 2024-04-23T18:39:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 20 client.17013 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f9c270 0x555a54f9e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:22.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 4 client.17013 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.425+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a54e87200 con 0x555a54f9c270 2024-04-23T18:39:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.426+0000 7f45d27fc640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f9c270 0x555a54f9e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:22.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.426+0000 7f45b2ffd640 10 client.17013 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:01.426+0000 7f45d27fc640 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f9c270 0x555a54f9e690 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52875f8421038368 server_cookie=297db187502e13e1 in_seq=0 out_seq=0 2024-04-23T18:39:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45b2ffd640 1 -- 172.21.15.33:0/2806869002 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a54e87200 con 0x555a54f9c270 2024-04-23T18:39:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45b2ffd640 10 client.17013 handle_command_reply: tid=0 2024-04-23T18:39:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 client.17013 shutdown 2024-04-23T18:39:22.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 msgr2=0x555a54f1f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f45cc012280 tx=0x7f45cc004930 comp rx=0 tx=0).stop 2024-04-23T18:39:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 shutdown_connections 2024-04-23T18:39:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555a54f9c270 0x555a54f9e690 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a54f25800 0x555a54f1fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a54f1af40 0x555a54f1f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45b2ffd640 0 client.17013 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 --2- 172.21.15.33:0/2806869002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555a54f1a620 0x555a54f1f170 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 >> 172.21.15.33:0/2806869002 conn(0x555a54f16b30 msgr2=0x555a54f1bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:23.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 shutdown_connections 2024-04-23T18:39:23.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 1 -- 172.21.15.33:0/2806869002 wait complete. 2024-04-23T18:39:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.177+0000 7f45dc5e0740 20 client.17013 trim_cache size 0 max 16384 2024-04-23T18:39:23.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:39:23.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:39:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:39:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:39:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 msgr2=0x7f45bc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:23.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 0x7f45bc0a62b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f45c402f500 tx=0x7f45c402f490 comp rx=0 tx=0).stop 2024-04-23T18:39:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 msgr2=0x7f45d4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:23.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f45c8005fd0 tx=0x7f45c800dde0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 shutdown_connections 2024-04-23T18:39:23.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f45bc0a3df0 0x7f45bc0a62b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45d4074cd0 0x7f45d41a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d4107270 0x7f45d4074790 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 --2- 172.21.15.33:0/4110724574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45d41021a0 0x7f45d4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 >> 172.21.15.33:0/4110724574 conn(0x7f45d40fda00 msgr2=0x7f45d40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 shutdown_connections 2024-04-23T18:39:24.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:03.178+0000 7f45d98d9640 1 -- 172.21.15.33:0/4110724574 wait complete. 2024-04-23T18:39:24.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:02 smithi033 ceph-mon[29274]: pgmap v363: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 81 KiB/s wr, 37 op/s 2024-04-23T18:39:24.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:04 smithi033 ceph-mon[29274]: pgmap v364: 97 pgs: 97 active+clean; 10 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 67 KiB/s wr, 29 op/s 2024-04-23T18:39:24.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:04 smithi033 ceph-mon[29274]: osdmap e103: 12 total, 12 up, 12 in 2024-04-23T18:39:24.512 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:06 smithi033 ceph-mon[29274]: pgmap v366: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:39:24.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:08 smithi033 ceph-mon[29274]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:39:24.627 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:10 smithi033 ceph-mon[29274]: pgmap v368: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 86 KiB/s wr, 38 op/s 2024-04-23T18:39:24.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:10 smithi033 ceph-mon[29274]: osdmap e104: 12 total, 12 up, 12 in 2024-04-23T18:39:24.727 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:12 smithi033 ceph-mon[29274]: pgmap v370: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:39:24.777 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:14 smithi033 ceph-mon[29274]: pgmap v371: 97 pgs: 97 active+clean; 10 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 17 op/s 2024-04-23T18:39:24.879 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:12 smithi122 ceph-mon[32546]: pgmap v402: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:39:24.927 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:14 smithi122 ceph-mon[32546]: pgmap v403: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:39:24.994 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:16 smithi122 ceph-mon[32546]: pgmap v404: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:39:25.044 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:16 smithi122 ceph-mon[32546]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:39:25.121 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:18 smithi122 ceph-mon[32546]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:39:25.215 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:20 smithi122 ceph-mon[32546]: pgmap v407: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:39:25.269 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:22 smithi122 ceph-mon[32546]: pgmap v408: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:39:25.319 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:12 smithi106 ceph-mon[32716]: pgmap v402: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:39:25.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:14 smithi106 ceph-mon[32716]: pgmap v403: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:39:25.419 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:16 smithi106 ceph-mon[32716]: pgmap v404: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:39:25.469 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:16 smithi106 ceph-mon[32716]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:39:25.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:18 smithi106 ceph-mon[32716]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:39:25.610 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:20 smithi106 ceph-mon[32716]: pgmap v407: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:39:25.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:22 smithi106 ceph-mon[32716]: pgmap v408: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:39:25.712 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:24 smithi122 ceph-mon[32546]: pgmap v409: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:39:25.786 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:24 smithi106 ceph-mon[32716]: pgmap v409: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:39:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- 172.21.15.33:0/4070944971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 msgr2=0x7f93d0101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 --2- 172.21.15.33:0/4070944971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d0101160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009860 tx=0x7f93c002a340 comp rx=0 tx=0).stop 2024-04-23T18:39:26.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- 172.21.15.33:0/4070944971 shutdown_connections 2024-04-23T18:39:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 --2- 172.21.15.33:0/4070944971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d0110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 --2- 172.21.15.33:0/4070944971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 0x7f93d010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 --2- 172.21.15.33:0/4070944971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d0101160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- 172.21.15.33:0/4070944971 >> 172.21.15.33:0/4070944971 conn(0x7f93d0077fd0 msgr2=0x7f93d00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- 172.21.15.33:0/4070944971 shutdown_connections 2024-04-23T18:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- 172.21.15.33:0/4070944971 wait complete. 2024-04-23T18:39:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 Processor -- start 2024-04-23T18:39:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.281+0000 7f93d71e2640 1 -- start start 2024-04-23T18:39:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:26.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 0x7f93d01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:26.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01143e0 con 0x7f93d010e930 2024-04-23T18:39:26.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0114260 con 0x7f93d0100d80 2024-04-23T18:39:26.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d71e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93d0114560 con 0x7f93d01016a0 2024-04-23T18:39:26.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:27.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d4f7e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36780/0 (socket says 172.21.15.33:36780) 2024-04-23T18:39:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 -- 172.21.15.33:0/2397934717 learned_addr learned my addr 172.21.15.33:0/2397934717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93cffff640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 0x7f93d01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:27.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 -- 172.21.15.33:0/2397934717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 msgr2=0x7f93d01a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 0x7f93d01a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 -- 172.21.15.33:0/2397934717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 msgr2=0x7f93d01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 -- 172.21.15.33:0/2397934717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d019b060 con 0x7f93d010e930 2024-04-23T18:39:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.282+0000 7f93d577f640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93c800c3f0 tx=0x7f93c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.283+0000 7f93cdffb640 1 -- 172.21.15.33:0/2397934717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8003970 con 0x7f93d010e930 2024-04-23T18:39:27.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.283+0000 7f93cdffb640 1 -- 172.21.15.33:0/2397934717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c800b690 con 0x7f93d010e930 2024-04-23T18:39:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.283+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d019b350 con 0x7f93d010e930 2024-04-23T18:39:27.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.283+0000 7f93cdffb640 1 -- 172.21.15.33:0/2397934717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004380 con 0x7f93d010e930 2024-04-23T18:39:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.283+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d01a7610 con 0x7f93d010e930 2024-04-23T18:39:27.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.285+0000 7f93cdffb640 1 -- 172.21.15.33:0/2397934717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93c801d020 con 0x7f93d010e930 2024-04-23T18:39:27.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 Processor -- start 2024-04-23T18:39:28.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93cdffb640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 0x7f93b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 -- start start 2024-04-23T18:39:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93cdffb640 1 -- 172.21.15.33:0/2397934717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f93c80d05b0 con 0x7f93d010e930 2024-04-23T18:39:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93cffff640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 0x7f93b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a4fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b13a458970 con 0x55b13a4fe800 2024-04-23T18:39:28.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b13a3f54a0 con 0x55b13a4f3f40 2024-04-23T18:39:28.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b13a454930 con 0x55b13a4f3620 2024-04-23T18:39:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36788/0 (socket says 172.21.15.33:36788) 2024-04-23T18:39:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 -- 172.21.15.33:0/2467831837 learned_addr learned my addr 172.21.15.33:0/2467831837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:28.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93cffff640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a4fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93cffff640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 0x7f93b80a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93d019bfa0 tx=0x7f93c4011440 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 -- 172.21.15.33:0/2467831837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 msgr2=0x55b13a4f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 -- 172.21.15.33:0/2467831837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 msgr2=0x55b13a4fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a4fe2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d577f640 1 -- 172.21.15.33:0/2467831837 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13a16b210 con 0x55b13a4fe800 2024-04-23T18:39:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.286+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93d577f640 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f93c80a4c40 tx=0x7f93c809c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652408572cb09e23 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:29.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93b6ffd640 1 -- 172.21.15.33:0/2467831837 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80a5170 con 0x55b13a4fe800 2024-04-23T18:39:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93b6ffd640 1 -- 172.21.15.33:0/2467831837 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c800bb40 con 0x55b13a4fe800 2024-04-23T18:39:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93b6ffd640 1 -- 172.21.15.33:0/2467831837 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c80a4ce0 con 0x55b13a4fe800 2024-04-23T18:39:29.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93d9fd3740 1 -- 172.21.15.33:0/2467831837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 msgr2=0x55b13a500bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:29.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.287+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f93c80a4c40 tx=0x7f93c809c500 comp rx=0 tx=0).stop 2024-04-23T18:39:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- 172.21.15.33:0/2467831837 shutdown_connections 2024-04-23T18:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4fe800 0x55b13a500bf0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a4fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/2467831837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:29.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- 172.21.15.33:0/2467831837 >> 172.21.15.33:0/2467831837 conn(0x55b13a4efb30 msgr2=0x55b13a4eff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:29.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- 172.21.15.33:0/2467831837 shutdown_connections 2024-04-23T18:39:29.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- 172.21.15.33:0/2467831837 wait complete. 2024-04-23T18:39:29.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- start start 2024-04-23T18:39:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 Processor -- start 2024-04-23T18:39:29.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 0x55b13a4fbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.288+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b13a5013d0 con 0x55b13a4f3f40 2024-04-23T18:39:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b13a3f30b0 con 0x55b13a4fe800 2024-04-23T18:39:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b13a501550 con 0x55b13a4f3620 2024-04-23T18:39:30.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d9fd3740 10 client.0 _ll_register_callbacks cb 0x55b13a3e0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:30.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:30.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d4f7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36796/0 (socket says 172.21.15.33:36796) 2024-04-23T18:39:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 -- 172.21.15.33:0/16937594 learned_addr learned my addr 172.21.15.33:0/16937594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:30.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d577f640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 0x55b13a4fbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 -- 172.21.15.33:0/16937594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 msgr2=0x55b13a4fe1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:30.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4fe1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 -- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 msgr2=0x55b13a4fbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:30.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 0x55b13a4fbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 -- 172.21.15.33:0/16937594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13a237620 con 0x55b13a4f3f40 2024-04-23T18:39:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d577f640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 0x55b13a4fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93cffff640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f93c400fa90 tx=0x7f93c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b093b79fa1ac74eb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:30.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93affff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:31.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4017040 con 0x55b13a4f3f40 2024-04-23T18:39:31.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f93c401e8e0 con 0x55b13a4f3f40 2024-04-23T18:39:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.289+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c800bb40 con 0x55b13a4f3f40 2024-04-23T18:39:31.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93c401eae0 con 0x55b13a4f3f40 2024-04-23T18:39:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 10 client.17028 fetch_fsmap learned FSMap version 36 2024-04-23T18:39:31.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b13a50d3c0 con 0x55b13a4f3f40 2024-04-23T18:39:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c800bb40 con 0x55b13a4f3f40 2024-04-23T18:39:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f93c4025070 con 0x55b13a4f3f40 2024-04-23T18:39:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 10 client.17028 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:39:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:31.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 20 client.17028 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a50e940 0x55b13a510d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:31.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 4 client.17028 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b13a466390 con 0x55b13a50e940 2024-04-23T18:39:31.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.290+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a50e940 0x55b13a510d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:31.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93affff640 10 client.17028 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:31.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a50e940 0x55b13a510d30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c53b30604e5033 server_cookie=9cdf42f517cf67ec in_seq=0 out_seq=0 2024-04-23T18:39:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93affff640 1 -- 172.21.15.33:0/16937594 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b13a466390 con 0x55b13a50e940 2024-04-23T18:39:32.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93affff640 10 client.17028 handle_command_reply: tid=0 2024-04-23T18:39:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93d9fd3740 1 client.17028 shutdown 2024-04-23T18:39:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 msgr2=0x55b13a4fb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f93c400fa90 tx=0x7f93c4004730 comp rx=0 tx=0).stop 2024-04-23T18:39:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.291+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 shutdown_connections 2024-04-23T18:39:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a50e940 0x55b13a510d30 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4fe800 0x55b13a4fbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fb4c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93affff640 0 client.17028 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/16937594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4f3620 0x55b13a4fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:32.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 >> 172.21.15.33:0/16937594 conn(0x55b13a4efb30 msgr2=0x55b13a4f5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:32.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 shutdown_connections 2024-04-23T18:39:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 1 -- 172.21.15.33:0/16937594 wait complete. 2024-04-23T18:39:32.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.292+0000 7f93d9fd3740 20 client.17028 trim_cache size 0 max 16384 2024-04-23T18:39:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 Processor -- start 2024-04-23T18:39:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 -- start start 2024-04-23T18:39:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 0x55b13a50a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a5165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01afd50 con 0x55b13a4f3f40 2024-04-23T18:39:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b1890 con 0x55b13a4f3620 2024-04-23T18:39:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b0420 con 0x55b13a4fe800 2024-04-23T18:39:33.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36812/0 (socket says 172.21.15.33:36812) 2024-04-23T18:39:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d4f7e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 0x55b13a50a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:33.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 -- 172.21.15.33:0/3943030291 learned_addr learned my addr 172.21.15.33:0/3943030291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:33.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d577f640 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a5165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 -- 172.21.15.33:0/3943030291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 msgr2=0x55b13a5165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a5165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 -- 172.21.15.33:0/3943030291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 msgr2=0x55b13a50a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 0x55b13a50a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93cffff640 1 -- 172.21.15.33:0/3943030291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13a5110a0 con 0x55b13a4f3f40 2024-04-23T18:39:33.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.307+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 0x55b13a50a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93cffff640 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009800 tx=0x7f93c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e25a9265c73f5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:34.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93b67fc640 1 -- 172.21.15.33:0/3943030291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4033070 con 0x55b13a4f3f40 2024-04-23T18:39:34.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93b67fc640 1 -- 172.21.15.33:0/3943030291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c4019050 con 0x55b13a4f3f40 2024-04-23T18:39:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93b67fc640 1 -- 172.21.15.33:0/3943030291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4017070 con 0x55b13a4f3f40 2024-04-23T18:39:34.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 -- 172.21.15.33:0/3943030291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 msgr2=0x55b13a4fdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009800 tx=0x7f93c4000bc0 comp rx=0 tx=0).stop 2024-04-23T18:39:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 -- 172.21.15.33:0/3943030291 shutdown_connections 2024-04-23T18:39:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a5165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3f40 0x55b13a4fdd00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/3943030291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3620 0x55b13a50a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:34.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 -- 172.21.15.33:0/3943030291 >> 172.21.15.33:0/3943030291 conn(0x55b13a4efb30 msgr2=0x55b13a4f6af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.308+0000 7f93d9fd3740 1 -- 172.21.15.33:0/3943030291 shutdown_connections 2024-04-23T18:39:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 -- 172.21.15.33:0/3943030291 wait complete. 2024-04-23T18:39:34.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 -- start start 2024-04-23T18:39:34.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 Processor -- start 2024-04-23T18:39:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a5028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a502e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b11c0 con 0x55b13a4f3620 2024-04-23T18:39:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01b0af0 con 0x55b13a4f3f40 2024-04-23T18:39:35.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93d01add30 con 0x55b13a4fe800 2024-04-23T18:39:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d9fd3740 10 client.0 _ll_register_callbacks cb 0x7f93c400f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d4f7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:35.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a5028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d4f7e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36814/0 (socket says 172.21.15.33:36814) 2024-04-23T18:39:35.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.309+0000 7f93d4f7e640 1 -- 172.21.15.33:0/357756545 learned_addr learned my addr 172.21.15.33:0/357756545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d577f640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a502e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 -- 172.21.15.33:0/357756545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 msgr2=0x55b13a502e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a502e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:35.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 -- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 msgr2=0x55b13a5028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:35.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a5028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 -- 172.21.15.33:0/357756545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13a4fa470 con 0x55b13a4f3620 2024-04-23T18:39:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93cffff640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a5028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:35.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d4f7e640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f93c002a930 tx=0x7f93c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d967550ab866a5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:35.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93affff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0037040 con 0x55b13a4f3620 2024-04-23T18:39:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f93c0004300 con 0x55b13a4f3620 2024-04-23T18:39:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c4019050 con 0x55b13a4f3620 2024-04-23T18:39:36.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f93c0055050 con 0x55b13a4f3620 2024-04-23T18:39:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d9fd3740 10 client.17034 fetch_fsmap learned FSMap version 36 2024-04-23T18:39:36.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b13a575120 con 0x55b13a4f3620 2024-04-23T18:39:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.310+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c4019050 con 0x55b13a4f3620 2024-04-23T18:39:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f93c0050030 con 0x55b13a4f3620 2024-04-23T18:39:36.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 10 client.17034 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:39:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 10 client.17034 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 20 client.17034 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:36.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a575890 0x55b13a577cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 4 client.17034 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:36.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b13a460200 con 0x55b13a575890 2024-04-23T18:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93cffff640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a575890 0x55b13a577cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93affff640 10 client.17034 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.311+0000 7f93cffff640 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a575890 0x55b13a577cb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bdb1e912e2ff92 server_cookie=b0c69e4f3c6ef25a in_seq=0 out_seq=0 2024-04-23T18:39:36.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93affff640 1 -- 172.21.15.33:0/357756545 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b13a460200 con 0x55b13a575890 2024-04-23T18:39:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93affff640 10 client.17034 handle_command_reply: tid=0 2024-04-23T18:39:37.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 client.17034 shutdown 2024-04-23T18:39:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 msgr2=0x55b13a5023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f93c002a930 tx=0x7f93c0000bc0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 shutdown_connections 2024-04-23T18:39:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b13a575890 0x55b13a577cb0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b13a4fe800 0x55b13a502e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b13a4f3f40 0x55b13a5028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 --2- 172.21.15.33:0/357756545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b13a4f3620 0x55b13a5023b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 >> 172.21.15.33:0/357756545 conn(0x55b13a4efb30 msgr2=0x55b13a4f4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93affff640 0 client.17034 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 shutdown_connections 2024-04-23T18:39:37.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 1 -- 172.21.15.33:0/357756545 wait complete. 2024-04-23T18:39:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.320+0000 7f93d9fd3740 20 client.17034 trim_cache size 0 max 16384 2024-04-23T18:39:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:39:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:39:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:39:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:39:38.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 msgr2=0x7f93b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 0x7f93b80a6300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f93d019bfa0 tx=0x7f93c4011440 comp rx=0 tx=0).stop 2024-04-23T18:39:38.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 msgr2=0x7f93d019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:38.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f93c800c3f0 tx=0x7f93c800c8b0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 shutdown_connections 2024-04-23T18:39:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93b80a3e40 0x7f93b80a6300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93d010e930 0x7f93d019a7a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d01016a0 0x7f93d01a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 --2- 172.21.15.33:0/2397934717 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93d0100d80 0x7f93d01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 >> 172.21.15.33:0/2397934717 conn(0x7f93d0077fd0 msgr2=0x7f93d010fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 shutdown_connections 2024-04-23T18:39:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:16.321+0000 7f93d71e2640 1 -- 172.21.15.33:0/2397934717 wait complete. 2024-04-23T18:39:38.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:15 smithi033 ceph-mon[29274]: daemon mds.f restarted 2024-04-23T18:39:38.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:15 smithi033 ceph-mon[29274]: osdmap e105: 12 total, 12 up, 12 in 2024-04-23T18:39:38.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/3654262177,v1:172.21.15.122:6835/3654262177] up:boot 2024-04-23T18:39:39.038 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:15 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-23T18:39:39.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-23T18:39:39.138 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:16 smithi033 ceph-mon[29274]: pgmap v372: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:39:39.188 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:18 smithi033 ceph-mon[29274]: pgmap v374: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-23T18:39:39.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:20 smithi033 ceph-mon[29274]: pgmap v375: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 17 KiB/s wr, 7 op/s 2024-04-23T18:39:39.299 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:22 smithi033 ceph-mon[29274]: pgmap v376: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:39:39.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:25 smithi033 ceph-mon[29274]: pgmap v377: 97 pgs: 97 active+clean; 11 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-23T18:39:39.438 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:27 smithi033 ceph-mon[29274]: pgmap v378: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 32 KiB/s wr, 18 op/s 2024-04-23T18:39:39.488 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:26 smithi122 ceph-mon[32546]: pgmap v410: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:39:39.538 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:28 smithi122 ceph-mon[32546]: pgmap v411: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:39:39.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:30 smithi122 ceph-mon[32546]: pgmap v412: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:39.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:32 smithi122 ceph-mon[32546]: pgmap v413: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:39.755 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:34 smithi122 ceph-mon[32546]: pgmap v414: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:39.805 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:36 smithi122 ceph-mon[32546]: pgmap v415: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:39:39.863 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:26 smithi106 ceph-mon[32716]: pgmap v410: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:39:39.905 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:28 smithi106 ceph-mon[32716]: pgmap v411: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:39:39.947 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:30 smithi106 ceph-mon[32716]: pgmap v412: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:39.980 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:32 smithi106 ceph-mon[32716]: pgmap v413: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:40.013 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:34 smithi106 ceph-mon[32716]: pgmap v414: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:40.047 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:36 smithi106 ceph-mon[32716]: pgmap v415: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:39:40.098 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:38 smithi122 ceph-mon[32546]: pgmap v416: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:40.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:38 smithi106 ceph-mon[32716]: pgmap v416: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 -- 172.21.15.33:0/3460373870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba8101a90 msgr2=0x7fcba8105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/3460373870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba8101a90 0x7fcba8105ad0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009900 tx=0x7fcb9002a410 comp rx=0 tx=0).stop 2024-04-23T18:39:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 -- 172.21.15.33:0/3460373870 shutdown_connections 2024-04-23T18:39:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/3460373870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 0x7fcba810dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/3460373870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba8101a90 0x7fcba8105ad0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/3460373870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 -- 172.21.15.33:0/3460373870 >> 172.21.15.33:0/3460373870 conn(0x7fcba80771d0 msgr2=0x7fcba80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:40.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 -- 172.21.15.33:0/3460373870 shutdown_connections 2024-04-23T18:39:40.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.423+0000 7fcbacc7b640 1 -- 172.21.15.33:0/3460373870 wait complete. 2024-04-23T18:39:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 Processor -- start 2024-04-23T18:39:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 -- start start 2024-04-23T18:39:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba8197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 0x7fcba8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81110d0 con 0x7fcba819be10 2024-04-23T18:39:40.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8110f50 con 0x7fcba81061b0 2024-04-23T18:39:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.424+0000 7fcbacc7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8111250 con 0x7fcba81010e0 2024-04-23T18:39:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba8197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35310/0 (socket says 172.21.15.33:35310) 2024-04-23T18:39:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 -- 172.21.15.33:0/1433202184 learned_addr learned my addr 172.21.15.33:0/1433202184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:41.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 0x7fcba8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 -- 172.21.15.33:0/1433202184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 msgr2=0x7fcba8197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba8197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 -- 172.21.15.33:0/1433202184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 msgr2=0x7fcba8197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:41.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 0x7fcba8197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 -- 172.21.15.33:0/1433202184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819c900 con 0x7fcba819be10 2024-04-23T18:39:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba659c640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:39:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcba6d9d640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fcba000eb10 tx=0x7fcba000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:41.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcb9f7fe640 1 -- 172.21.15.33:0/1433202184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0004b20 con 0x7fcba819be10 2024-04-23T18:39:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcb9f7fe640 1 -- 172.21.15.33:0/1433202184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba0004cc0 con 0x7fcba819be10 2024-04-23T18:39:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcb9f7fe640 1 -- 172.21.15.33:0/1433202184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0010670 con 0x7fcba819be10 2024-04-23T18:39:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819cbf0 con 0x7fcba819be10 2024-04-23T18:39:41.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.425+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a4360 con 0x7fcba819be10 2024-04-23T18:39:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcb9f7fe640 1 -- 172.21.15.33:0/1433202184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcba0005020 con 0x7fcba819be10 2024-04-23T18:39:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 Processor -- start 2024-04-23T18:39:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 -- start start 2024-04-23T18:39:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cbb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cbba10 0x561d20cbde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d20c13970 con 0x561d20cbba10 2024-04-23T18:39:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561d20bb04a0 con 0x561d20caede0 2024-04-23T18:39:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d20c0f930 con 0x561d20cae4c0 2024-04-23T18:39:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.428+0000 7fcb9f7fe640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 0x7fcb940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:42.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcba659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cbb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcba6d9d640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 0x7fcb940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:42.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:43.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:47246/0 (socket says 172.21.15.33:47246) 2024-04-23T18:39:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3999368797 learned_addr learned my addr 172.21.15.33:0/3999368797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcba659c640 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cbb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:41406/0 (socket says 172.21.15.33:41406) 2024-04-23T18:39:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cbba10 0x561d20cbde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9f7fe640 1 -- 172.21.15.33:0/1433202184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fcba00d0cf0 con 0x7fcba819be10 2024-04-23T18:39:43.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3999368797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 msgr2=0x561d20cbb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cbb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3999368797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cbba10 msgr2=0x561d20cbde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cbba10 0x561d20cbde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:43.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3999368797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d20926210 con 0x561d20cae4c0 2024-04-23T18:39:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcba6d9d640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 0x7fcb940a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcba00ce500 tx=0x7fcba006e6e0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcba806a570 tx=0x7fcb90005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e3812b4cddf2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.429+0000 7fcb9cff9640 1 -- 172.21.15.33:0/3999368797 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90035030 con 0x561d20cae4c0 2024-04-23T18:39:43.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcb9cff9640 1 -- 172.21.15.33:0/3999368797 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb900317e0 con 0x561d20cae4c0 2024-04-23T18:39:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcb9cff9640 1 -- 172.21.15.33:0/3999368797 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb900038e0 con 0x561d20cae4c0 2024-04-23T18:39:43.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3999368797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 msgr2=0x561d20cae8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fcba806a570 tx=0x7fcb90005780 comp rx=0 tx=0).stop 2024-04-23T18:39:44.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3999368797 shutdown_connections 2024-04-23T18:39:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cbba10 0x561d20cbde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cbb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3999368797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cae8a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3999368797 >> 172.21.15.33:0/3999368797 conn(0x561d20caab30 msgr2=0x561d20caaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3999368797 shutdown_connections 2024-04-23T18:39:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.430+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3999368797 wait complete. 2024-04-23T18:39:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 -- start start 2024-04-23T18:39:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 Processor -- start 2024-04-23T18:39:44.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cb6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d20cb1440 con 0x561d20cae4c0 2024-04-23T18:39:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561d20bae0b0 con 0x561d20caede0 2024-04-23T18:39:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d20cb15c0 con 0x561d20cbba10 2024-04-23T18:39:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcbaf98b740 10 client.0 _ll_register_callbacks cb 0x561d20b9bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:44.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcb9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35322/0 (socket says 172.21.15.33:35322) 2024-04-23T18:39:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3920754872 learned_addr learned my addr 172.21.15.33:0/3920754872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcba659c640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cb6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.431+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3920754872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 msgr2=0x561d20cb7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 msgr2=0x561d20cb6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cb6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 -- 172.21.15.33:0/3920754872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d209f2620 con 0x561d20cae4c0 2024-04-23T18:39:45.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcb90032c20 tx=0x7fcb90032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c084c3b39688c25a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:45.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90053060 con 0x561d20cae4c0 2024-04-23T18:39:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fcb9004c680 con 0x561d20cae4c0 2024-04-23T18:39:45.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb900317e0 con 0x561d20cae4c0 2024-04-23T18:39:45.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcb90035070 con 0x561d20cae4c0 2024-04-23T18:39:45.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcbaf98b740 10 client.17046 fetch_fsmap learned FSMap version 36 2024-04-23T18:39:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d20cc8380 con 0x561d20cae4c0 2024-04-23T18:39:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.432+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb900317e0 con 0x561d20cae4c0 2024-04-23T18:39:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fcb90057830 con 0x561d20cae4c0 2024-04-23T18:39:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 10 client.17046 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:39:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 20 client.17046 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20cc96d0 0x561d20ccbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:46.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 4 client.17046 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d20c21390 con 0x561d20cc96d0 2024-04-23T18:39:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcba659c640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20cc96d0 0x561d20ccbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcba659c640 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20cc96d0 0x561d20ccbac0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e390f9e0f7e16fa server_cookie=1da399c648fb66ef in_seq=0 out_seq=0 2024-04-23T18:39:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.433+0000 7fcb8dffb640 10 client.17046 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcb8dffb640 1 -- 172.21.15.33:0/3920754872 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d20c21390 con 0x561d20cc96d0 2024-04-23T18:39:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcb8dffb640 10 client.17046 handle_command_reply: tid=0 2024-04-23T18:39:46.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 client.17046 shutdown 2024-04-23T18:39:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 msgr2=0x561d20cb6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:46.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fcb90032c20 tx=0x7fcb90032d60 comp rx=0 tx=0).stop 2024-04-23T18:39:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 shutdown_connections 2024-04-23T18:39:46.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20cc96d0 0x561d20ccbac0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20caede0 0x561d20cb6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcb8dffb640 0 client.17046 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:47.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/3920754872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20cae4c0 0x561d20cb6a80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 >> 172.21.15.33:0/3920754872 conn(0x561d20caab30 msgr2=0x561d20cb09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 shutdown_connections 2024-04-23T18:39:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 1 -- 172.21.15.33:0/3920754872 wait complete. 2024-04-23T18:39:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.434+0000 7fcbaf98b740 20 client.17046 trim_cache size 0 max 16384 2024-04-23T18:39:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.449+0000 7fcbaf98b740 1 Processor -- start 2024-04-23T18:39:47.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.449+0000 7fcbaf98b740 1 -- start start 2024-04-23T18:39:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.449+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 0x561d20cb7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.449+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.449+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81ab890 con 0x561d20caede0 2024-04-23T18:39:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81ad3d0 con 0x561d20cae4c0 2024-04-23T18:39:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81abf60 con 0x561d20cbba10 2024-04-23T18:39:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:47.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35338/0 (socket says 172.21.15.33:35338) 2024-04-23T18:39:47.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 -- 172.21.15.33:0/2706825152 learned_addr learned my addr 172.21.15.33:0/2706825152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 0x561d20cb7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 -- 172.21.15.33:0/2706825152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 msgr2=0x561d20cb4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 -- 172.21.15.33:0/2706825152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 msgr2=0x561d20cb7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 0x561d20cb7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 -- 172.21.15.33:0/2706825152 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d20caf340 con 0x561d20caede0 2024-04-23T18:39:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 0x561d20cb7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcba659c640 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x561d20cb62e0 tx=0x7fcb98005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c844dc1b23454c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcb8ffff640 1 -- 172.21.15.33:0/2706825152 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98004020 con 0x561d20caede0 2024-04-23T18:39:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcb8ffff640 1 -- 172.21.15.33:0/2706825152 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb98020e90 con 0x561d20caede0 2024-04-23T18:39:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.450+0000 7fcb8ffff640 1 -- 172.21.15.33:0/2706825152 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98025790 con 0x561d20caede0 2024-04-23T18:39:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2706825152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 msgr2=0x561d20cb3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x561d20cb62e0 tx=0x7fcb98005f90 comp rx=0 tx=0).stop 2024-04-23T18:39:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2706825152 shutdown_connections 2024-04-23T18:39:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cbba10 0x561d20cb4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cb3ce0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2706825152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cae4c0 0x561d20cb7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2706825152 >> 172.21.15.33:0/2706825152 conn(0x561d20caab30 msgr2=0x561d20cb0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2706825152 shutdown_connections 2024-04-23T18:39:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2706825152 wait complete. 2024-04-23T18:39:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 -- start start 2024-04-23T18:39:48.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.451+0000 7fcbaf98b740 1 Processor -- start 2024-04-23T18:39:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cc4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:48.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 0x561d20cbff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81acd00 con 0x561d20caede0 2024-04-23T18:39:49.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81ac630 con 0x561d20cbba10 2024-04-23T18:39:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81a9870 con 0x561d20cae4c0 2024-04-23T18:39:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 10 client.0 _ll_register_callbacks cb 0x7fcb90031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35344/0 (socket says 172.21.15.33:35344) 2024-04-23T18:39:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 -- 172.21.15.33:0/2375644684 learned_addr learned my addr 172.21.15.33:0/2375644684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cc4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 0x561d20cbff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:49.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 -- 172.21.15.33:0/2375644684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 msgr2=0x561d20cc4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:49.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cc4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 -- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 msgr2=0x561d20cbff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:49.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 0x561d20cbff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:49.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 -- 172.21.15.33:0/2375644684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d20cc06d0 con 0x561d20caede0 2024-04-23T18:39:49.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 0x561d20cbff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcba659c640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005bc0 tx=0x7fcb980209e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649a35917dfe9fff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb98020cc0 con 0x561d20caede0 2024-04-23T18:39:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fcb980256c0 con 0x561d20caede0 2024-04-23T18:39:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.452+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb98020e90 con 0x561d20caede0 2024-04-23T18:39:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcb9803c050 con 0x561d20caede0 2024-04-23T18:39:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 10 client.17052 fetch_fsmap learned FSMap version 36 2024-04-23T18:39:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561d20d2f510 con 0x561d20caede0 2024-04-23T18:39:50.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb98020e90 con 0x561d20caede0 2024-04-23T18:39:50.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fcb980046c0 con 0x561d20caede0 2024-04-23T18:39:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 10 client.17052 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:39:50.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 10 client.17052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:39:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 20 client.17052 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:50.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20d2fc50 0x561d20d32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:50.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 4 client.17052 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:39:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d20c1b200 con 0x561d20d2fc50 2024-04-23T18:39:50.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.453+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20d2fc50 0x561d20d32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:50.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.454+0000 7fcb8dffb640 10 client.17052 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.454+0000 7fcb9ffff640 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20d2fc50 0x561d20d32070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6531f5f1d2aa1f server_cookie=9ae2c4d9d04adf6f in_seq=0 out_seq=0 2024-04-23T18:39:50.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcb8dffb640 1 -- 172.21.15.33:0/2375644684 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d20c1b200 con 0x561d20d2fc50 2024-04-23T18:39:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcb8dffb640 10 client.17052 handle_command_reply: tid=0 2024-04-23T18:39:51.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 client.17052 shutdown 2024-04-23T18:39:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 msgr2=0x561d20cc4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:51.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005bc0 tx=0x7fcb980209e0 comp rx=0 tx=0).stop 2024-04-23T18:39:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 shutdown_connections 2024-04-23T18:39:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561d20d2fc50 0x561d20d32070 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561d20cbba10 0x561d20cbff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:51.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d20caede0 0x561d20cc4ed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:51.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 --2- 172.21.15.33:0/2375644684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d20cae4c0 0x561d20cc4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:51.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcb8dffb640 0 client.17052 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 >> 172.21.15.33:0/2375644684 conn(0x561d20caab30 msgr2=0x561d20ca9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 shutdown_connections 2024-04-23T18:39:51.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 1 -- 172.21.15.33:0/2375644684 wait complete. 2024-04-23T18:39:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.461+0000 7fcbaf98b740 20 client.17052 trim_cache size 0 max 16384 2024-04-23T18:39:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:39:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:39:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:39:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:39:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 msgr2=0x7fcb940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 0x7fcb940a6280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcba00ce500 tx=0x7fcba006e6e0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 msgr2=0x7fcba819c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fcba000eb10 tx=0x7fcba000c2d0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 shutdown_connections 2024-04-23T18:39:52.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcb940a3dc0 0x7fcb940a6280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcba819be10 0x7fcba819c2c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81061b0 0x7fcba8197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 --2- 172.21.15.33:0/1433202184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcba81010e0 0x7fcba8197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:52.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 >> 172.21.15.33:0/1433202184 conn(0x7fcba80771d0 msgr2=0x7fcba80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:52.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 shutdown_connections 2024-04-23T18:39:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:29.462+0000 7fcbacc7b640 1 -- 172.21.15.33:0/1433202184 wait complete. 2024-04-23T18:39:52.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:29 smithi033 ceph-mon[29274]: pgmap v379: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:39:52.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:40 smithi122 ceph-mon[32546]: pgmap v417: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:52.938 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:42 smithi122 ceph-mon[32546]: pgmap v418: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:52.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:44 smithi122 ceph-mon[32546]: pgmap v419: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:53.071 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:46 smithi122 ceph-mon[32546]: pgmap v420: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:53.121 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:48 smithi122 ceph-mon[32546]: pgmap v421: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:39:53.171 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:53.221 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:53.280 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:53.330 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:48 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:39:53.363 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:50 smithi122 ceph-mon[32546]: pgmap v422: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:39:53.427 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:40 smithi106 ceph-mon[32716]: pgmap v417: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:53.455 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:42 smithi106 ceph-mon[32716]: pgmap v418: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:53.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:44 smithi106 ceph-mon[32716]: pgmap v419: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:39:53.522 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:46 smithi106 ceph-mon[32716]: pgmap v420: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:53.605 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:48 smithi106 ceph-mon[32716]: pgmap v421: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:39:53.638 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:39:53.672 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:39:53.705 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:39:53.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:48 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:39:53.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:50 smithi106 ceph-mon[32716]: pgmap v422: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:39:53.815 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:52 smithi122 ceph-mon[32546]: pgmap v423: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:53.898 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:52 smithi106 ceph-mon[32716]: pgmap v423: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:39:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 -- 172.21.15.33:0/4045612012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 msgr2=0x7f80c8102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:53.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/4045612012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8102580 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009ca0 tx=0x7f80c002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:39:54.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 -- 172.21.15.33:0/4045612012 shutdown_connections 2024-04-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/4045612012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:54.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/4045612012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8102b50 0x7f80c8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:54.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/4045612012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8102580 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 -- 172.21.15.33:0/4045612012 >> 172.21.15.33:0/4045612012 conn(0x7f80c8079350 msgr2=0x7f80c8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:54.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 -- 172.21.15.33:0/4045612012 shutdown_connections 2024-04-23T18:39:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.123+0000 7f80ccd9b640 1 -- 172.21.15.33:0/4045612012 wait complete. 2024-04-23T18:39:54.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 Processor -- start 2024-04-23T18:39:54.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 -- start start 2024-04-23T18:39:54.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:54.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8074cd0 0x7f80c81a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81121d0 con 0x7f80c81021a0 2024-04-23T18:39:54.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8112050 con 0x7f80c8107270 2024-04-23T18:39:54.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80ccd9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80c8112350 con 0x7f80c8074cd0 2024-04-23T18:39:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:54.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35372/0 (socket says 172.21.15.33:35372) 2024-04-23T18:39:54.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 -- 172.21.15.33:0/1611582968 learned_addr learned my addr 172.21.15.33:0/1611582968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:54.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 -- 172.21.15.33:0/1611582968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8074cd0 msgr2=0x7f80c81a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:54.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8074cd0 0x7f80c81a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:54.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8074cd0 0x7f80c81a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:54.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 -- 172.21.15.33:0/1611582968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 msgr2=0x7f80c8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:55.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:55.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 -- 172.21.15.33:0/1611582968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c81a7020 con 0x7f80c81021a0 2024-04-23T18:39:55.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:39:55.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.124+0000 7f80c659c640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f80c002ade0 tx=0x7f80c0031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:55.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.125+0000 7f80b77fe640 1 -- 172.21.15.33:0/1611582968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c0003b70 con 0x7f80c81021a0 2024-04-23T18:39:55.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.125+0000 7f80b77fe640 1 -- 172.21.15.33:0/1611582968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0003d10 con 0x7f80c81021a0 2024-04-23T18:39:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.125+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c81a72b0 con 0x7f80c81021a0 2024-04-23T18:39:55.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.125+0000 7f80b77fe640 1 -- 172.21.15.33:0/1611582968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c002fa50 con 0x7f80c81021a0 2024-04-23T18:39:55.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.125+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c81a7800 con 0x7f80c81021a0 2024-04-23T18:39:55.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.127+0000 7f80b77fe640 1 -- 172.21.15.33:0/1611582968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80c0004580 con 0x7f80c81021a0 2024-04-23T18:39:55.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.127+0000 7f80cfb94740 1 Processor -- start 2024-04-23T18:39:55.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 -- start start 2024-04-23T18:39:55.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:55.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702d3090 0x5619702d5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:55.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56197022d370 con 0x5619702c80a0 2024-04-23T18:39:55.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619701ca2e0 con 0x5619702c89c0 2024-04-23T18:39:55.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80cfb94740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561970229330 con 0x5619702d3090 2024-04-23T18:39:55.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80b77fe640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 0x7f80b00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35388/0 (socket says 172.21.15.33:35388) 2024-04-23T18:39:55.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 -- 172.21.15.33:0/1312948020 learned_addr learned my addr 172.21.15.33:0/1312948020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:55.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 0x7f80b00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:55.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80b77fe640 1 -- 172.21.15.33:0/1611582968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f80c00f1bc0 con 0x7f80c81021a0 2024-04-23T18:39:56.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c659c640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:56.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702d3090 0x5619702d5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:56.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 -- 172.21.15.33:0/1312948020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702d3090 msgr2=0x5619702d5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702d3090 0x5619702d5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 -- 172.21.15.33:0/1312948020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 msgr2=0x5619702d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:56.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:56.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c6d9d640 1 -- 172.21.15.33:0/1312948020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56197000bd80 con 0x5619702c80a0 2024-04-23T18:39:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.128+0000 7f80c659c640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:39:56.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.129+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f80b8013a80 tx=0x7f80b8013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b777cc39562ba7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:56.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.129+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 0x7f80b00a6210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80b80058c0 tx=0x7f80b8015000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:56.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.129+0000 7f80b4ff9640 1 -- 172.21.15.33:0/1312948020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8013250 con 0x5619702c80a0 2024-04-23T18:39:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.129+0000 7f80b4ff9640 1 -- 172.21.15.33:0/1312948020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b80133f0 con 0x5619702c80a0 2024-04-23T18:39:56.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.129+0000 7f80b4ff9640 1 -- 172.21.15.33:0/1312948020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8004c40 con 0x5619702c80a0 2024-04-23T18:39:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- 172.21.15.33:0/1312948020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 msgr2=0x5619702c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:56.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f80b8013a80 tx=0x7f80b8013f40 comp rx=0 tx=0).stop 2024-04-23T18:39:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- 172.21.15.33:0/1312948020 shutdown_connections 2024-04-23T18:39:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702d3090 0x5619702d5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:57.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1312948020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702c8480 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:57.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- 172.21.15.33:0/1312948020 >> 172.21.15.33:0/1312948020 conn(0x5619702c43c0 msgr2=0x5619702c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:39:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- 172.21.15.33:0/1312948020 shutdown_connections 2024-04-23T18:39:57.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- 172.21.15.33:0/1312948020 wait complete. 2024-04-23T18:39:57.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- start start 2024-04-23T18:39:57.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 Processor -- start 2024-04-23T18:39:57.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:57.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5619702d5e20 con 0x5619702c89c0 2024-04-23T18:39:57.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619701c7fb0 con 0x5619702d3090 2024-04-23T18:39:57.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619702d5fa0 con 0x5619702c80a0 2024-04-23T18:39:58.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.130+0000 7f80cfb94740 10 client.0 _ll_register_callbacks cb 0x5619701c79a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:39:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:58.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35392/0 (socket says 172.21.15.33:35392) 2024-04-23T18:39:58.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 -- 172.21.15.33:0/3029272572 learned_addr learned my addr 172.21.15.33:0/3029272572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:39:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c659c640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:58.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 -- 172.21.15.33:0/3029272572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 msgr2=0x5619702cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:58.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:58.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 -- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 msgr2=0x5619702d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:39:58.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:39:58.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 -- 172.21.15.33:0/3029272572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56197006c510 con 0x5619702c89c0 2024-04-23T18:39:58.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0124a0 tx=0x7f80bc012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27dad8fc8285470 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:39:58.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:39:58.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc01f060 con 0x5619702c89c0 2024-04-23T18:39:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f80bc004460 con 0x5619702c89c0 2024-04-23T18:39:58.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b80133f0 con 0x5619702c89c0 2024-04-23T18:39:58.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80bc0035f0 con 0x5619702c89c0 2024-04-23T18:39:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80cfb94740 10 client.17064 fetch_fsmap learned FSMap version 36 2024-04-23T18:39:59.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.131+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56196feff670 con 0x5619702c89c0 2024-04-23T18:39:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b80133f0 con 0x5619702c89c0 2024-04-23T18:39:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f80bc019030 con 0x5619702c89c0 2024-04-23T18:39:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 10 client.17064 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:39:59.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 10 client.17064 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:39:59.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 20 client.17064 populate_metadata read hostname 'smithi033' 2024-04-23T18:39:59.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5619702e33c0 0x5619702e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:39:59.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 4 client.17064 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:39:59.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56197023ad90 con 0x5619702e33c0 2024-04-23T18:39:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.132+0000 7f80c659c640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5619702e33c0 0x5619702e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:39:59.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80adffb640 10 client.17064 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:39:59.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80c659c640 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5619702e33c0 0x5619702e5800 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25962fa8b7cdaa0f server_cookie=e5dc86be4ab5ead6 in_seq=0 out_seq=0 2024-04-23T18:39:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80adffb640 1 -- 172.21.15.33:0/3029272572 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56197023ad90 con 0x5619702e33c0 2024-04-23T18:39:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80adffb640 10 client.17064 handle_command_reply: tid=0 2024-04-23T18:39:59.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 client.17064 shutdown 2024-04-23T18:39:59.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 msgr2=0x5619702d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:39:59.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0124a0 tx=0x7f80bc012960 comp rx=0 tx=0).stop 2024-04-23T18:40:00.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 shutdown_connections 2024-04-23T18:40:00.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5619702e33c0 0x5619702e5800 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:00.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c89c0 0x5619702d04f0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:00.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80adffb640 0 client.17064 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:00.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 --2- 172.21.15.33:0/3029272572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:00.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 >> 172.21.15.33:0/3029272572 conn(0x5619702c43c0 msgr2=0x5619702c3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.133+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 shutdown_connections 2024-04-23T18:40:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.134+0000 7f80cfb94740 1 -- 172.21.15.33:0/3029272572 wait complete. 2024-04-23T18:40:00.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.134+0000 7f80cfb94740 20 client.17064 trim_cache size 0 max 16384 2024-04-23T18:40:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 Processor -- start 2024-04-23T18:40:00.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 -- start start 2024-04-23T18:40:00.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c89c0 0x5619702c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:00.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81afdf0 con 0x5619702c80a0 2024-04-23T18:40:01.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b1930 con 0x5619702d3090 2024-04-23T18:40:01.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80cfb94740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b04c0 con 0x5619702c89c0 2024-04-23T18:40:01.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35408/0 (socket says 172.21.15.33:35408) 2024-04-23T18:40:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c659c640 1 -- 172.21.15.33:0/1059567867 learned_addr learned my addr 172.21.15.33:0/1059567867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:01.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c89c0 0x5619702c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:01.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.151+0000 7f80c659c640 1 -- 172.21.15.33:0/1059567867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c89c0 msgr2=0x5619702c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:01.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c659c640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c89c0 0x5619702c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:01.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c659c640 1 -- 172.21.15.33:0/1059567867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 msgr2=0x5619702c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:01.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c659c640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:01.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c659c640 1 -- 172.21.15.33:0/1059567867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619702ce500 con 0x5619702c80a0 2024-04-23T18:40:01.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:01.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80c659c640 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f80c00bd930 tx=0x7f80c00b9840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21f711a93fb0ba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:01.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80affff640 1 -- 172.21.15.33:0/1059567867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00db070 con 0x5619702c80a0 2024-04-23T18:40:01.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80affff640 1 -- 172.21.15.33:0/1059567867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0036040 con 0x5619702c80a0 2024-04-23T18:40:01.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80affff640 1 -- 172.21.15.33:0/1059567867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00ef5a0 con 0x5619702c80a0 2024-04-23T18:40:01.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 -- 172.21.15.33:0/1059567867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 msgr2=0x5619702dece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f80c00bd930 tx=0x7f80c00b9840 comp rx=0 tx=0).stop 2024-04-23T18:40:01.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 -- 172.21.15.33:0/1059567867 shutdown_connections 2024-04-23T18:40:01.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702d3090 0x5619702c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c89c0 0x5619702c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 --2- 172.21.15.33:0/1059567867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702c80a0 0x5619702dece0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.152+0000 7f80cfb94740 1 -- 172.21.15.33:0/1059567867 >> 172.21.15.33:0/1059567867 conn(0x5619702c43c0 msgr2=0x5619702c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:02.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- 172.21.15.33:0/1059567867 shutdown_connections 2024-04-23T18:40:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- 172.21.15.33:0/1059567867 wait complete. 2024-04-23T18:40:02.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- start start 2024-04-23T18:40:02.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 Processor -- start 2024-04-23T18:40:02.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:02.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b1260 con 0x5619702d3090 2024-04-23T18:40:02.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81b0b90 con 0x5619702c89c0 2024-04-23T18:40:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80c81addd0 con 0x5619702c80a0 2024-04-23T18:40:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80cfb94740 10 client.0 _ll_register_callbacks cb 0x7f80bc009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35412/0 (socket says 172.21.15.33:35412) 2024-04-23T18:40:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80c6d9d640 1 -- 172.21.15.33:0/2950532977 learned_addr learned my addr 172.21.15.33:0/2950532977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:02.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80c5d9b640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.153+0000 7f80c659c640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:02.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 -- 172.21.15.33:0/2950532977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 msgr2=0x5619702ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:02.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 -- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 msgr2=0x5619702cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:02.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:02.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 -- 172.21.15.33:0/2950532977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619702cdda0 con 0x5619702d3090 2024-04-23T18:40:03.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80c6d9d640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f80b8010800 tx=0x7f80b8010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255ca1343344dea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:03.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:03.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8013460 con 0x5619702d3090 2024-04-23T18:40:03.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f80b800b760 con 0x5619702d3090 2024-04-23T18:40:03.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c0036040 con 0x5619702d3090 2024-04-23T18:40:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f80b8013600 con 0x5619702d3090 2024-04-23T18:40:03.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80cfb94740 10 client.17070 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561970349ea0 con 0x5619702d3090 2024-04-23T18:40:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c0036040 con 0x5619702d3090 2024-04-23T18:40:03.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.154+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f80b802f070 con 0x5619702d3090 2024-04-23T18:40:03.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 10 client.17070 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 10 client.17070 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:40:03.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 20 client.17070 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:03.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56197034a4f0 0x56197034c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:03.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 4 client.17070 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}] 2024-04-23T18:40:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 82.9, "await_for": 42.0}) -- 0x561970234c00 con 0x56197034a4f0 2024-04-23T18:40:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80c659c640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56197034a4f0 0x56197034c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:03.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80c659c640 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56197034a4f0 0x56197034c910 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc09023296bddbe0 server_cookie=55e952773aa65bc4 in_seq=0 out_seq=0 2024-04-23T18:40:03.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.155+0000 7f80adffb640 10 client.17070 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:03.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80adffb640 1 -- 172.21.15.33:0/2950532977 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561970234c00 con 0x56197034a4f0 2024-04-23T18:40:03.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80adffb640 10 client.17070 handle_command_reply: tid=0 2024-04-23T18:40:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 client.17070 shutdown 2024-04-23T18:40:03.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 msgr2=0x561970348940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f80b8010800 tx=0x7f80b8010900 comp rx=0 tx=0).stop 2024-04-23T18:40:04.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 shutdown_connections 2024-04-23T18:40:04.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56197034a4f0 0x56197034c910 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619702d3090 0x561970348940 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80adffb640 0 client.17070 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:04.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619702c89c0 0x5619702cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 --2- 172.21.15.33:0/2950532977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619702c80a0 0x5619702ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 >> 172.21.15.33:0/2950532977 conn(0x5619702c43c0 msgr2=0x5619702ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 shutdown_connections 2024-04-23T18:40:04.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 1 -- 172.21.15.33:0/2950532977 wait complete. 2024-04-23T18:40:04.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80cfb94740 20 client.17070 trim_cache size 0 max 16384 2024-04-23T18:40:04.522 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:40:04.584 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 36, 2024-04-23T18:40:04.671 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:40:04.705 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 34, 2024-04-23T18:40:04.738 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:40:04.771 INFO:teuthology.orchestra.run.smithi033.stdout: "6bf9e7c1": { 2024-04-23T18:40:04.805 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 34, 2024-04-23T18:40:04.838 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 0.0, 2024-04-23T18:40:04.871 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:40:04.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:40:04.938 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:40:04.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:40:05.068 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:40:05.147 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 82.9, 2024-04-23T18:40:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:40:05.213 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:40:05.247 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:40:05.280 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:40:05.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:40:05.364 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:40:05.398 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:40:05.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:40:05.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:40:05.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:40:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:40:05.637 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:40:05.724 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:54 smithi122 ceph-mon[32546]: pgmap v424: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:40:05.755 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:56 smithi122 ceph-mon[32546]: pgmap v425: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-23T18:40:05.789 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:39:58 smithi122 ceph-mon[32546]: pgmap v426: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:40:05.822 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:00 smithi122 ceph-mon[32546]: pgmap v427: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:40:05.855 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:00 smithi122 ceph-mon[32546]: overall HEALTH_OK 2024-04-23T18:40:05.889 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:02 smithi122 ceph-mon[32546]: pgmap v428: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-23T18:40:05.922 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:54 smithi106 ceph-mon[32716]: pgmap v424: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:40:05.955 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:56 smithi106 ceph-mon[32716]: pgmap v425: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-23T18:40:05.989 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:39:58 smithi106 ceph-mon[32716]: pgmap v426: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:40:06.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:00 smithi106 ceph-mon[32716]: pgmap v427: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:40:06.110 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:00 smithi106 ceph-mon[32716]: overall HEALTH_OK 2024-04-23T18:40:06.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:02 smithi106 ceph-mon[32716]: pgmap v428: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-23T18:40:06.189 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:04 smithi106 ceph-mon[32716]: pgmap v429: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T18:40:06.222 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:04 smithi106 ceph-mon[32716]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:40:06.256 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:40:06.289 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: in thread 7f23354b5700 thread_name:md_log_replay 2024-04-23T18:40:06.322 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:40:06.369 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2346876d20] 2024-04-23T18:40:06.406 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55927669838f] 2024-04-23T18:40:06.439 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55927669bb23] 2024-04-23T18:40:06.473 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 4: (MDLog::_replay_thread()+0x617) [0x5592766f6647] 2024-04-23T18:40:06.523 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5592766ffd31] 2024-04-23T18:40:06.556 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f234686c1ca] 2024-04-23T18:40:06.589 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 7: clone() 2024-04-23T18:40:06.623 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:04.012+0000 7f23354b5700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:40:06.678 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: in thread 7f23354b5700 thread_name:md_log_replay 2024-04-23T18:40:06.773 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:06.806 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:40:06.856 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2346876d20] 2024-04-23T18:40:06.889 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55927669838f] 2024-04-23T18:40:06.923 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55927669bb23] 2024-04-23T18:40:06.956 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 4: (MDLog::_replay_thread()+0x617) [0x5592766f6647] 2024-04-23T18:40:06.990 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5592766ffd31] 2024-04-23T18:40:07.023 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f234686c1ca] 2024-04-23T18:40:07.056 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 7: clone() 2024-04-23T18:40:07.140 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:40:07.173 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:07.240 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 0> 2024-04-23T18:40:04.012+0000 7f23354b5700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:40:07.273 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: in thread 7f23354b5700 thread_name:md_log_replay 2024-04-23T18:40:07.306 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:07.340 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:40:07.399 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2346876d20] 2024-04-23T18:40:07.432 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55927669838f] 2024-04-23T18:40:07.465 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55927669bb23] 2024-04-23T18:40:07.498 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 4: (MDLog::_replay_thread()+0x617) [0x5592766f6647] 2024-04-23T18:40:07.565 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5592766ffd31] 2024-04-23T18:40:07.598 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f234686c1ca] 2024-04-23T18:40:07.632 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 7: clone() 2024-04-23T18:40:07.665 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:40:07.737 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:07.848 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 0> 2024-04-23T18:40:04.012+0000 7f23354b5700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:40:07.924 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: in thread 7f23354b5700 thread_name:md_log_replay 2024-04-23T18:40:07.965 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:08.012 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:40:08.057 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2346876d20] 2024-04-23T18:40:08.090 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55927669838f] 2024-04-23T18:40:08.124 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55927669bb23] 2024-04-23T18:40:08.157 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 4: (MDLog::_replay_thread()+0x617) [0x5592766f6647] 2024-04-23T18:40:08.190 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5592766ffd31] 2024-04-23T18:40:08.224 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f234686c1ca] 2024-04-23T18:40:08.249 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 7: clone() 2024-04-23T18:40:08.329 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:40:08.377 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f[69802]: 2024-04-23T18:40:08.390 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 podman[70230]: 2024-04-23 18:40:04.066821556 +0000 UTC m=+0.021573813 container died 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:40:08.408 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-mon[32546]: pgmap v429: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T18:40:08.424 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:04 smithi122 ceph-mon[32546]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:40:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 msgr2=0x7f80b00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:08.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 0x7f80b00a6210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80b80058c0 tx=0x7f80b8015000 comp rx=0 tx=0).stop 2024-04-23T18:40:08.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 msgr2=0x7f80c8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:08.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.171+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f80c002ade0 tx=0x7f80c0031770 comp rx=0 tx=0).stop 2024-04-23T18:40:08.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 shutdown_connections 2024-04-23T18:40:08.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80b00a3d50 0x7f80b00a6210 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80c8074cd0 0x7f80c81a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80c8107270 0x7f80c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 --2- 172.21.15.33:0/1611582968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c81021a0 0x7f80c8074250 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 >> 172.21.15.33:0/1611582968 conn(0x7f80c8079350 msgr2=0x7f80c81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:08.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 shutdown_connections 2024-04-23T18:40:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:38:31.172+0000 7f80ccd9b640 1 -- 172.21.15.33:0/1611582968 wait complete. 2024-04-23T18:40:08.975 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:06 smithi106 ceph-mon[32716]: pgmap v431: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:09.008 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 podman[70230]: 2024-04-23 18:40:04.799655631 +0000 UTC m=+0.754407878 container remove 6fa2de67d6b9a251a20c1abf37f9c8fc6486383d564ed45da5bac77c9615e972 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:40:09.041 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:04 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:40:09.074 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:05 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:40:09.108 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:06 smithi122 ceph-mon[32546]: pgmap v431: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:09.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:31 smithi033 ceph-mon[29274]: pgmap v380: 97 pgs: 97 active+clean; 11 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-23T18:40:09.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:33 smithi033 ceph-mon[29274]: pgmap v381: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-23T18:40:09.241 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:35 smithi033 ceph-mon[29274]: pgmap v382: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-23T18:40:09.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:37 smithi033 ceph-mon[29274]: pgmap v383: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 29 KiB/s wr, 19 op/s 2024-04-23T18:40:09.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:39 smithi033 ceph-mon[29274]: pgmap v384: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:40:09.400 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:41 smithi033 ceph-mon[29274]: pgmap v385: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-23T18:40:09.435 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6bf9e7c1', quiesce took 210.2 seconds. Will release after: 22.9 2024-04-23T18:40:09.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:08 smithi122 ceph-mon[32546]: pgmap v432: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:09.500 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:08 smithi106 ceph-mon[32716]: pgmap v432: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 -- 172.21.15.33:0/24608328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd48103aa0 msgr2=0x7fbd4810bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 --2- 172.21.15.33:0/24608328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd48103aa0 0x7fbd4810bf80 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009900 tx=0x7fbd4402a480 comp rx=0 tx=0).stop 2024-04-23T18:40:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 -- 172.21.15.33:0/24608328 shutdown_connections 2024-04-23T18:40:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 --2- 172.21.15.33:0/24608328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd4810c550 0x7fbd4810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 --2- 172.21.15.33:0/24608328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd48103aa0 0x7fbd4810bf80 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 --2- 172.21.15.33:0/24608328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 0x7fbd48103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 -- 172.21.15.33:0/24608328 >> 172.21.15.33:0/24608328 conn(0x7fbd48077fd0 msgr2=0x7fbd480783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 -- 172.21.15.33:0/24608328 shutdown_connections 2024-04-23T18:40:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.562+0000 7fbd506a0640 1 -- 172.21.15.33:0/24608328 wait complete. 2024-04-23T18:40:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 Processor -- start 2024-04-23T18:40:09.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 -- start start 2024-04-23T18:40:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 0x7fbd48198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd48103aa0 0x7fbd48198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481121d0 con 0x7fbd4810c550 2024-04-23T18:40:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48112050 con 0x7fbd48103aa0 2024-04-23T18:40:10.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd506a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd48112350 con 0x7fbd48103180 2024-04-23T18:40:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42496/0 (socket says 172.21.15.33:42496) 2024-04-23T18:40:10.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 -- 172.21.15.33:0/3927950429 learned_addr learned my addr 172.21.15.33:0/3927950429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd48103aa0 0x7fbd48198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 0x7fbd48198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 -- 172.21.15.33:0/3927950429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 msgr2=0x7fbd48198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 0x7fbd48198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 -- 172.21.15.33:0/3927950429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd48103aa0 msgr2=0x7fbd48198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd48103aa0 0x7fbd48198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.563+0000 7fbd4ec3d640 1 -- 172.21.15.33:0/3927950429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4819d740 con 0x7fbd4810c550 2024-04-23T18:40:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd4ec3d640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000eaa0 tx=0x7fbd4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd3b7fe640 1 -- 172.21.15.33:0/3927950429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000cdb0 con 0x7fbd4810c550 2024-04-23T18:40:10.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4819da30 con 0x7fbd4810c550 2024-04-23T18:40:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd3b7fe640 1 -- 172.21.15.33:0/3927950429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd40005020 con 0x7fbd4810c550 2024-04-23T18:40:11.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd3b7fe640 1 -- 172.21.15.33:0/3927950429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd400116d0 con 0x7fbd4810c550 2024-04-23T18:40:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.564+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a52c0 con 0x7fbd4810c550 2024-04-23T18:40:11.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd3b7fe640 1 -- 172.21.15.33:0/3927950429 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbd400051d0 con 0x7fbd4810c550 2024-04-23T18:40:11.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 Processor -- start 2024-04-23T18:40:11.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 -- start start 2024-04-23T18:40:11.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917cdca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 0x55f917ce9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:11.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f917c41970 con 0x55f917cdcf40 2024-04-23T18:40:11.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f917bde4a0 con 0x55f917cdc620 2024-04-23T18:40:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.567+0000 7fbd533ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f917c3d930 con 0x55f917ce7800 2024-04-23T18:40:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:11.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42500/0 (socket says 172.21.15.33:42500) 2024-04-23T18:40:11.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 -- 172.21.15.33:0/3372868943 learned_addr learned my addr 172.21.15.33:0/3372868943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:11.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd3b7fe640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 0x7fbd340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:11.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 0x55f917ce9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:11.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917cdca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4ec3d640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 0x7fbd340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 -- 172.21.15.33:0/3372868943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 msgr2=0x55f917ce9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 0x55f917ce9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:11.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 -- 172.21.15.33:0/3372868943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 msgr2=0x55f917cdca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:12.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917cdca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 -- 172.21.15.33:0/3372868943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f917954210 con 0x55f917cdcf40 2024-04-23T18:40:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 0x55f917ce9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd3b7fe640 1 -- 172.21.15.33:0/3927950429 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fbd400d0a00 con 0x7fbd4810c550 2024-04-23T18:40:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00bcf0 tx=0x7fbd3c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66d21fe93f3d761 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:12.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd4ec3d640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 0x7fbd340a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd40099380 tx=0x7fbd4008e040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.568+0000 7fbd38ff9640 1 -- 172.21.15.33:0/3372868943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c015070 con 0x55f917cdcf40 2024-04-23T18:40:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd38ff9640 1 -- 172.21.15.33:0/3372868943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0118f0 con 0x55f917cdcf40 2024-04-23T18:40:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd38ff9640 1 -- 172.21.15.33:0/3372868943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0107f0 con 0x55f917cdcf40 2024-04-23T18:40:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 -- 172.21.15.33:0/3372868943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 msgr2=0x55f917ce72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00bcf0 tx=0x7fbd3c00f730 comp rx=0 tx=0).stop 2024-04-23T18:40:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 -- 172.21.15.33:0/3372868943 shutdown_connections 2024-04-23T18:40:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917ce7800 0x55f917ce9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce72c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3372868943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917cdca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:13.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 -- 172.21.15.33:0/3372868943 >> 172.21.15.33:0/3372868943 conn(0x55f917cd8b30 msgr2=0x55f917cd8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 -- 172.21.15.33:0/3372868943 shutdown_connections 2024-04-23T18:40:13.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.569+0000 7fbd533ba740 1 -- 172.21.15.33:0/3372868943 wait complete. 2024-04-23T18:40:13.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 -- start start 2024-04-23T18:40:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 Processor -- start 2024-04-23T18:40:13.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917ce70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 0x55f917ce48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:13.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f917cea430 con 0x55f917cdcf40 2024-04-23T18:40:13.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f917bdc0b0 con 0x55f917ce7800 2024-04-23T18:40:13.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f917cea5b0 con 0x55f917cdc620 2024-04-23T18:40:13.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd533ba740 10 client.0 _ll_register_callbacks cb 0x55f917bc9d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd4e43c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd4e43c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42512/0 (socket says 172.21.15.33:42512) 2024-04-23T18:40:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd4e43c640 1 -- 172.21.15.33:0/2807729664 learned_addr learned my addr 172.21.15.33:0/2807729664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:13.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917ce70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.570+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 0x55f917ce48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 -- 172.21.15.33:0/2807729664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 msgr2=0x55f917ce70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:14.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917ce70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 -- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 msgr2=0x55f917ce48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 0x55f917ce48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 -- 172.21.15.33:0/2807729664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f917a20620 con 0x55f917cdcf40 2024-04-23T18:40:14.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 0x55f917ce48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b840 tx=0x7fbd3c011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f87c4f4f736033 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:14.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c028060 con 0x55f917cdcf40 2024-04-23T18:40:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fbd3c011cb0 con 0x55f917cdcf40 2024-04-23T18:40:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c0118f0 con 0x55f917cdcf40 2024-04-23T18:40:14.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd3c015070 con 0x55f917cdcf40 2024-04-23T18:40:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd533ba740 10 client.17082 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f917cf62a0 con 0x55f917cdcf40 2024-04-23T18:40:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0118f0 con 0x55f917cdcf40 2024-04-23T18:40:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.571+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbd3c004a60 con 0x55f917cdcf40 2024-04-23T18:40:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 10 client.17082 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:15.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 20 client.17082 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917cf76a0 0x55f917cf9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 4 client.17082 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f917c4f390 con 0x55f917cf76a0 2024-04-23T18:40:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917cf76a0 0x55f917cf9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:15.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917cf76a0 0x55f917cf9a90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3442fca5e2a6b2e4 server_cookie=5bf5506d1d12ad14 in_seq=0 out_seq=0 2024-04-23T18:40:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.572+0000 7fbd31ffb640 10 client.17082 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd31ffb640 1 -- 172.21.15.33:0/2807729664 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f917c4f390 con 0x55f917cf76a0 2024-04-23T18:40:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd31ffb640 10 client.17082 handle_command_reply: tid=0 2024-04-23T18:40:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 client.17082 shutdown 2024-04-23T18:40:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 msgr2=0x55f917ce43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00b840 tx=0x7fbd3c011670 comp rx=0 tx=0).stop 2024-04-23T18:40:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 shutdown_connections 2024-04-23T18:40:15.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917cf76a0 0x55f917cf9a90 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917ce7800 0x55f917ce48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cdcf40 0x55f917ce43a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd31ffb640 0 client.17082 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:15.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 --2- 172.21.15.33:0/2807729664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917ce70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 >> 172.21.15.33:0/2807729664 conn(0x55f917cd8b30 msgr2=0x55f917cde8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 shutdown_connections 2024-04-23T18:40:15.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.573+0000 7fbd533ba740 1 -- 172.21.15.33:0/2807729664 wait complete. 2024-04-23T18:40:16.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.574+0000 7fbd533ba740 20 client.17082 trim_cache size 0 max 16384 2024-04-23T18:40:16.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.588+0000 7fbd533ba740 1 Processor -- start 2024-04-23T18:40:16.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.588+0000 7fbd533ba740 1 -- start start 2024-04-23T18:40:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917ce6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdcf40 0x55f917cfcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481adb00 con 0x55f917cd8b30 2024-04-23T18:40:16.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481af640 con 0x55f917cdc620 2024-04-23T18:40:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd533ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481ae1d0 con 0x55f917cdcf40 2024-04-23T18:40:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:16.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42518/0 (socket says 172.21.15.33:42518) 2024-04-23T18:40:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/1819389301 learned_addr learned my addr 172.21.15.33:0/1819389301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:16.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917ce6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdcf40 0x55f917cfcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/1819389301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdcf40 msgr2=0x55f917cfcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdcf40 0x55f917cfcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/1819389301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 msgr2=0x55f917ce6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917ce6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/1819389301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f917cdcd70 con 0x55f917cd8b30 2024-04-23T18:40:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005ab0 tx=0x7fbd44005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad70c334d683e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:17.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd33fff640 1 -- 172.21.15.33:0/1819389301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44035040 con 0x55f917cd8b30 2024-04-23T18:40:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd33fff640 1 -- 172.21.15.33:0/1819389301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd440054f0 con 0x55f917cd8b30 2024-04-23T18:40:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.589+0000 7fbd33fff640 1 -- 172.21.15.33:0/1819389301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44047520 con 0x55f917cd8b30 2024-04-23T18:40:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- 172.21.15.33:0/1819389301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 msgr2=0x55f917cdf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:17.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbd44005ab0 tx=0x7fbd44005b50 comp rx=0 tx=0).stop 2024-04-23T18:40:17.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- 172.21.15.33:0/1819389301 shutdown_connections 2024-04-23T18:40:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdcf40 0x55f917cfcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:17.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdc620 0x55f917ce6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:17.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 --2- 172.21.15.33:0/1819389301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cdf850 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- 172.21.15.33:0/1819389301 >> 172.21.15.33:0/1819389301 conn(0x55f917ce4a40 msgr2=0x55f917cdf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- 172.21.15.33:0/1819389301 shutdown_connections 2024-04-23T18:40:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- 172.21.15.33:0/1819389301 wait complete. 2024-04-23T18:40:17.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 -- start start 2024-04-23T18:40:17.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.590+0000 7fbd533ba740 1 Processor -- start 2024-04-23T18:40:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917cfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:17.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdcf40 0x55f917cfa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481aef70 con 0x55f917cd8b30 2024-04-23T18:40:17.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481ae8a0 con 0x55f917cdcf40 2024-04-23T18:40:18.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd481abae0 con 0x55f917cdc620 2024-04-23T18:40:18.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 10 client.0 _ll_register_callbacks cb 0x7fbd3c00bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42520/0 (socket says 172.21.15.33:42520) 2024-04-23T18:40:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/3154848138 learned_addr learned my addr 172.21.15.33:0/3154848138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdcf40 0x55f917cfa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917cfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/3154848138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 msgr2=0x55f917cfa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917cfa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdcf40 msgr2=0x55f917cfa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:18.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdcf40 0x55f917cfa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:18.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 -- 172.21.15.33:0/3154848138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f917cfb060 con 0x55f917cd8b30 2024-04-23T18:40:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd4dc3b640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbd44047300 tx=0x7fbd44047290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a588729a2c18a580 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44035040 con 0x55f917cd8b30 2024-04-23T18:40:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7fbd44047400 con 0x55f917cd8b30 2024-04-23T18:40:18.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.591+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd440054f0 con 0x55f917cd8b30 2024-04-23T18:40:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd44047600 con 0x55f917cd8b30 2024-04-23T18:40:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 10 client.17088 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:19.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f917d5d930 con 0x55f917cd8b30 2024-04-23T18:40:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd440054f0 con 0x55f917cd8b30 2024-04-23T18:40:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbd4405b050 con 0x55f917cd8b30 2024-04-23T18:40:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 10 client.17088 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 20 client.17088 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917ce7800 0x55f917d5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 4 client.17088 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:19.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f917c49200 con 0x55f917ce7800 2024-04-23T18:40:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.592+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917ce7800 0x55f917d5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.593+0000 7fbd31ffb640 10 client.17088 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.593+0000 7fbd4e43c640 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917ce7800 0x55f917d5ffc0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eb5118d6853d2bc server_cookie=da2c89b75b5c25d3 in_seq=0 out_seq=0 2024-04-23T18:40:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd31ffb640 1 -- 172.21.15.33:0/3154848138 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f917c49200 con 0x55f917ce7800 2024-04-23T18:40:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd31ffb640 10 client.17088 handle_command_reply: tid=0 2024-04-23T18:40:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 client.17088 shutdown 2024-04-23T18:40:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 msgr2=0x55f917cfcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fbd44047300 tx=0x7fbd44047290 comp rx=0 tx=0).stop 2024-04-23T18:40:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 shutdown_connections 2024-04-23T18:40:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f917ce7800 0x55f917d5ffc0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f917cdcf40 0x55f917cfa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f917cdc620 0x55f917cfa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 --2- 172.21.15.33:0/3154848138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f917cd8b30 0x55f917cfcf20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd31ffb640 0 client.17088 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:20.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.600+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 >> 172.21.15.33:0/3154848138 conn(0x55f917ce4a40 msgr2=0x55f917cf3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 shutdown_connections 2024-04-23T18:40:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd533ba740 1 -- 172.21.15.33:0/3154848138 wait complete. 2024-04-23T18:40:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd533ba740 20 client.17088 trim_cache size 0 max 16384 2024-04-23T18:40:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:40:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:40:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:40:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:40:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 msgr2=0x7fbd340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:20.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 0x7fbd340a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbd40099380 tx=0x7fbd4008e040 comp rx=0 tx=0).stop 2024-04-23T18:40:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 msgr2=0x7fbd4819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbd4000eaa0 tx=0x7fbd4000ef60 comp rx=0 tx=0).stop 2024-04-23T18:40:20.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 shutdown_connections 2024-04-23T18:40:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbd340a3dc0 0x7fbd340a6280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd4810c550 0x7fbd4819d010 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd48103aa0 0x7fbd48198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 --2- 172.21.15.33:0/3927950429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd48103180 0x7fbd48198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:20.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 >> 172.21.15.33:0/3927950429 conn(0x7fbd48077fd0 msgr2=0x7fbd480fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:20.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 shutdown_connections 2024-04-23T18:40:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:42.601+0000 7fbd506a0640 1 -- 172.21.15.33:0/3927950429 wait complete. 2024-04-23T18:40:21.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:43 smithi033 ceph-mon[29274]: pgmap v386: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-23T18:40:21.125 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:45 smithi033 ceph-mon[29274]: pgmap v387: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:40:21.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:47 smithi033 ceph-mon[29274]: pgmap v388: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-04-23T18:40:21.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:47 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:40:21.271 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:40:21.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:40:21.358 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:40:21.415 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:40:21.441 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: pgmap v389: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:40:21.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:40:21.508 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:40:21.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:40:21.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:50 smithi033 ceph-mon[29274]: pgmap v390: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:40:21.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:52 smithi033 ceph-mon[29274]: pgmap v391: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 4.0 KiB/s wr, 14 op/s 2024-04-23T18:40:21.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:54 smithi033 ceph-mon[29274]: pgmap v392: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-04-23T18:40:21.838 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:10 smithi122 ceph-mon[32546]: pgmap v433: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:21.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:10 smithi122 ceph-mon[32546]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:40:21.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:12 smithi122 ceph-mon[32546]: pgmap v435: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-04-23T18:40:21.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:14 smithi122 ceph-mon[32546]: pgmap v436: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-23T18:40:22.037 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Scheduled restart job, restart counter is at 6. 2024-04-23T18:40:22.086 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:15 smithi122 systemd[1]: Stopped Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:40:22.136 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Start request repeated too quickly. 2024-04-23T18:40:22.186 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:15 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service: Failed with result 'exit-code'. 2024-04-23T18:40:22.270 INFO:journalctl@ceph.mds.f.smithi122.stdout:Apr 23 18:40:15 smithi122 systemd[1]: Failed to start Ceph mds.f for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:40:22.320 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:10 smithi106 ceph-mon[32716]: pgmap v433: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:40:22.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:10 smithi106 ceph-mon[32716]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:40:22.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:12 smithi106 ceph-mon[32716]: pgmap v435: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-04-23T18:40:22.498 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:14 smithi106 ceph-mon[32716]: pgmap v436: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-23T18:40:22.545 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:16 smithi106 ceph-mon[32716]: pgmap v437: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:18 smithi106 ceph-mon[32716]: pgmap v438: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:20 smithi106 ceph-mon[32716]: pgmap v439: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.687 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:20 smithi106 ceph-mon[32716]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-23T18:40:22.728 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:20 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:40:22.763 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:16 smithi122 ceph-mon[32546]: pgmap v437: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.795 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:18 smithi122 ceph-mon[32546]: pgmap v438: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.895 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:20 smithi122 ceph-mon[32546]: pgmap v439: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:40:22.937 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:20 smithi122 ceph-mon[32546]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-23T18:40:22.970 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:20 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:40:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 -- 172.21.15.33:0/1964344119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138102b70 msgr2=0x7f7138106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 --2- 172.21.15.33:0/1964344119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138102b70 0x7f7138106bb0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f712c009900 tx=0x7f712c02a410 comp rx=0 tx=0).stop 2024-04-23T18:40:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 -- 172.21.15.33:0/1964344119 shutdown_connections 2024-04-23T18:40:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 --2- 172.21.15.33:0/1964344119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7138107290 0x7f713810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 --2- 172.21.15.33:0/1964344119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138102b70 0x7f7138106bb0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 --2- 172.21.15.33:0/1964344119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 0x7f71381025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.700+0000 7f713e61e640 1 -- 172.21.15.33:0/1964344119 >> 172.21.15.33:0/1964344119 conn(0x7f71380fda20 msgr2=0x7f71380ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:23.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.701+0000 7f713e61e640 1 -- 172.21.15.33:0/1964344119 shutdown_connections 2024-04-23T18:40:23.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.701+0000 7f713e61e640 1 -- 172.21.15.33:0/1964344119 wait complete. 2024-04-23T18:40:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.701+0000 7f713e61e640 1 Processor -- start 2024-04-23T18:40:23.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.701+0000 7f713e61e640 1 -- start start 2024-04-23T18:40:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 0x7f7138198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f713819cf40 0x7f713819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112200 con 0x7f7138107290 2024-04-23T18:40:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112080 con 0x7f71381021c0 2024-04-23T18:40:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713e61e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112380 con 0x7f713819cf40 2024-04-23T18:40:23.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:23.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48142/0 (socket says 172.21.15.33:48142) 2024-04-23T18:40:23.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 -- 172.21.15.33:0/2921520898 learned_addr learned my addr 172.21.15.33:0/2921520898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f7137fff640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 0x7f7138198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f713cbbb640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f713819cf40 0x7f713819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 -- 172.21.15.33:0/2921520898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f713819cf40 msgr2=0x7f713819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f713819cf40 0x7f713819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 -- 172.21.15.33:0/2921520898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 msgr2=0x7f7138198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 0x7f7138198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 -- 172.21.15.33:0/2921520898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713819da30 con 0x7f7138107290 2024-04-23T18:40:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.702+0000 7f71377fe640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f712c005ab0 tx=0x7f712c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:24.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.703+0000 7f71357fa640 1 -- 172.21.15.33:0/2921520898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c03f070 con 0x7f7138107290 2024-04-23T18:40:24.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.703+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713819dcc0 con 0x7f7138107290 2024-04-23T18:40:24.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.703+0000 7f71357fa640 1 -- 172.21.15.33:0/2921520898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f712c031d60 con 0x7f7138107290 2024-04-23T18:40:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.703+0000 7f71357fa640 1 -- 172.21.15.33:0/2921520898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c035030 con 0x7f7138107290 2024-04-23T18:40:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.703+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381a5480 con 0x7f7138107290 2024-04-23T18:40:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.705+0000 7f71357fa640 1 -- 172.21.15.33:0/2921520898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f712c042070 con 0x7f7138107290 2024-04-23T18:40:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.705+0000 7f714140d740 1 Processor -- start 2024-04-23T18:40:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.705+0000 7f714140d740 1 -- start start 2024-04-23T18:40:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf6592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:24.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf65bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612bf5b3970 con 0x5612bf64e620 2024-04-23T18:40:24.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612bf5504a0 con 0x5612bf64ef40 2024-04-23T18:40:24.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f714140d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612bf5af930 con 0x5612bf659800 2024-04-23T18:40:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f71357fa640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 0x7f71200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48158/0 (socket says 172.21.15.33:48158) 2024-04-23T18:40:25.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 -- 172.21.15.33:0/1042261061 learned_addr learned my addr 172.21.15.33:0/1042261061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f71377fe640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf65bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f7137fff640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf6592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:25.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 0x7f71200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:25.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f71357fa640 1 -- 172.21.15.33:0/2921520898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f712c0f0fc0 con 0x7f7138107290 2024-04-23T18:40:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 -- 172.21.15.33:0/1042261061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 msgr2=0x5612bf65bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf65bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 -- 172.21.15.33:0/1042261061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 msgr2=0x5612bf6592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf6592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 -- 172.21.15.33:0/1042261061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bf2c6210 con 0x5612bf64e620 2024-04-23T18:40:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f71377fe640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf65bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.706+0000 7f713cbbb640 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7130014a60 tx=0x7f7130014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2539512d17b812 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:25.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f711e7fc640 1 -- 172.21.15.33:0/1042261061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713001ec50 con 0x5612bf64e620 2024-04-23T18:40:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f711e7fc640 1 -- 172.21.15.33:0/1042261061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7130023070 con 0x5612bf64e620 2024-04-23T18:40:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f711e7fc640 1 -- 172.21.15.33:0/1042261061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130005520 con 0x5612bf64e620 2024-04-23T18:40:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f713cbbb640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 0x7f71200a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7130022000 tx=0x7f713001e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 -- 172.21.15.33:0/1042261061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 msgr2=0x5612bf64ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7130014a60 tx=0x7f7130014f20 comp rx=0 tx=0).stop 2024-04-23T18:40:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 -- 172.21.15.33:0/1042261061 shutdown_connections 2024-04-23T18:40:26.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf65bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf6592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 --2- 172.21.15.33:0/1042261061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf64ea00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 -- 172.21.15.33:0/1042261061 >> 172.21.15.33:0/1042261061 conn(0x5612bf64ab30 msgr2=0x5612bf64af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 -- 172.21.15.33:0/1042261061 shutdown_connections 2024-04-23T18:40:26.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.707+0000 7f714140d740 1 -- 172.21.15.33:0/1042261061 wait complete. 2024-04-23T18:40:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 -- start start 2024-04-23T18:40:26.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 Processor -- start 2024-04-23T18:40:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf659100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:26.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf659800 0x5612bf656930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:26.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612bf65c430 con 0x5612bf64ef40 2024-04-23T18:40:26.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5612bf54e0b0 con 0x5612bf659800 2024-04-23T18:40:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612bf65c5b0 con 0x5612bf64e620 2024-04-23T18:40:27.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f714140d740 10 client.0 _ll_register_callbacks cb 0x5612bf53bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f71377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f71377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48174/0 (socket says 172.21.15.33:48174) 2024-04-23T18:40:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f7137fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf659100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:27.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.708+0000 7f71377fe640 1 -- 172.21.15.33:0/2640905963 learned_addr learned my addr 172.21.15.33:0/2640905963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f713cbbb640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf659800 0x5612bf656930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 -- 172.21.15.33:0/2640905963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 msgr2=0x5612bf659100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:27.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf659100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 -- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf659800 msgr2=0x5612bf656930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf659800 0x5612bf656930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 -- 172.21.15.33:0/2640905963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bf392620 con 0x5612bf64ef40 2024-04-23T18:40:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f7137fff640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf659100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:40:27.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71377fe640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f712c0b9420 tx=0x7f712c08b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e90def2581b445 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71177fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:27.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c049050 con 0x5612bf64ef40 2024-04-23T18:40:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f712c03b4f0 con 0x5612bf64ef40 2024-04-23T18:40:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7130023070 con 0x5612bf64ef40 2024-04-23T18:40:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f712c0c12c0 con 0x5612bf64ef40 2024-04-23T18:40:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f714140d740 10 client.17100 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612bf668340 con 0x5612bf64ef40 2024-04-23T18:40:28.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.709+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7130023070 con 0x5612bf64ef40 2024-04-23T18:40:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f712c0bd220 con 0x5612bf64ef40 2024-04-23T18:40:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 10 client.17100 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:28.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 20 client.17100 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6697d0 0x5612bf66bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 4 client.17100 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612bf5c1390 con 0x5612bf6697d0 2024-04-23T18:40:28.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f7137fff640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6697d0 0x5612bf66bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:28.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f7137fff640 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6697d0 0x5612bf66bc10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e4bec435bf7587 server_cookie=26e560609e9fe897 in_seq=0 out_seq=0 2024-04-23T18:40:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.710+0000 7f71177fe640 10 client.17100 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.711+0000 7f71177fe640 1 -- 172.21.15.33:0/2640905963 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612bf5c1390 con 0x5612bf6697d0 2024-04-23T18:40:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.711+0000 7f71177fe640 10 client.17100 handle_command_reply: tid=0 2024-04-23T18:40:29.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.711+0000 7f714140d740 1 client.17100 shutdown 2024-04-23T18:40:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 msgr2=0x5612bf6563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f712c0b9420 tx=0x7f712c08b970 comp rx=0 tx=0).stop 2024-04-23T18:40:29.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 shutdown_connections 2024-04-23T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6697d0 0x5612bf66bc10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf659800 0x5612bf656930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64ef40 0x5612bf6563f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 --2- 172.21.15.33:0/2640905963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf659100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f71177fe640 0 client.17100 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 >> 172.21.15.33:0/2640905963 conn(0x5612bf64ab30 msgr2=0x5612bf6508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 shutdown_connections 2024-04-23T18:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 1 -- 172.21.15.33:0/2640905963 wait complete. 2024-04-23T18:40:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.712+0000 7f714140d740 20 client.17100 trim_cache size 0 max 16384 2024-04-23T18:40:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 Processor -- start 2024-04-23T18:40:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 -- start start 2024-04-23T18:40:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf671520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf671a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:29.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71381ada60 con 0x5612bf64e620 2024-04-23T18:40:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71381af5a0 con 0x5612bf64ef40 2024-04-23T18:40:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f714140d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71381ae130 con 0x5612bf659800 2024-04-23T18:40:29.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f7137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f7137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48176/0 (socket says 172.21.15.33:48176) 2024-04-23T18:40:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f7137fff640 1 -- 172.21.15.33:0/3849543828 learned_addr learned my addr 172.21.15.33:0/3849543828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.727+0000 7f713cbbb640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf671a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f71377fe640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf671520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 -- 172.21.15.33:0/3849543828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 msgr2=0x5612bf671a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf671a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 -- 172.21.15.33:0/3849543828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 msgr2=0x5612bf671520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf671520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 -- 172.21.15.33:0/3849543828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612bf66c290 con 0x5612bf64e620 2024-04-23T18:40:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f71377fe640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf671520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f7137fff640 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f712800b840 tx=0x7f712800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14f6d93860b3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f711dffb640 1 -- 172.21.15.33:0/3849543828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128003da0 con 0x5612bf64e620 2024-04-23T18:40:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f711dffb640 1 -- 172.21.15.33:0/3849543828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71280054d0 con 0x5612bf64e620 2024-04-23T18:40:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f711dffb640 1 -- 172.21.15.33:0/3849543828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71280201b0 con 0x5612bf64e620 2024-04-23T18:40:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.728+0000 7f714140d740 1 -- 172.21.15.33:0/3849543828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 msgr2=0x5612bf6652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f712800b840 tx=0x7f712800b390 comp rx=0 tx=0).stop 2024-04-23T18:40:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- 172.21.15.33:0/3849543828 shutdown_connections 2024-04-23T18:40:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf659800 0x5612bf671a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf671520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- 172.21.15.33:0/3849543828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf64e620 0x5612bf6652f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- 172.21.15.33:0/3849543828 >> 172.21.15.33:0/3849543828 conn(0x5612bf64ab30 msgr2=0x5612bf649cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- 172.21.15.33:0/3849543828 shutdown_connections 2024-04-23T18:40:31.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- 172.21.15.33:0/3849543828 wait complete. 2024-04-23T18:40:31.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- start start 2024-04-23T18:40:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 Processor -- start 2024-04-23T18:40:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf65fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:31.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf65ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:31.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71381aeed0 con 0x5612bf659800 2024-04-23T18:40:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f71381ae800 con 0x5612bf64ef40 2024-04-23T18:40:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.729+0000 7f714140d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71381aba60 con 0x5612bf64e620 2024-04-23T18:40:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f714140d740 10 client.0 _ll_register_callbacks cb 0x7f712c0bcd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:31.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:31.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f7137fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf65fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48182/0 (socket says 172.21.15.33:48182) 2024-04-23T18:40:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 -- 172.21.15.33:0/366369985 learned_addr learned my addr 172.21.15.33:0/366369985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71377fe640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf65ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:31.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 -- 172.21.15.33:0/366369985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 msgr2=0x5612bf65fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf65fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:31.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 -- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 msgr2=0x5612bf65ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:31.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf65ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 -- 172.21.15.33:0/366369985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612bf6cf330 con 0x5612bf659800 2024-04-23T18:40:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71377fe640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf65ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:32.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f713cbbb640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71300092a0 tx=0x7f71300144f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120a30c06de8847 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71177fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:32.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130012c70 con 0x5612bf659800 2024-04-23T18:40:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8498+0+0 (secure 0 0 0) 0x7f713001ec50 con 0x5612bf659800 2024-04-23T18:40:32.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71280054d0 con 0x5612bf659800 2024-04-23T18:40:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7130041050 con 0x5612bf659800 2024-04-23T18:40:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f714140d740 10 client.17106 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.730+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5612bf6d05b0 con 0x5612bf659800 2024-04-23T18:40:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71280054d0 con 0x5612bf659800 2024-04-23T18:40:32.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f713003c030 con 0x5612bf659800 2024-04-23T18:40:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 10 client.17106 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:32.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 20 client.17106 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:32.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6d0c10 0x5612bf6d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 4 client.17106 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612bf5bb200 con 0x5612bf6d0c10 2024-04-23T18:40:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f7137fff640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6d0c10 0x5612bf6d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f71177fe640 10 client.17106 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.731+0000 7f7137fff640 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6d0c10 0x5612bf6d3030 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0544c86fb8aaa9e server_cookie=28468d63bebf75df in_seq=0 out_seq=0 2024-04-23T18:40:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.735+0000 7f71177fe640 1 -- 172.21.15.33:0/366369985 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612bf5bb200 con 0x5612bf6d0c10 2024-04-23T18:40:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.735+0000 7f71177fe640 10 client.17106 handle_command_reply: tid=0 2024-04-23T18:40:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.735+0000 7f714140d740 1 client.17106 shutdown 2024-04-23T18:40:33.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 msgr2=0x5612bf6ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:33.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f71300092a0 tx=0x7f71300144f0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 shutdown_connections 2024-04-23T18:40:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5612bf6d0c10 0x5612bf6d3030 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612bf659800 0x5612bf6ceda0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5612bf64ef40 0x5612bf65ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 --2- 172.21.15.33:0/366369985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612bf64e620 0x5612bf65fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:33.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f71177fe640 0 client.17106 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 >> 172.21.15.33:0/366369985 conn(0x5612bf64ab30 msgr2=0x5612bf6642e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 shutdown_connections 2024-04-23T18:40:33.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 1 -- 172.21.15.33:0/366369985 wait complete. 2024-04-23T18:40:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f714140d740 20 client.17106 trim_cache size 0 max 16384 2024-04-23T18:40:34.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:40:34.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:40:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:40:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:40:34.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 msgr2=0x7f71200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 0x7f71200a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7130022000 tx=0x7f713001e000 comp rx=0 tx=0).stop 2024-04-23T18:40:34.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 msgr2=0x7f7138198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.736+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f712c005ab0 tx=0x7f712c003cd0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 shutdown_connections 2024-04-23T18:40:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f71200a3dc0 0x7f71200a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f713819cf40 0x7f713819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138107290 0x7f7138198bc0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 --2- 172.21.15.33:0/2921520898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71381021c0 0x7f7138198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 >> 172.21.15.33:0/2921520898 conn(0x7f71380fda20 msgr2=0x7f71380fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 shutdown_connections 2024-04-23T18:40:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:38:55.737+0000 7f713e61e640 1 -- 172.21.15.33:0/2921520898 wait complete. 2024-04-23T18:40:34.955 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:56 smithi033 ceph-mon[29274]: pgmap v393: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.8 KiB/s wr, 14 op/s 2024-04-23T18:40:35.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:38:58 smithi033 ceph-mon[29274]: pgmap v394: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:40:35.096 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:00 smithi033 ceph-mon[29274]: pgmap v395: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:40:35.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:02 smithi033 ceph-mon[29274]: pgmap v396: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 47 KiB/s wr, 29 op/s 2024-04-23T18:40:35.213 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:04 smithi033 ceph-mon[29274]: pgmap v397: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:40:35.271 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:06 smithi033 ceph-mon[29274]: pgmap v398: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:40:35.341 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:40:35.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:22 smithi106 ceph-mon[32716]: pgmap v440: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-23T18:40:35.563 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:24 smithi106 ceph-mon[32716]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T18:40:35.596 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:26 smithi106 ceph-mon[32716]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T18:40:35.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:28 smithi106 ceph-mon[32716]: pgmap v443: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:40:35.663 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:30 smithi106 ceph-mon[32716]: pgmap v444: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:40:35.696 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:32 smithi106 ceph-mon[32716]: pgmap v445: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-23T18:40:35.730 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:34 smithi106 ceph-mon[32716]: pgmap v446: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T18:40:35.763 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:22 smithi122 ceph-mon[32546]: pgmap v440: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-23T18:40:35.797 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:24 smithi122 ceph-mon[32546]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T18:40:35.830 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:26 smithi122 ceph-mon[32546]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T18:40:35.863 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:28 smithi122 ceph-mon[32546]: pgmap v443: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:40:35.896 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:30 smithi122 ceph-mon[32546]: pgmap v444: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:40:35.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:32 smithi122 ceph-mon[32546]: pgmap v445: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-23T18:40:35.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:34 smithi122 ceph-mon[32546]: pgmap v446: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T18:40:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- 172.21.15.33:0/856594556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401010e0 msgr2=0x7fbb401014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/856594556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401010e0 0x7fbb401014c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c02a410 comp rx=0 tx=0).stop 2024-04-23T18:40:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- 172.21.15.33:0/856594556 shutdown_connections 2024-04-23T18:40:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/856594556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb401061b0 0x7fbb4010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/856594556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb40101a90 0x7fbb40105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/856594556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401010e0 0x7fbb401014c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- 172.21.15.33:0/856594556 >> 172.21.15.33:0/856594556 conn(0x7fbb400771d0 msgr2=0x7fbb400775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:36.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- 172.21.15.33:0/856594556 shutdown_connections 2024-04-23T18:40:36.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- 172.21.15.33:0/856594556 wait complete. 2024-04-23T18:40:36.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 Processor -- start 2024-04-23T18:40:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.839+0000 7fbb47aa3640 1 -- start start 2024-04-23T18:40:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 0x7fbb401986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 0x7fbb4019d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40112230 con 0x7fbb401061b0 2024-04-23T18:40:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401120b0 con 0x7fbb4019cf20 2024-04-23T18:40:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb47aa3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401123b0 con 0x7fbb401010e0 2024-04-23T18:40:36.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52052/0 (socket says 172.21.15.33:52052) 2024-04-23T18:40:36.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 -- 172.21.15.33:0/2073783708 learned_addr learned my addr 172.21.15.33:0/2073783708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb46040640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 0x7fbb4019d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:37.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 0x7fbb401986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:37.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 -- 172.21.15.33:0/2073783708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 msgr2=0x7fbb401986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 0x7fbb401986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 -- 172.21.15.33:0/2073783708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 msgr2=0x7fbb4019d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:37.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 0x7fbb4019d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 -- 172.21.15.33:0/2073783708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019da10 con 0x7fbb401061b0 2024-04-23T18:40:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 0x7fbb401986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:37.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb46040640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 0x7fbb4019d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:37.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.840+0000 7fbb4503e640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb3000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:37.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.841+0000 7fbb36ffd640 1 -- 172.21.15.33:0/2073783708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30017070 con 0x7fbb401061b0 2024-04-23T18:40:37.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.841+0000 7fbb36ffd640 1 -- 172.21.15.33:0/2073783708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb30009c30 con 0x7fbb401061b0 2024-04-23T18:40:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.841+0000 7fbb36ffd640 1 -- 172.21.15.33:0/2073783708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb30003d30 con 0x7fbb401061b0 2024-04-23T18:40:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.841+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019dd00 con 0x7fbb401061b0 2024-04-23T18:40:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.841+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a54f0 con 0x7fbb401061b0 2024-04-23T18:40:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.843+0000 7fbb36ffd640 1 -- 172.21.15.33:0/2073783708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb30003ed0 con 0x7fbb401061b0 2024-04-23T18:40:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 Processor -- start 2024-04-23T18:40:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 -- start start 2024-04-23T18:40:38.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f096aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f0977bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:38.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578f08cf970 con 0x5578f096af40 2024-04-23T18:40:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578f086c4a0 con 0x5578f0975800 2024-04-23T18:40:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578f08cb930 con 0x5578f096a620 2024-04-23T18:40:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb36ffd640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4583f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:38.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4583f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52062/0 (socket says 172.21.15.33:52062) 2024-04-23T18:40:38.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4583f640 1 -- 172.21.15.33:0/262007239 learned_addr learned my addr 172.21.15.33:0/262007239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb46040640 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f096aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:38.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb46040640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.844+0000 7fbb4503e640 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f0977bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb36ffd640 1 -- 172.21.15.33:0/2073783708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7fbb300d05a0 con 0x7fbb401061b0 2024-04-23T18:40:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 -- 172.21.15.33:0/262007239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 msgr2=0x5578f096aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f096aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 -- 172.21.15.33:0/262007239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 msgr2=0x5578f0977bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f0977bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 -- 172.21.15.33:0/262007239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f05e2210 con 0x5578f096af40 2024-04-23T18:40:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb4583f640 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b8c776bf7bf6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb46040640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009fd0 tx=0x7fbb38005bd0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb2bfff640 1 -- 172.21.15.33:0/262007239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c03f070 con 0x5578f096af40 2024-04-23T18:40:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb2bfff640 1 -- 172.21.15.33:0/262007239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c0316a0 con 0x5578f096af40 2024-04-23T18:40:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.845+0000 7fbb2bfff640 1 -- 172.21.15.33:0/262007239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c035070 con 0x5578f096af40 2024-04-23T18:40:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/262007239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 msgr2=0x5578f09752c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009900 tx=0x7fbb3c032cd0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/262007239 shutdown_connections 2024-04-23T18:40:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f0977bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096af40 0x5578f09752c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/262007239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f096aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/262007239 >> 172.21.15.33:0/262007239 conn(0x5578f0966b30 msgr2=0x5578f0966f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/262007239 shutdown_connections 2024-04-23T18:40:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/262007239 wait complete. 2024-04-23T18:40:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 -- start start 2024-04-23T18:40:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.846+0000 7fbb4a7b7740 1 Processor -- start 2024-04-23T18:40:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096af40 0x5578f0972450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f09729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578f09783d0 con 0x5578f096a620 2024-04-23T18:40:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5578f086a0b0 con 0x5578f0975800 2024-04-23T18:40:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578f0978550 con 0x5578f096af40 2024-04-23T18:40:40.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4a7b7740 10 client.0 _ll_register_callbacks cb 0x5578f0857d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:40.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52066/0 (socket says 172.21.15.33:52066) 2024-04-23T18:40:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 -- 172.21.15.33:0/3572449793 learned_addr learned my addr 172.21.15.33:0/3572449793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:40.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4503e640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096af40 0x5578f0972450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb46040640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f09729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 -- 172.21.15.33:0/3572449793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096af40 msgr2=0x5578f0972450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096af40 0x5578f0972450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 -- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 msgr2=0x5578f09729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f09729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:41.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.847+0000 7fbb4583f640 1 -- 172.21.15.33:0/3572449793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f06ae620 con 0x5578f096a620 2024-04-23T18:40:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb4583f640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217f16d5b76c7fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb297fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c04a020 con 0x5578f096a620 2024-04-23T18:40:41.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7fbb3c0035f0 con 0x5578f096a620 2024-04-23T18:40:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb3c0316a0 con 0x5578f096a620 2024-04-23T18:40:41.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb3c055050 con 0x5578f096a620 2024-04-23T18:40:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb4a7b7740 10 client.17118 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578f0984300 con 0x5578f096a620 2024-04-23T18:40:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.848+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c0316a0 con 0x5578f096a620 2024-04-23T18:40:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbb3c03b870 con 0x5578f096a620 2024-04-23T18:40:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 10 client.17118 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:41.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 10 client.17118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 20 client.17118 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f0985650 0x5578f0987a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 4 client.17118 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f08dd390 con 0x5578f0985650 2024-04-23T18:40:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4503e640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f0985650 0x5578f0987a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb297fa640 10 client.17118 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:41.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.849+0000 7fbb4503e640 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f0985650 0x5578f0987a40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a95ae419f701f86c server_cookie=ac13957843285a62 in_seq=0 out_seq=0 2024-04-23T18:40:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb297fa640 1 -- 172.21.15.33:0/3572449793 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578f08dd390 con 0x5578f0985650 2024-04-23T18:40:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb297fa640 10 client.17118 handle_command_reply: tid=0 2024-04-23T18:40:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 client.17118 shutdown 2024-04-23T18:40:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 msgr2=0x5578f0975160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c000c00 tx=0x7fbb3c004520 comp rx=0 tx=0).stop 2024-04-23T18:40:42.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 shutdown_connections 2024-04-23T18:40:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f0985650 0x5578f0987a40 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f0975800 0x5578f09729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096af40 0x5578f0972450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb297fa640 0 client.17118 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/3572449793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f096a620 0x5578f0975160 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:42.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 >> 172.21.15.33:0/3572449793 conn(0x5578f0966b30 msgr2=0x5578f096c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.850+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 shutdown_connections 2024-04-23T18:40:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.851+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/3572449793 wait complete. 2024-04-23T18:40:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.851+0000 7fbb4a7b7740 20 client.17118 trim_cache size 0 max 16384 2024-04-23T18:40:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.865+0000 7fbb4a7b7740 1 Processor -- start 2024-04-23T18:40:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 -- start start 2024-04-23T18:40:42.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f0981290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f0965b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401adb60 con 0x5578f0975800 2024-04-23T18:40:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401af6a0 con 0x5578f096af40 2024-04-23T18:40:43.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401ae230 con 0x5578f096a620 2024-04-23T18:40:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52080/0 (socket says 172.21.15.33:52080) 2024-04-23T18:40:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 -- 172.21.15.33:0/1713865674 learned_addr learned my addr 172.21.15.33:0/1713865674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4583f640 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f0981290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb4503e640 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f0965b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 -- 172.21.15.33:0/1713865674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 msgr2=0x5578f0981290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f0981290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:43.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 -- 172.21.15.33:0/1713865674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 msgr2=0x5578f0965b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:43.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f0965b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:43.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 -- 172.21.15.33:0/1713865674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f0972d20 con 0x5578f0975800 2024-04-23T18:40:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.866+0000 7fbb46040640 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbb380129f0 tx=0x7fbb380096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4256d5fe1c9c2011 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb2b7fe640 1 -- 172.21.15.33:0/1713865674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802e070 con 0x5578f0975800 2024-04-23T18:40:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb2b7fe640 1 -- 172.21.15.33:0/1713865674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb38021da0 con 0x5578f0975800 2024-04-23T18:40:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb2b7fe640 1 -- 172.21.15.33:0/1713865674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38029c50 con 0x5578f0975800 2024-04-23T18:40:43.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/1713865674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 msgr2=0x5578f098d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbb380129f0 tx=0x7fbb380096f0 comp rx=0 tx=0).stop 2024-04-23T18:40:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/1713865674 shutdown_connections 2024-04-23T18:40:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f098d070 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:44.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f0965b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:44.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/1713865674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f0981290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/1713865674 >> 172.21.15.33:0/1713865674 conn(0x5578f0966b30 msgr2=0x5578f0965420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:44.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/1713865674 shutdown_connections 2024-04-23T18:40:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.867+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/1713865674 wait complete. 2024-04-23T18:40:44.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 -- start start 2024-04-23T18:40:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 Processor -- start 2024-04-23T18:40:44.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f097d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:44.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f097ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401aefd0 con 0x5578f0975800 2024-04-23T18:40:44.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401ae900 con 0x5578f096af40 2024-04-23T18:40:44.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401abb60 con 0x5578f096a620 2024-04-23T18:40:44.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4a7b7740 10 client.0 _ll_register_callbacks cb 0x7fbb3c0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb46040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb46040640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52088/0 (socket says 172.21.15.33:52088) 2024-04-23T18:40:45.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb46040640 1 -- 172.21.15.33:0/2675173124 learned_addr learned my addr 172.21.15.33:0/2675173124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:45.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4503e640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f097ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f097d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb46040640 1 -- 172.21.15.33:0/2675173124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 msgr2=0x5578f097d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:45.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.868+0000 7fbb46040640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f097d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb46040640 1 -- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 msgr2=0x5578f097ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:45.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb46040640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f097ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:45.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb46040640 1 -- 172.21.15.33:0/2675173124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f097b790 con 0x5578f0975800 2024-04-23T18:40:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f097d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb46040640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbb3802c040 tx=0x7fbb380127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899bc40c7c815dd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb297fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802e070 con 0x5578f0975800 2024-04-23T18:40:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb38021da0 con 0x5578f0975800 2024-04-23T18:40:45.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8586+0+0 (secure 0 0 0) 0x7fbb38004a40 con 0x5578f0975800 2024-04-23T18:40:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb38004c40 con 0x5578f0975800 2024-04-23T18:40:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb4a7b7740 10 client.17124 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578f09eb690 con 0x5578f0975800 2024-04-23T18:40:46.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.869+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb38021da0 con 0x5578f0975800 2024-04-23T18:40:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbb38028030 con 0x5578f0975800 2024-04-23T18:40:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 10 client.17124 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:46.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 10 client.17124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:46.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 20 client.17124 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f09ebd80 0x5578f09ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 4 client.17124 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:40:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578f08d7200 con 0x5578f09ebd80 2024-04-23T18:40:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f09ebd80 0x5578f09ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb297fa640 10 client.17124 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:46.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.870+0000 7fbb4583f640 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f09ebd80 0x5578f09ee1a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b75bde7105195f server_cookie=68f54c6a5d044a31 in_seq=0 out_seq=0 2024-04-23T18:40:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.886+0000 7fbb297fa640 1 -- 172.21.15.33:0/2675173124 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578f08d7200 con 0x5578f09ebd80 2024-04-23T18:40:46.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.886+0000 7fbb297fa640 10 client.17124 handle_command_reply: tid=0 2024-04-23T18:40:46.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.886+0000 7fbb4a7b7740 1 client.17124 shutdown 2024-04-23T18:40:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 msgr2=0x5578f097b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fbb3802c040 tx=0x7fbb380127f0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 shutdown_connections 2024-04-23T18:40:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5578f09ebd80 0x5578f09ee1a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578f0975800 0x5578f097b1d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb297fa640 0 client.17124 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:47.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5578f096af40 0x5578f097ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 --2- 172.21.15.33:0/2675173124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578f096a620 0x5578f097d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 >> 172.21.15.33:0/2675173124 conn(0x5578f0966b30 msgr2=0x5578f0965cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:47.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 shutdown_connections 2024-04-23T18:40:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 1 -- 172.21.15.33:0/2675173124 wait complete. 2024-04-23T18:40:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb4a7b7740 20 client.17124 trim_cache size 0 max 16384 2024-04-23T18:40:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:40:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:40:47.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:40:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:40:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 msgr2=0x7fbb2c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009fd0 tx=0x7fbb38005bd0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 msgr2=0x7fbb40198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.887+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb3000dde0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 shutdown_connections 2024-04-23T18:40:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb4019cf20 0x7fbb4019d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb401061b0 0x7fbb40198bf0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 --2- 172.21.15.33:0/2073783708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401010e0 0x7fbb401986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 >> 172.21.15.33:0/2073783708 conn(0x7fbb400771d0 msgr2=0x7fbb400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 shutdown_connections 2024-04-23T18:40:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:08.888+0000 7fbb47aa3640 1 -- 172.21.15.33:0/2073783708 wait complete. 2024-04-23T18:40:48.130 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:08 smithi033 ceph-mon[29274]: pgmap v399: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 26 op/s 2024-04-23T18:40:48.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:09 smithi033 ceph-mon[29274]: osdmap e106: 12 total, 12 up, 12 in 2024-04-23T18:40:48.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:10 smithi033 ceph-mon[29274]: pgmap v401: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:40:48.279 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:12 smithi033 ceph-mon[29274]: pgmap v402: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:40:48.382 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:14 smithi033 ceph-mon[29274]: pgmap v403: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:40:48.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:16 smithi033 ceph-mon[29274]: pgmap v404: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:40:48.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:16 smithi033 ceph-mon[29274]: osdmap e107: 12 total, 12 up, 12 in 2024-04-23T18:40:48.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:18 smithi033 ceph-mon[29274]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:40:48.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:20 smithi033 ceph-mon[29274]: pgmap v407: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-04-23T18:40:48.632 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:36 smithi106 ceph-mon[32716]: pgmap v447: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-23T18:40:48.679 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:38 smithi106 ceph-mon[32716]: pgmap v448: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:40:48.729 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:40 smithi106 ceph-mon[32716]: pgmap v449: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:40:48.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:42 smithi106 ceph-mon[32716]: pgmap v450: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:40:48.829 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:44 smithi106 ceph-mon[32716]: pgmap v451: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:40:48.880 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:46 smithi106 ceph-mon[32716]: pgmap v452: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:40:48.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:36 smithi122 ceph-mon[32546]: pgmap v447: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-23T18:40:49.038 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:38 smithi122 ceph-mon[32546]: pgmap v448: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:40:49.088 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:40 smithi122 ceph-mon[32546]: pgmap v449: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:40:49.138 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:42 smithi122 ceph-mon[32546]: pgmap v450: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:40:49.188 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:44 smithi122 ceph-mon[32546]: pgmap v451: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:40:49.238 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:46 smithi122 ceph-mon[32546]: pgmap v452: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:40:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 -- 172.21.15.33:0/3422595710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7010e910 msgr2=0x7f7a70110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 --2- 172.21.15.33:0/3422595710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7010e910 0x7f7a70110d00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009ca0 tx=0x7f7a6802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 -- 172.21.15.33:0/3422595710 shutdown_connections 2024-04-23T18:40:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 --2- 172.21.15.33:0/3422595710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a7010e910 0x7f7a70110d00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 --2- 172.21.15.33:0/3422595710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 0x7f7a7010e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 --2- 172.21.15.33:0/3422595710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a70100d60 0x7f7a70101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:49.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 -- 172.21.15.33:0/3422595710 >> 172.21.15.33:0/3422595710 conn(0x7f7a70077fd0 msgr2=0x7f7a700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:49.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 -- 172.21.15.33:0/3422595710 shutdown_connections 2024-04-23T18:40:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 -- 172.21.15.33:0/3422595710 wait complete. 2024-04-23T18:40:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.988+0000 7f7a7610f640 1 Processor -- start 2024-04-23T18:40:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 -- start start 2024-04-23T18:40:49.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:49.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 0x7f7a701a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:50.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7010e910 0x7f7a7019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701143b0 con 0x7f7a70100d60 2024-04-23T18:40:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70114230 con 0x7f7a70101680 2024-04-23T18:40:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a7610f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70114530 con 0x7f7a7010e910 2024-04-23T18:40:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55880/0 (socket says 172.21.15.33:55880) 2024-04-23T18:40:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/4002498652 learned_addr learned my addr 172.21.15.33:0/4002498652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6effd640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 0x7f7a701a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7010e910 0x7f7a7019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/4002498652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7010e910 msgr2=0x7f7a7019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:50.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7010e910 0x7f7a7019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/4002498652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 msgr2=0x7f7a701a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:50.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 0x7f7a701a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.989+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/4002498652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7019af90 con 0x7f7a70100d60 2024-04-23T18:40:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a6cff9640 1 -- 172.21.15.33:0/4002498652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60017070 con 0x7f7a70100d60 2024-04-23T18:40:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a6cff9640 1 -- 172.21.15.33:0/4002498652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a60009c30 con 0x7f7a70100d60 2024-04-23T18:40:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7019b280 con 0x7f7a70100d60 2024-04-23T18:40:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a6cff9640 1 -- 172.21.15.33:0/4002498652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000f620 con 0x7f7a70100d60 2024-04-23T18:40:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.990+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a7540 con 0x7f7a70100d60 2024-04-23T18:40:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.992+0000 7f7a6cff9640 1 -- 172.21.15.33:0/4002498652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7a60010460 con 0x7f7a70100d60 2024-04-23T18:40:51.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 Processor -- start 2024-04-23T18:40:51.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 -- start start 2024-04-23T18:40:51.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d05ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:51.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:51.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55637cfc3970 con 0x55637d05ef40 2024-04-23T18:40:51.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55637cf604a0 con 0x55637d05e620 2024-04-23T18:40:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a78e21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55637cfbf930 con 0x55637d069800 2024-04-23T18:40:51.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55888/0 (socket says 172.21.15.33:55888) 2024-04-23T18:40:51.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/1705539000 learned_addr learned my addr 172.21.15.33:0/1705539000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d05ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6effd640 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6cff9640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 0x7f7a580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 0x7f7a580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/1705539000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 msgr2=0x55637d06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:52.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.993+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/1705539000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 msgr2=0x55637d05ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d05ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/1705539000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637ccd6210 con 0x55637d05ef40 2024-04-23T18:40:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6cff9640 1 -- 172.21.15.33:0/4002498652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f7a600d0490 con 0x7f7a70100d60 2024-04-23T18:40:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7a60098dd0 tx=0x7f7a6006adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671e164d341d8b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 0x7f7a580a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005cc0 tx=0x7f7a6802a8c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a55ffb640 1 -- 172.21.15.33:0/1705539000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60020070 con 0x55637d05ef40 2024-04-23T18:40:52.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a55ffb640 1 -- 172.21.15.33:0/1705539000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6009cb00 con 0x55637d05ef40 2024-04-23T18:40:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.994+0000 7f7a55ffb640 1 -- 172.21.15.33:0/1705539000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6001e120 con 0x55637d05ef40 2024-04-23T18:40:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- 172.21.15.33:0/1705539000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 msgr2=0x55637d0692c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7a60098dd0 tx=0x7f7a6006adb0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- 172.21.15.33:0/1705539000 shutdown_connections 2024-04-23T18:40:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0692c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 --2- 172.21.15.33:0/1705539000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d05ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- 172.21.15.33:0/1705539000 >> 172.21.15.33:0/1705539000 conn(0x55637d05ab30 msgr2=0x55637d05af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- 172.21.15.33:0/1705539000 shutdown_connections 2024-04-23T18:40:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- 172.21.15.33:0/1705539000 wait complete. 2024-04-23T18:40:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.995+0000 7f7a78e21740 1 -- start start 2024-04-23T18:40:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 Processor -- start 2024-04-23T18:40:53.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d066420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55637d06c3d0 con 0x55637d069800 2024-04-23T18:40:53.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55637cf5e0b0 con 0x55637d05ef40 2024-04-23T18:40:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55637d06c550 con 0x55637d05e620 2024-04-23T18:40:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a78e21740 10 client.0 _ll_register_callbacks cb 0x55637cf4bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55894/0 (socket says 172.21.15.33:55894) 2024-04-23T18:40:54.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:37656/0 (socket says 172.21.15.33:37656) 2024-04-23T18:40:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6ffff640 1 -- 172.21.15.33:0/2887031515 learned_addr learned my addr 172.21.15.33:0/2887031515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 msgr2=0x55637d066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d066420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/2887031515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 msgr2=0x55637d066960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/2887031515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637cda2620 con 0x55637d05e620 2024-04-23T18:40:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.996+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:40:54.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55637d05fbc0 tx=0x7f7a600cdbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec1293f1bf179ff server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:40:54.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60020040 con 0x55637d05e620 2024-04-23T18:40:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f7a600d3c10 con 0x55637d05e620 2024-04-23T18:40:54.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6009cb00 con 0x55637d05e620 2024-04-23T18:40:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7a6009c400 con 0x55637d05e620 2024-04-23T18:40:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 10 client.26560 fetch_fsmap learned FSMap version 36 2024-04-23T18:40:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55637d06c3d0 con 0x55637d05e620 2024-04-23T18:40:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6009cb00 con 0x55637d05e620 2024-04-23T18:40:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7a600cde30 con 0x55637d05e620 2024-04-23T18:40:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 10 client.26560 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:40:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 10 client.26560 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:40:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 20 client.26560 populate_metadata read hostname 'smithi033' 2024-04-23T18:40:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d07b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 4 client.26560 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:40:55.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.997+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637cfd1390 con 0x55637d0794a0 2024-04-23T18:40:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.998+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d07b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.998+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d07b890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce0e4ef4e2c1e60 server_cookie=98809a21640e1a28 in_seq=0 out_seq=0 2024-04-23T18:40:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.998+0000 7f7a4effd640 10 client.26560 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a4effd640 1 -- 172.21.15.33:0/2887031515 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55637cfd1390 con 0x55637d0794a0 2024-04-23T18:40:55.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a4effd640 10 client.26560 handle_command_reply: tid=0 2024-04-23T18:40:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 client.26560 shutdown 2024-04-23T18:40:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 msgr2=0x55637d069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55637d05fbc0 tx=0x7f7a600cdbc0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 shutdown_connections 2024-04-23T18:40:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d07b890 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d069800 0x55637d066960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a4effd640 0 client.26560 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:40:56.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2887031515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d05e620 0x55637d069130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 >> 172.21.15.33:0/2887031515 conn(0x55637d05ab30 msgr2=0x55637d0608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 shutdown_connections 2024-04-23T18:40:56.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 1 -- 172.21.15.33:0/2887031515 wait complete. 2024-04-23T18:40:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:21.999+0000 7f7a78e21740 20 client.26560 trim_cache size 0 max 16384 2024-04-23T18:40:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.014+0000 7f7a78e21740 1 Processor -- start 2024-04-23T18:40:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.014+0000 7f7a78e21740 1 -- start start 2024-04-23T18:40:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.014+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d068c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:56.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:56.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a78e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701afe80 con 0x55637d05ef40 2024-04-23T18:40:56.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a78e21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b19c0 con 0x55637d05e620 2024-04-23T18:40:57.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a78e21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b0550 con 0x55637d069800 2024-04-23T18:40:57.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55898/0 (socket says 172.21.15.33:55898) 2024-04-23T18:40:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 -- 172.21.15.33:0/2220384977 learned_addr learned my addr 172.21.15.33:0/2220384977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:57.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d068c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 -- 172.21.15.33:0/2220384977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 msgr2=0x55637d071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:57.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d071ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 -- 172.21.15.33:0/2220384977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 msgr2=0x55637d068c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:57.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d068c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 -- 172.21.15.33:0/2220384977 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637d06c010 con 0x55637d05ef40 2024-04-23T18:40:57.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:40:57.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a6effd640 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55637d05bcc0 tx=0x7f7a6400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abfbf73f265d72d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a557fa640 1 -- 172.21.15.33:0/2220384977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400bc80 con 0x55637d05ef40 2024-04-23T18:40:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a557fa640 1 -- 172.21.15.33:0/2220384977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64024020 con 0x55637d05ef40 2024-04-23T18:40:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.015+0000 7f7a557fa640 1 -- 172.21.15.33:0/2220384977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640118a0 con 0x55637d05ef40 2024-04-23T18:40:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- 172.21.15.33:0/2220384977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 msgr2=0x55637d0717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:40:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55637d05bcc0 tx=0x7f7a6400b390 comp rx=0 tx=0).stop 2024-04-23T18:40:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- 172.21.15.33:0/2220384977 shutdown_connections 2024-04-23T18:40:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05ef40 0x55637d0717a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 --2- 172.21.15.33:0/2220384977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05e620 0x55637d068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- 172.21.15.33:0/2220384977 >> 172.21.15.33:0/2220384977 conn(0x55637d05ab30 msgr2=0x55637d066780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:40:58.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- 172.21.15.33:0/2220384977 shutdown_connections 2024-04-23T18:40:58.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- 172.21.15.33:0/2220384977 wait complete. 2024-04-23T18:40:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.016+0000 7f7a78e21740 1 -- start start 2024-04-23T18:40:58.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 Processor -- start 2024-04-23T18:40:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d0de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:40:58.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b12f0 con 0x55637d05e620 2024-04-23T18:40:58.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b0c20 con 0x55637d05ef40 2024-04-23T18:40:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701ade80 con 0x55637d069800 2024-04-23T18:40:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a78e21740 10 client.0 _ll_register_callbacks cb 0x7f7a6009bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:40:58.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55906/0 (socket says 172.21.15.33:55906) 2024-04-23T18:40:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/3599784916 learned_addr learned my addr 172.21.15.33:0/3599784916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:40:58.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6effd640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:58.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d0de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:40:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/3599784916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 msgr2=0x55637d0de8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:58.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d0de8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 msgr2=0x55637d06fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:40:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d06fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:40:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 -- 172.21.15.33:0/3599784916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637d0dee10 con 0x55637d05e620 2024-04-23T18:40:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6ffff640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d0de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a6f7fe640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7a60098860 tx=0x7f7a60005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c057e284951177 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:40:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.017+0000 7f7a4effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:40:59.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60020050 con 0x55637d05e620 2024-04-23T18:40:59.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f7a600bb2b0 con 0x55637d05e620 2024-04-23T18:40:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a64024020 con 0x55637d05e620 2024-04-23T18:40:59.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7a6009cb00 con 0x55637d05e620 2024-04-23T18:40:59.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 10 client.17142 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55637d0dee10 con 0x55637d05e620 2024-04-23T18:41:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a64024020 con 0x55637d05e620 2024-04-23T18:41:00.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7a600cde30 con 0x55637d05e620 2024-04-23T18:41:00.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 10 client.17142 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:00.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 10 client.17142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 20 client.17142 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:00.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d0e2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 4 client.17142 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55637cfcb200 con 0x55637d0794a0 2024-04-23T18:41:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.018+0000 7f7a6effd640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d0e2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:00.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.019+0000 7f7a4effd640 10 client.17142 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.019+0000 7f7a6effd640 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d0e2530 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f281d64a8a4a6d0 server_cookie=f041cebe1fd4c580 in_seq=0 out_seq=0 2024-04-23T18:41:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a4effd640 1 -- 172.21.15.33:0/3599784916 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55637cfcb200 con 0x55637d0794a0 2024-04-23T18:41:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a4effd640 10 client.17142 handle_command_reply: tid=0 2024-04-23T18:41:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 client.17142 shutdown 2024-04-23T18:41:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 msgr2=0x55637d06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7a60098860 tx=0x7f7a60005b20 comp rx=0 tx=0).stop 2024-04-23T18:41:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 shutdown_connections 2024-04-23T18:41:01.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55637d0794a0 0x55637d0e2530 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55637d069800 0x55637d0de8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55637d05ef40 0x55637d06fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a4effd640 0 client.17142 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 --2- 172.21.15.33:0/3599784916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55637d05e620 0x55637d06f8f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 >> 172.21.15.33:0/3599784916 conn(0x55637d05ab30 msgr2=0x55637d074b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:01.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.023+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 shutdown_connections 2024-04-23T18:41:01.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a78e21740 1 -- 172.21.15.33:0/3599784916 wait complete. 2024-04-23T18:41:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a78e21740 20 client.17142 trim_cache size 0 max 16384 2024-04-23T18:41:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:41:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:41:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:41:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:41:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 msgr2=0x7f7a580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:01.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 0x7f7a580a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a68005cc0 tx=0x7f7a6802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:41:01.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 msgr2=0x7f7a701a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000d950 tx=0x7f7a6000de10 comp rx=0 tx=0).stop 2024-04-23T18:41:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 shutdown_connections 2024-04-23T18:41:02.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7a580a3dc0 0x7f7a580a6280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a7010e910 0x7f7a7019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a70101680 0x7f7a701a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 --2- 172.21.15.33:0/4002498652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a70100d60 0x7f7a701a05c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:02.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 >> 172.21.15.33:0/4002498652 conn(0x7f7a70077fd0 msgr2=0x7f7a7010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 shutdown_connections 2024-04-23T18:41:02.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:22.024+0000 7f7a7610f640 1 -- 172.21.15.33:0/4002498652 wait complete. 2024-04-23T18:41:02.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:22 smithi033 ceph-mon[29274]: pgmap v408: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:41:02.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:24 smithi033 ceph-mon[29274]: pgmap v409: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:41:02.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:26 smithi033 ceph-mon[29274]: pgmap v410: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-23T18:41:02.477 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:28 smithi033 ceph-mon[29274]: pgmap v411: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:41:02.504 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:30 smithi033 ceph-mon[29274]: pgmap v412: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:02.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:32 smithi033 ceph-mon[29274]: pgmap v413: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:41:02.591 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:48 smithi122 ceph-mon[32546]: pgmap v453: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:02.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:49 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:02.654 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:49 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:02.688 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:49 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:02.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:49 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:41:02.896 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:50 smithi122 ceph-mon[32546]: pgmap v454: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:02.963 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:52 smithi122 ceph-mon[32546]: pgmap v455: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:41:02.996 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:54 smithi122 ceph-mon[32546]: pgmap v456: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:03.061 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:56 smithi122 ceph-mon[32546]: pgmap v457: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:41:03.107 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:40:58 smithi122 ceph-mon[32546]: pgmap v458: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:41:03.146 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:00 smithi122 ceph-mon[32546]: pgmap v459: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:41:03.224 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:48 smithi106 ceph-mon[32716]: pgmap v453: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:03.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:49 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:03.313 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:49 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:03.346 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:49 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:03.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:49 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:41:03.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:50 smithi106 ceph-mon[32716]: pgmap v454: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:03.563 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:52 smithi106 ceph-mon[32716]: pgmap v455: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:41:03.613 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:54 smithi106 ceph-mon[32716]: pgmap v456: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:03.663 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:56 smithi106 ceph-mon[32716]: pgmap v457: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:41:03.713 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:40:58 smithi106 ceph-mon[32716]: pgmap v458: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:41:03.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:00 smithi106 ceph-mon[32716]: pgmap v459: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:41:03.815 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:02 smithi122 ceph-mon[32546]: pgmap v460: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:41:03.868 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:02 smithi106 ceph-mon[32716]: pgmap v460: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:41:03.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.123+0000 7fbb18da2640 1 -- 172.21.15.33:0/390328558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb14100700 msgr2=0x7fbb14100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.123+0000 7fbb18da2640 1 --2- 172.21.15.33:0/390328558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb14100700 0x7fbb14100ae0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009900 tx=0x7fbb0c02a3f0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 -- 172.21.15.33:0/390328558 shutdown_connections 2024-04-23T18:41:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 --2- 172.21.15.33:0/390328558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 0x7fbb1410ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 --2- 172.21.15.33:0/390328558 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb141010b0 0x7fbb14106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 --2- 172.21.15.33:0/390328558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb14100700 0x7fbb14100ae0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 -- 172.21.15.33:0/390328558 >> 172.21.15.33:0/390328558 conn(0x7fbb14077370 msgr2=0x7fbb140fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:04.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 -- 172.21.15.33:0/390328558 shutdown_connections 2024-04-23T18:41:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 -- 172.21.15.33:0/390328558 wait complete. 2024-04-23T18:41:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.124+0000 7fbb18da2640 1 Processor -- start 2024-04-23T18:41:04.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 -- start start 2024-04-23T18:41:04.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb14100700 0x7fbb14101670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:04.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 0x7fbb14101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:04.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb14112250 con 0x7fbb141020f0 2024-04-23T18:41:04.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141120d0 con 0x7fbb14100700 2024-04-23T18:41:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb18da2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141123d0 con 0x7fbb14107310 2024-04-23T18:41:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51682/0 (socket says 172.21.15.33:51682) 2024-04-23T18:41:04.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 -- 172.21.15.33:0/2686584100 learned_addr learned my addr 172.21.15.33:0/2686584100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:05.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb14100700 0x7fbb14101670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 0x7fbb14101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 -- 172.21.15.33:0/2686584100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 msgr2=0x7fbb14101bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 0x7fbb14101bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 -- 172.21.15.33:0/2686584100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb14100700 msgr2=0x7fbb14101670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb14100700 0x7fbb14101670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:05.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 -- 172.21.15.33:0/2686584100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141a6fa0 con 0x7fbb141020f0 2024-04-23T18:41:05.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.125+0000 7fbb12d9d640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400cce0 tx=0x7fbb0400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:05.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.126+0000 7fbb0b7fe640 1 -- 172.21.15.33:0/2686584100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400f450 con 0x7fbb141020f0 2024-04-23T18:41:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.126+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141a7290 con 0x7fbb141020f0 2024-04-23T18:41:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.126+0000 7fbb0b7fe640 1 -- 172.21.15.33:0/2686584100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0400faa0 con 0x7fbb141020f0 2024-04-23T18:41:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.126+0000 7fbb0b7fe640 1 -- 172.21.15.33:0/2686584100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04018b70 con 0x7fbb141020f0 2024-04-23T18:41:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.126+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb141a7790 con 0x7fbb141020f0 2024-04-23T18:41:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.128+0000 7fbb0b7fe640 1 -- 172.21.15.33:0/2686584100 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb04018d10 con 0x7fbb141020f0 2024-04-23T18:41:05.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.128+0000 7fbb1bb96740 1 Processor -- start 2024-04-23T18:41:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 -- start start 2024-04-23T18:41:05.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:05.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d34bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:06.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7c8c970 con 0x55e3f7d27f40 2024-04-23T18:41:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7c294a0 con 0x55e3f7d32800 2024-04-23T18:41:06.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7c88930 con 0x55e3f7d27620 2024-04-23T18:41:06.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51686/0 (socket says 172.21.15.33:51686) 2024-04-23T18:41:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 -- 172.21.15.33:0/2275154992 learned_addr learned my addr 172.21.15.33:0/2275154992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb0b7fe640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 0x7fbb000a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d34bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb12d9d640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 0x7fbb000a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 -- 172.21.15.33:0/2275154992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 msgr2=0x55e3f7d27a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d27a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 -- 172.21.15.33:0/2275154992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 msgr2=0x55e3f7d34bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d34bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb1259c640 1 -- 172.21.15.33:0/2275154992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f799f210 con 0x55e3f7d27f40 2024-04-23T18:41:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.129+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d27a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb0b7fe640 1 -- 172.21.15.33:0/2686584100 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbb040d0be0 con 0x7fbb141020f0 2024-04-23T18:41:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009230 tx=0x7fbb0c004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cf63aed9f7c0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb08ff9640 1 -- 172.21.15.33:0/2275154992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c035040 con 0x55e3f7d27f40 2024-04-23T18:41:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb12d9d640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 0x7fbb000a62b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409c870 tx=0x7fbb04018610 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:07.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb08ff9640 1 -- 172.21.15.33:0/2275154992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c004770 con 0x55e3f7d27f40 2024-04-23T18:41:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.130+0000 7fbb08ff9640 1 -- 172.21.15.33:0/2275154992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c003850 con 0x55e3f7d27f40 2024-04-23T18:41:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2275154992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 msgr2=0x55e3f7d322c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:07.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009230 tx=0x7fbb0c004310 comp rx=0 tx=0).stop 2024-04-23T18:41:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2275154992 shutdown_connections 2024-04-23T18:41:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d34bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d322c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2275154992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d27a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2275154992 >> 172.21.15.33:0/2275154992 conn(0x55e3f7d23b30 msgr2=0x55e3f7d23f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2275154992 shutdown_connections 2024-04-23T18:41:07.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2275154992 wait complete. 2024-04-23T18:41:07.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 -- start start 2024-04-23T18:41:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 Processor -- start 2024-04-23T18:41:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.131+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d32190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d2f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:07.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7d353d0 con 0x55e3f7d32800 2024-04-23T18:41:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7c270b0 con 0x55e3f7d27f40 2024-04-23T18:41:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e3f7d35550 con 0x55e3f7d27620 2024-04-23T18:41:07.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 10 client.0 _ll_register_callbacks cb 0x55e3f7c14d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51692/0 (socket says 172.21.15.33:51692) 2024-04-23T18:41:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 -- 172.21.15.33:0/2820944490 learned_addr learned my addr 172.21.15.33:0/2820944490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d32190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d2f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 -- 172.21.15.33:0/2820944490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 msgr2=0x55e3f7d32190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d32190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 -- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 msgr2=0x55e3f7d2f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d2f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:08.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 -- 172.21.15.33:0/2820944490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f7a6b620 con 0x55e3f7d32800 2024-04-23T18:41:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406a110 tx=0x7fbafc00c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca90c7a4a42b726 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:08.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00ccb0 con 0x55e3f7d32800 2024-04-23T18:41:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbafc004030 con 0x55e3f7d32800 2024-04-23T18:41:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c004770 con 0x55e3f7d32800 2024-04-23T18:41:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbafc026050 con 0x55e3f7d32800 2024-04-23T18:41:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.132+0000 7fbb1bb96740 10 client.17154 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3f7d412f0 con 0x55e3f7d32800 2024-04-23T18:41:08.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c004770 con 0x55e3f7d32800 2024-04-23T18:41:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbafc011440 con 0x55e3f7d32800 2024-04-23T18:41:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 10 client.17154 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 20 client.17154 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7d427b0 0x55e3f7d44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 4 client.17154 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:09.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3f7c9a390 con 0x55e3f7d427b0 2024-04-23T18:41:09.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.133+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7d427b0 0x55e3f7d44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.134+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7d427b0 0x55e3f7d44bc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7f50622cf317aa server_cookie=31fe3b4f8bd78a0f in_seq=0 out_seq=0 2024-04-23T18:41:09.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.134+0000 7fbaf9ffb640 10 client.17154 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.134+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/2820944490 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e3f7c9a390 con 0x55e3f7d427b0 2024-04-23T18:41:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.134+0000 7fbaf9ffb640 10 client.17154 handle_command_reply: tid=0 2024-04-23T18:41:09.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 client.17154 shutdown 2024-04-23T18:41:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 msgr2=0x55e3f7d2fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406a110 tx=0x7fbafc00c3e0 comp rx=0 tx=0).stop 2024-04-23T18:41:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 shutdown_connections 2024-04-23T18:41:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7d427b0 0x55e3f7d44bc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d32800 0x55e3f7d2fa00 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d2f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:09.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbaf9ffb640 0 client.17154 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2820944490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d32190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 >> 172.21.15.33:0/2820944490 conn(0x55e3f7d23b30 msgr2=0x55e3f7d29860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 shutdown_connections 2024-04-23T18:41:09.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2820944490 wait complete. 2024-04-23T18:41:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.135+0000 7fbb1bb96740 20 client.17154 trim_cache size 0 max 16384 2024-04-23T18:41:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 Processor -- start 2024-04-23T18:41:10.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 -- start start 2024-04-23T18:41:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:10.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141afdd0 con 0x55e3f7d27f40 2024-04-23T18:41:10.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b1910 con 0x55e3f7d32800 2024-04-23T18:41:10.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b04a0 con 0x55e3f7d27620 2024-04-23T18:41:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:10.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51694/0 (socket says 172.21.15.33:51694) 2024-04-23T18:41:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 -- 172.21.15.33:0/2086315665 learned_addr learned my addr 172.21.15.33:0/2086315665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 -- 172.21.15.33:0/2086315665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 msgr2=0x55e3f7d3e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d3e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:10.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 -- 172.21.15.33:0/2086315665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 msgr2=0x55e3f7d4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d4aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:11.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb1259c640 1 -- 172.21.15.33:0/2086315665 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3f7d34f60 con 0x55e3f7d27f40 2024-04-23T18:41:11.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.150+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:41:11.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1259c640 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02a9e0 tx=0x7fbb0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adba9feab7e0482 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:11.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbafbfff640 1 -- 172.21.15.33:0/2086315665 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c035040 con 0x55e3f7d27f40 2024-04-23T18:41:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbafbfff640 1 -- 172.21.15.33:0/2086315665 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c0506e0 con 0x55e3f7d27f40 2024-04-23T18:41:11.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbafbfff640 1 -- 172.21.15.33:0/2086315665 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c031970 con 0x55e3f7d27f40 2024-04-23T18:41:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2086315665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 msgr2=0x55e3f7d4a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c02a9e0 tx=0x7fbb0c009260 comp rx=0 tx=0).stop 2024-04-23T18:41:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2086315665 shutdown_connections 2024-04-23T18:41:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d32800 0x55e3f7d4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:11.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27f40 0x55e3f7d4a4d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/2086315665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d27620 0x55e3f7d3e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2086315665 >> 172.21.15.33:0/2086315665 conn(0x55e3f7d23b30 msgr2=0x55e3f7d22420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:11.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2086315665 shutdown_connections 2024-04-23T18:41:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.151+0000 7fbb1bb96740 1 -- 172.21.15.33:0/2086315665 wait complete. 2024-04-23T18:41:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 -- start start 2024-04-23T18:41:11.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 Processor -- start 2024-04-23T18:41:11.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d32800 0x55e3f7d38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:12.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b1240 con 0x55e3f7d27620 2024-04-23T18:41:12.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141b0b70 con 0x55e3f7d27f40 2024-04-23T18:41:12.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb141addd0 con 0x55e3f7d32800 2024-04-23T18:41:12.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1bb96740 10 client.0 _ll_register_callbacks cb 0x7fbafc012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51700/0 (socket says 172.21.15.33:51700) 2024-04-23T18:41:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 -- 172.21.15.33:0/1448253314 learned_addr learned my addr 172.21.15.33:0/1448253314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:12.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb1259c640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d32800 0x55e3f7d38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:12.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 -- 172.21.15.33:0/1448253314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d32800 msgr2=0x55e3f7d38fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d32800 0x55e3f7d38fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 -- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 msgr2=0x55e3f7d38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.152+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:12.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb11d9b640 1 -- 172.21.15.33:0/1448253314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3f7da8040 con 0x55e3f7d27620 2024-04-23T18:41:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb11d9b640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55e3f7d1e0f0 tx=0x7fbafc00c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df66cdaa2f09d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc011440 con 0x55e3f7d27620 2024-04-23T18:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbafc0115e0 con 0x55e3f7d27620 2024-04-23T18:41:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb0c0506e0 con 0x55e3f7d27620 2024-04-23T18:41:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbafc03e050 con 0x55e3f7d27620 2024-04-23T18:41:13.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 10 client.17160 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e3f7da9350 con 0x55e3f7d27620 2024-04-23T18:41:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb0c0506e0 con 0x55e3f7d27620 2024-04-23T18:41:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbafc037060 con 0x55e3f7d27620 2024-04-23T18:41:13.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 10 client.17160 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:13.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 20 client.17160 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7da99b0 0x55e3f7dabda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 4 client.17160 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.153+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3f7c94200 con 0x55e3f7da99b0 2024-04-23T18:41:13.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.154+0000 7fbb1259c640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7da99b0 0x55e3f7dabda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.154+0000 7fbaf9ffb640 10 client.17160 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.154+0000 7fbb1259c640 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7da99b0 0x55e3f7dabda0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64c9588e2ac2361 server_cookie=6cac46b111fab9e1 in_seq=0 out_seq=0 2024-04-23T18:41:13.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.158+0000 7fbaf9ffb640 1 -- 172.21.15.33:0/1448253314 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3f7c94200 con 0x55e3f7da99b0 2024-04-23T18:41:14.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.158+0000 7fbaf9ffb640 10 client.17160 handle_command_reply: tid=0 2024-04-23T18:41:14.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.158+0000 7fbb1bb96740 1 client.17160 shutdown 2024-04-23T18:41:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.158+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 msgr2=0x55e3f7d38520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:14.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.158+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x55e3f7d1e0f0 tx=0x7fbafc00c3e0 comp rx=0 tx=0).stop 2024-04-23T18:41:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 shutdown_connections 2024-04-23T18:41:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e3f7da99b0 0x55e3f7dabda0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:14.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3f7d32800 0x55e3f7d38fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e3f7d27f40 0x55e3f7d38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:14.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 --2- 172.21.15.33:0/1448253314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e3f7d27620 0x55e3f7d38520 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbaf9ffb640 0 client.17160 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 >> 172.21.15.33:0/1448253314 conn(0x55e3f7d23b30 msgr2=0x55e3f7d3db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 shutdown_connections 2024-04-23T18:41:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 1 -- 172.21.15.33:0/1448253314 wait complete. 2024-04-23T18:41:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb1bb96740 20 client.17160 trim_cache size 0 max 16384 2024-04-23T18:41:14.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:41:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:41:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:41:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:41:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 msgr2=0x7fbb000a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 0x7fbb000a62b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbb0409c870 tx=0x7fbb04018610 comp rx=0 tx=0).stop 2024-04-23T18:41:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 msgr2=0x7fbb141a68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400cce0 tx=0x7fbb0400a620 comp rx=0 tx=0).stop 2024-04-23T18:41:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 shutdown_connections 2024-04-23T18:41:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb000a3df0 0x7fbb000a62b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb141020f0 0x7fbb141a68a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb14107310 0x7fbb14101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 --2- 172.21.15.33:0/2686584100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb14100700 0x7fbb14101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 >> 172.21.15.33:0/2686584100 conn(0x7fbb14077370 msgr2=0x7fbb14105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 shutdown_connections 2024-04-23T18:41:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:35.159+0000 7fbb18da2640 1 -- 172.21.15.33:0/2686584100 wait complete. 2024-04-23T18:41:15.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:34 smithi033 ceph-mon[29274]: pgmap v414: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:15.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:36 smithi033 ceph-mon[29274]: pgmap v415: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:41:15.993 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:38 smithi033 ceph-mon[29274]: pgmap v416: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:16.043 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:40 smithi033 ceph-mon[29274]: pgmap v417: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:16.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:42 smithi033 ceph-mon[29274]: pgmap v418: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:41:16.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:44 smithi033 ceph-mon[29274]: pgmap v419: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:16.243 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:46 smithi033 ceph-mon[29274]: pgmap v420: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:41:16.328 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:04 smithi122 ceph-mon[32546]: pgmap v461: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:16.376 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:06 smithi122 ceph-mon[32546]: pgmap v462: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 86 KiB/s wr, 37 op/s 2024-04-23T18:41:16.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:09 smithi122 ceph-mon[32546]: pgmap v463: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-04-23T18:41:16.476 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:10 smithi122 ceph-mon[32546]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:41:16.558 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:11 smithi122 ceph-mon[32546]: pgmap v465: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-04-23T18:41:16.602 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:13 smithi122 ceph-mon[32546]: pgmap v466: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:41:16.677 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:15 smithi122 ceph-mon[32546]: pgmap v467: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:41:16.727 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:04 smithi106 ceph-mon[32716]: pgmap v461: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:41:16.791 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:06 smithi106 ceph-mon[32716]: pgmap v462: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 86 KiB/s wr, 37 op/s 2024-04-23T18:41:16.852 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:09 smithi106 ceph-mon[32716]: pgmap v463: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-04-23T18:41:16.902 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:10 smithi106 ceph-mon[32716]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:41:16.985 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:11 smithi106 ceph-mon[32716]: pgmap v465: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-04-23T18:41:17.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:13 smithi106 ceph-mon[32716]: pgmap v466: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:41:17.077 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:15 smithi106 ceph-mon[32716]: pgmap v467: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:41:17.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:16 smithi106 ceph-mon[32716]: pgmap v468: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:17.214 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:16 smithi106 ceph-mon[32716]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:41:17.255 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:16 smithi122 ceph-mon[32546]: pgmap v468: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:17.294 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:16 smithi122 ceph-mon[32546]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:41:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 -- 172.21.15.33:0/3926812669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 msgr2=0x7f6fa010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/3926812669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa010ee70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009ca0 tx=0x7f6f9802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:41:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 -- 172.21.15.33:0/3926812669 shutdown_connections 2024-04-23T18:41:17.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/3926812669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa010ee70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:17.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/3926812669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa0102b80 0x7f6fa0106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/3926812669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa01021d0 0x7f6fa01025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 -- 172.21.15.33:0/3926812669 >> 172.21.15.33:0/3926812669 conn(0x7f6fa00793e0 msgr2=0x7f6fa00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 -- 172.21.15.33:0/3926812669 shutdown_connections 2024-04-23T18:41:17.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.286+0000 7f6fa7fad640 1 -- 172.21.15.33:0/3926812669 wait complete. 2024-04-23T18:41:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 Processor -- start 2024-04-23T18:41:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 -- start start 2024-04-23T18:41:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa01021d0 0x7f6fa0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:17.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 0x7f6fa019d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:18.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01121f0 con 0x7f6fa01072a0 2024-04-23T18:41:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0112070 con 0x7f6fa019cef0 2024-04-23T18:41:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa7fad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa0112370 con 0x7f6fa01021d0 2024-04-23T18:41:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:18.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42290/0 (socket says 172.21.15.33:42290) 2024-04-23T18:41:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa5548640 1 -- 172.21.15.33:0/2654502333 learned_addr learned my addr 172.21.15.33:0/2654502333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa01021d0 0x7f6fa0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.287+0000 7f6fa654a640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 0x7f6fa019d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 -- 172.21.15.33:0/2654502333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa01021d0 msgr2=0x7f6fa0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:18.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa01021d0 0x7f6fa0198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 -- 172.21.15.33:0/2654502333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 msgr2=0x7f6fa019d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:18.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 0x7f6fa019d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:18.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 -- 172.21.15.33:0/2654502333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa019d9e0 con 0x7f6fa01072a0 2024-04-23T18:41:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa654a640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 0x7f6fa019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:41:18.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa5548640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000e9c0 tx=0x7f6f9000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6f96ffd640 1 -- 172.21.15.33:0/2654502333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000cd80 con 0x7f6fa01072a0 2024-04-23T18:41:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa019dcd0 con 0x7f6fa01072a0 2024-04-23T18:41:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6f96ffd640 1 -- 172.21.15.33:0/2654502333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f900054d0 con 0x7f6fa01072a0 2024-04-23T18:41:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6f96ffd640 1 -- 172.21.15.33:0/2654502333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90010430 con 0x7f6fa01072a0 2024-04-23T18:41:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.288+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01a5510 con 0x7f6fa01072a0 2024-04-23T18:41:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.291+0000 7f6faada4740 1 Processor -- start 2024-04-23T18:41:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.291+0000 7f6faada4740 1 -- start start 2024-04-23T18:41:19.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.291+0000 7f6f96ffd640 1 -- 172.21.15.33:0/2654502333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f90004fd0 con 0x7f6fa01072a0 2024-04-23T18:41:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e1aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:19.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:19.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e27bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557342d7f970 con 0x557342e25800 2024-04-23T18:41:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557342d1c4a0 con 0x557342e1af40 2024-04-23T18:41:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6faada4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557342d7b930 con 0x557342e1a620 2024-04-23T18:41:19.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e27bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e27bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42298/0 (socket says 172.21.15.33:42298) 2024-04-23T18:41:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:47412/0 (socket says 172.21.15.33:47412) 2024-04-23T18:41:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5548640 1 -- 172.21.15.33:0/3432916952 learned_addr learned my addr 172.21.15.33:0/3432916952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa654a640 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e1aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3432916952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 msgr2=0x557342e1aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e1aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3432916952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 msgr2=0x557342e27bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e27bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:19.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3432916952 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557342a92210 con 0x557342e1af40 2024-04-23T18:41:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6f96ffd640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 0x7f6f8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:20.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.292+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009c20 tx=0x7f6f9c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de677ddabf54e6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.293+0000 7f6fa654a640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 0x7f6f8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.293+0000 7f6f96ffd640 1 -- 172.21.15.33:0/2654502333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f6f900d0c30 con 0x7f6fa01072a0 2024-04-23T18:41:20.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.293+0000 7f6f8bfff640 1 -- 172.21.15.33:0/3432916952 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e070 con 0x557342e1af40 2024-04-23T18:41:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.293+0000 7f6f8bfff640 1 -- 172.21.15.33:0/3432916952 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c009dd0 con 0x557342e1af40 2024-04-23T18:41:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.293+0000 7f6f8bfff640 1 -- 172.21.15.33:0/3432916952 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c018710 con 0x557342e1af40 2024-04-23T18:41:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6fa654a640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 0x7f6f8c0a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005e30 tx=0x7f6f98005d80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- 172.21.15.33:0/3432916952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 msgr2=0x557342e252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009c20 tx=0x7f6f9c011810 comp rx=0 tx=0).stop 2024-04-23T18:41:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- 172.21.15.33:0/3432916952 shutdown_connections 2024-04-23T18:41:20.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e27bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e252c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 --2- 172.21.15.33:0/3432916952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e1aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- 172.21.15.33:0/3432916952 >> 172.21.15.33:0/3432916952 conn(0x557342e16b30 msgr2=0x557342e16f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:20.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- 172.21.15.33:0/3432916952 shutdown_connections 2024-04-23T18:41:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- 172.21.15.33:0/3432916952 wait complete. 2024-04-23T18:41:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 -- start start 2024-04-23T18:41:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.294+0000 7f6faada4740 1 Processor -- start 2024-04-23T18:41:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 0x557342e22440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e229a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557342e28430 con 0x557342e1af40 2024-04-23T18:41:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557342d1a0b0 con 0x557342e25800 2024-04-23T18:41:21.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557342e285b0 con 0x557342e1a620 2024-04-23T18:41:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6faada4740 10 client.0 _ll_register_callbacks cb 0x557342d07d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:21.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 0x557342e22440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa654a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:40124/0 (socket says 172.21.15.33:40124) 2024-04-23T18:41:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 -- 172.21.15.33:0/1207594761 learned_addr learned my addr 172.21.15.33:0/1207594761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:21.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa654a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e229a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:47422/0 (socket says 172.21.15.33:47422) 2024-04-23T18:41:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5548640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 0x557342e22440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42302/0 (socket says 172.21.15.33:42302) 2024-04-23T18:41:21.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 -- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 msgr2=0x557342e229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:22.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 -- 172.21.15.33:0/1207594761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 msgr2=0x557342e22440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:22.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 0x557342e22440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 -- 172.21.15.33:0/1207594761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557342b5e620 con 0x557342e1a620 2024-04-23T18:41:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c01c040 tx=0x7f6f9c009950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e77435395ef3c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:22.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.295+0000 7f6f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:41:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e040 con 0x557342e1a620 2024-04-23T18:41:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c009dd0 con 0x557342e1a620 2024-04-23T18:41:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f6f9c0051f0 con 0x557342e1a620 2024-04-23T18:41:22.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c032050 con 0x557342e1a620 2024-04-23T18:41:22.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 10 client.26584 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:22.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557342e343a0 con 0x557342e1a620 2024-04-23T18:41:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c009dd0 con 0x557342e1a620 2024-04-23T18:41:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6f9c021030 con 0x557342e1a620 2024-04-23T18:41:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 10 client.26584 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:22.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 20 client.26584 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e357b0 0x557342e37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 4 client.26584 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.296+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557342d8d390 con 0x557342e357b0 2024-04-23T18:41:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.297+0000 7f6fa5548640 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e357b0 0x557342e37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.297+0000 7f6fa5548640 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e357b0 0x557342e37ba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0ced4c172c25b6 server_cookie=64772f6607b6533 in_seq=0 out_seq=0 2024-04-23T18:41:23.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.297+0000 7f6f897fa640 10 client.26584 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6f897fa640 1 -- 172.21.15.33:0/1207594761 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557342d8d390 con 0x557342e357b0 2024-04-23T18:41:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6f897fa640 10 client.26584 handle_command_reply: tid=0 2024-04-23T18:41:23.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 client.26584 shutdown 2024-04-23T18:41:23.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 msgr2=0x557342e25150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c01c040 tx=0x7f6f9c009950 comp rx=0 tx=0).stop 2024-04-23T18:41:23.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 shutdown_connections 2024-04-23T18:41:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e357b0 0x557342e37ba0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6f897fa640 0 client.26584 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:23.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1af40 0x557342e22440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 --2- 172.21.15.33:0/1207594761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e25150 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 >> 172.21.15.33:0/1207594761 conn(0x557342e16b30 msgr2=0x557342e1c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 shutdown_connections 2024-04-23T18:41:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 1 -- 172.21.15.33:0/1207594761 wait complete. 2024-04-23T18:41:24.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.298+0000 7f6faada4740 20 client.26584 trim_cache size 0 max 16384 2024-04-23T18:41:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 Processor -- start 2024-04-23T18:41:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 -- start start 2024-04-23T18:41:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e21980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:24.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01adc40 con 0x557342e25800 2024-04-23T18:41:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01af780 con 0x557342e1af40 2024-04-23T18:41:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6faada4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01ae310 con 0x557342e1a620 2024-04-23T18:41:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.313+0000 7f6fa654a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa654a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42318/0 (socket says 172.21.15.33:42318) 2024-04-23T18:41:24.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa654a640 1 -- 172.21.15.33:0/3577688963 learned_addr learned my addr 172.21.15.33:0/3577688963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:24.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5548640 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e21980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:24.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3577688963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 msgr2=0x557342e21980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:24.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e21980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:24.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3577688963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 msgr2=0x557342e3d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 -- 172.21.15.33:0/3577688963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557342e16960 con 0x557342e1a620 2024-04-23T18:41:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa654a640 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:41:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e312f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009b40 tx=0x7f6f9c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=823c210aea4c3daa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:25.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3577688963 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e040 con 0x557342e1a620 2024-04-23T18:41:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3577688963 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c029070 con 0x557342e1a620 2024-04-23T18:41:25.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.314+0000 7f6f8b7fe640 1 -- 172.21.15.33:0/3577688963 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0048c0 con 0x557342e1a620 2024-04-23T18:41:25.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- 172.21.15.33:0/3577688963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 msgr2=0x557342e312f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:25.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e312f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009b40 tx=0x7f6f9c005f90 comp rx=0 tx=0).stop 2024-04-23T18:41:25.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- 172.21.15.33:0/3577688963 shutdown_connections 2024-04-23T18:41:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e25800 0x557342e3d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e1af40 0x557342e21980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 --2- 172.21.15.33:0/3577688963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1a620 0x557342e312f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- 172.21.15.33:0/3577688963 >> 172.21.15.33:0/3577688963 conn(0x557342e16b30 msgr2=0x557342e15cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- 172.21.15.33:0/3577688963 shutdown_connections 2024-04-23T18:41:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- 172.21.15.33:0/3577688963 wait complete. 2024-04-23T18:41:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.315+0000 7f6faada4740 1 -- start start 2024-04-23T18:41:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 Processor -- start 2024-04-23T18:41:25.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1af40 0x557342e2dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:26.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e2ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:26.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01af0b0 con 0x557342e1a620 2024-04-23T18:41:26.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01ae9e0 con 0x557342e25800 2024-04-23T18:41:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa01abc40 con 0x557342e1af40 2024-04-23T18:41:26.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6faada4740 10 client.0 _ll_register_callbacks cb 0x7f6fa006b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:26.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:26.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42330/0 (socket says 172.21.15.33:42330) 2024-04-23T18:41:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 -- 172.21.15.33:0/669997849 learned_addr learned my addr 172.21.15.33:0/669997849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:26.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5548640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1af40 0x557342e2dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa654a640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e2ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 -- 172.21.15.33:0/669997849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1af40 msgr2=0x557342e2dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1af40 0x557342e2dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:26.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 -- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 msgr2=0x557342e2ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e2ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.316+0000 7f6fa5d49640 1 -- 172.21.15.33:0/669997849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557342e2b580 con 0x557342e1a620 2024-04-23T18:41:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6fa5d49640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c003810 tx=0x7f6f9c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf7df900110c3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:41:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c01e040 con 0x557342e1a620 2024-04-23T18:41:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7f6f9c005270 con 0x557342e1a620 2024-04-23T18:41:27.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f9c029070 con 0x557342e1a620 2024-04-23T18:41:27.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6f9c046050 con 0x557342e1a620 2024-04-23T18:41:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 10 client.17178 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557342e9b7d0 con 0x557342e1a620 2024-04-23T18:41:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f9c029070 con 0x557342e1a620 2024-04-23T18:41:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f6f9c021070 con 0x557342e1a620 2024-04-23T18:41:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 10 client.17178 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 10 client.17178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:27.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 20 client.17178 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:27.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e9bee0 0x557342e9e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 4 client.17178 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:27.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.317+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557342d87200 con 0x557342e9bee0 2024-04-23T18:41:27.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.318+0000 7f6fa5548640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e9bee0 0x557342e9e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:28.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.318+0000 7f6f897fa640 10 client.17178 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.318+0000 7f6fa5548640 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e9bee0 0x557342e9e300 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec463cf63d83459a server_cookie=b6c229c6af83b12e in_seq=0 out_seq=0 2024-04-23T18:41:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6f897fa640 1 -- 172.21.15.33:0/669997849 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557342d87200 con 0x557342e9bee0 2024-04-23T18:41:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6f897fa640 10 client.17178 handle_command_reply: tid=0 2024-04-23T18:41:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 client.17178 shutdown 2024-04-23T18:41:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 msgr2=0x557342e2d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:28.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c003810 tx=0x7f6f9c0043c0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 shutdown_connections 2024-04-23T18:41:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557342e9bee0 0x557342e9e300 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557342e25800 0x557342e2ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557342e1af40 0x557342e2dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6f897fa640 0 client.17178 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 --2- 172.21.15.33:0/669997849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557342e1a620 0x557342e2d7f0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.322+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 >> 172.21.15.33:0/669997849 conn(0x557342e16b30 msgr2=0x557342e302e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 shutdown_connections 2024-04-23T18:41:29.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6faada4740 1 -- 172.21.15.33:0/669997849 wait complete. 2024-04-23T18:41:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6faada4740 20 client.17178 trim_cache size 0 max 16384 2024-04-23T18:41:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:41:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:41:29.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:41:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:41:29.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 msgr2=0x7f6f8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:29.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 0x7f6f8c0a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f98005e30 tx=0x7f6f98005d80 comp rx=0 tx=0).stop 2024-04-23T18:41:29.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 msgr2=0x7f6fa0198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000e9c0 tx=0x7f6f9000ee80 comp rx=0 tx=0).stop 2024-04-23T18:41:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 shutdown_connections 2024-04-23T18:41:29.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6f8c0a3dc0 0x7f6f8c0a6280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fa019cef0 0x7f6fa019d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa01072a0 0x7f6fa0198b70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:30.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 --2- 172.21.15.33:0/2654502333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa01021d0 0x7f6fa0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 >> 172.21.15.33:0/2654502333 conn(0x7f6fa00793e0 msgr2=0x7f6fa00ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:30.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 shutdown_connections 2024-04-23T18:41:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:39:48.323+0000 7f6fa7fad640 1 -- 172.21.15.33:0/2654502333 wait complete. 2024-04-23T18:41:30.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:48 smithi033 ceph-mon[29274]: pgmap v421: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:41:30.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:30.728 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:30.778 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:30.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:48 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:41:30.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:50 smithi033 ceph-mon[29274]: pgmap v422: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:41:30.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:52 smithi033 ceph-mon[29274]: pgmap v423: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:41:31.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:54 smithi033 ceph-mon[29274]: pgmap v424: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:41:31.095 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:56 smithi033 ceph-mon[29274]: pgmap v425: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-04-23T18:41:31.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:39:58 smithi033 ceph-mon[29274]: pgmap v426: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:41:31.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:00 smithi033 ceph-mon[29274]: pgmap v427: 97 pgs: 97 active+clean; 11 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 9 op/s 2024-04-23T18:41:31.279 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:00 smithi033 ceph-mon[29274]: overall HEALTH_OK 2024-04-23T18:41:31.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:18 smithi122 ceph-mon[32546]: pgmap v470: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:41:31.379 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:20 smithi122 ceph-mon[32546]: pgmap v471: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:31.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:22 smithi122 ceph-mon[32546]: pgmap v472: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:41:31.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:24 smithi122 ceph-mon[32546]: pgmap v473: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:41:31.604 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:26 smithi122 ceph-mon[32546]: pgmap v474: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:31.654 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:28 smithi122 ceph-mon[32546]: pgmap v475: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:31.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:30 smithi122 ceph-mon[32546]: pgmap v476: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:31.754 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:18 smithi106 ceph-mon[32716]: pgmap v470: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:41:31.804 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:20 smithi106 ceph-mon[32716]: pgmap v471: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:31.854 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:22 smithi106 ceph-mon[32716]: pgmap v472: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:41:31.887 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:24 smithi106 ceph-mon[32716]: pgmap v473: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:41:31.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:26 smithi106 ceph-mon[32716]: pgmap v474: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:41:31.997 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:28 smithi106 ceph-mon[32716]: pgmap v475: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:32.029 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:30 smithi106 ceph-mon[32716]: pgmap v476: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 -- 172.21.15.33:0/3545662580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb40107290 msgr2=0x7fbb4010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:32.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 --2- 172.21.15.33:0/3545662580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb40107290 0x7fbb4010ee60 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009ca0 tx=0x7fbb3802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 -- 172.21.15.33:0/3545662580 shutdown_connections 2024-04-23T18:41:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 --2- 172.21.15.33:0/3545662580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb40107290 0x7fbb4010ee60 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 --2- 172.21.15.33:0/3545662580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb40102b70 0x7fbb40106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 --2- 172.21.15.33:0/3545662580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb401021c0 0x7fbb401025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:32.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 -- 172.21.15.33:0/3545662580 >> 172.21.15.33:0/3545662580 conn(0x7fbb400fda20 msgr2=0x7fbb400ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:32.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 -- 172.21.15.33:0/3545662580 shutdown_connections 2024-04-23T18:41:32.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.423+0000 7fbb452fd640 1 -- 172.21.15.33:0/3545662580 wait complete. 2024-04-23T18:41:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 Processor -- start 2024-04-23T18:41:32.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 -- start start 2024-04-23T18:41:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401021c0 0x7fbb401a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb40107290 0x7fbb401a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:32.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:32.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40112230 con 0x7fbb4019a680 2024-04-23T18:41:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401120b0 con 0x7fbb40107290 2024-04-23T18:41:32.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.424+0000 7fbb452fd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401123b0 con 0x7fbb401021c0 2024-04-23T18:41:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:32.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51848/0 (socket says 172.21.15.33:51848) 2024-04-23T18:41:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 -- 172.21.15.33:0/1860870869 learned_addr learned my addr 172.21.15.33:0/1860870869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:33.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3effd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401021c0 0x7fbb401a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb37fff640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb40107290 0x7fbb401a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 -- 172.21.15.33:0/1860870869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401021c0 msgr2=0x7fbb401a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401021c0 0x7fbb401a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 -- 172.21.15.33:0/1860870869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb40107290 msgr2=0x7fbb401a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:33.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb40107290 0x7fbb401a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 -- 172.21.15.33:0/1860870869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019b280 con 0x7fbb4019a680 2024-04-23T18:41:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3f7fe640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009430 tx=0x7fbb380317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3cff9640 1 -- 172.21.15.33:0/1860870869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb38003b70 con 0x7fbb4019a680 2024-04-23T18:41:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3cff9640 1 -- 172.21.15.33:0/1860870869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb38003d10 con 0x7fbb4019a680 2024-04-23T18:41:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb3cff9640 1 -- 172.21.15.33:0/1860870869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802fa30 con 0x7fbb4019a680 2024-04-23T18:41:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019b510 con 0x7fbb4019a680 2024-04-23T18:41:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.425+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a76b0 con 0x7fbb4019a680 2024-04-23T18:41:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb3cff9640 1 -- 172.21.15.33:0/1860870869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb3802fbd0 con 0x7fbb4019a680 2024-04-23T18:41:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 Processor -- start 2024-04-23T18:41:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 -- start start 2024-04-23T18:41:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:33.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e14326970 con 0x558e143cc800 2024-04-23T18:41:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558e142c34a0 con 0x558e143c1f40 2024-04-23T18:41:34.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb4800f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e14322930 con 0x558e143c1620 2024-04-23T18:41:34.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.428+0000 7fbb3cff9640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb3f7fe640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:34.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:34.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51850/0 (socket says 172.21.15.33:51850) 2024-04-23T18:41:34.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 -- 172.21.15.33:0/3367624482 learned_addr learned my addr 172.21.15.33:0/3367624482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb3effd640 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb3cff9640 1 -- 172.21.15.33:0/1860870869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbb380f1b40 con 0x7fbb4019a680 2024-04-23T18:41:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 -- 172.21.15.33:0/3367624482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 msgr2=0x558e143c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:34.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 -- 172.21.15.33:0/3367624482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 msgr2=0x558e143cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 -- 172.21.15.33:0/3367624482 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e14039210 con 0x558e143cc800 2024-04-23T18:41:34.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb3f7fe640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005e30 tx=0x7fbb3808f040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb37fff640 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbb280126d0 tx=0x7fbb2800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357362277e95500b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb357fa640 1 -- 172.21.15.33:0/3367624482 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28005020 con 0x558e143cc800 2024-04-23T18:41:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb357fa640 1 -- 172.21.15.33:0/3367624482 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb280047a0 con 0x558e143cc800 2024-04-23T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.429+0000 7fbb357fa640 1 -- 172.21.15.33:0/3367624482 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800f9a0 con 0x558e143cc800 2024-04-23T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 -- 172.21.15.33:0/3367624482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 msgr2=0x558e143cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbb280126d0 tx=0x7fbb2800b390 comp rx=0 tx=0).stop 2024-04-23T18:41:34.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 -- 172.21.15.33:0/3367624482 shutdown_connections 2024-04-23T18:41:35.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143cebf0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 --2- 172.21.15.33:0/3367624482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 -- 172.21.15.33:0/3367624482 >> 172.21.15.33:0/3367624482 conn(0x558e143bdb30 msgr2=0x558e143bdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 -- 172.21.15.33:0/3367624482 shutdown_connections 2024-04-23T18:41:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.430+0000 7fbb4800f740 1 -- 172.21.15.33:0/3367624482 wait complete. 2024-04-23T18:41:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 -- start start 2024-04-23T18:41:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 Processor -- start 2024-04-23T18:41:35.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558e143cf430 con 0x558e143c1620 2024-04-23T18:41:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558e142c10b0 con 0x558e143cc800 2024-04-23T18:41:35.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e143cf5b0 con 0x558e143c1f40 2024-04-23T18:41:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb4800f740 10 client.0 _ll_register_callbacks cb 0x558e142aed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:36.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51864/0 (socket says 172.21.15.33:51864) 2024-04-23T18:41:36.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 -- 172.21.15.33:0/1425514078 learned_addr learned my addr 172.21.15.33:0/1425514078 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb3effd640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 -- 172.21.15.33:0/1425514078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 msgr2=0x558e143c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:36.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 -- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 msgr2=0x558e143c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:36.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:36.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.431+0000 7fbb37fff640 1 -- 172.21.15.33:0/1425514078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e14105620 con 0x558e143c1620 2024-04-23T18:41:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb37fff640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb28001eb0 tx=0x7fbb280058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6246e14aadec57 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:41:36.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28004240 con 0x558e143c1620 2024-04-23T18:41:36.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbb280043e0 con 0x558e143c1620 2024-04-23T18:41:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb280047a0 con 0x558e143c1620 2024-04-23T18:41:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb28004f70 con 0x558e143c1620 2024-04-23T18:41:36.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb4800f740 10 client.17190 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558e143db400 con 0x558e143c1620 2024-04-23T18:41:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb280047a0 con 0x558e143c1620 2024-04-23T18:41:37.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbb2800fce0 con 0x558e143c1620 2024-04-23T18:41:37.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb4800f740 10 client.17190 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.432+0000 7fbb4800f740 10 client.17190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:37.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb4800f740 20 client.17190 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:37.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e143dc800 0x558e143debf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb4800f740 4 client.17190 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:41:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e14334390 con 0x558e143dc800 2024-04-23T18:41:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb3effd640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e143dc800 0x558e143debf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb3effd640 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e143dc800 0x558e143debf0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa0ebb3ab24bf31 server_cookie=28deb0cdc4d8f11f in_seq=0 out_seq=0 2024-04-23T18:41:37.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.433+0000 7fbb267fc640 10 client.17190 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:37.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb267fc640 1 -- 172.21.15.33:0/1425514078 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e14334390 con 0x558e143dc800 2024-04-23T18:41:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb267fc640 10 client.17190 handle_command_reply: tid=0 2024-04-23T18:41:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 client.17190 shutdown 2024-04-23T18:41:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 msgr2=0x558e143cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:37.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbb28001eb0 tx=0x7fbb280058e0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 shutdown_connections 2024-04-23T18:41:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e143dc800 0x558e143debf0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 --2- 172.21.15.33:0/1425514078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143cc1c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb267fc640 0 client.17190 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:38.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 >> 172.21.15.33:0/1425514078 conn(0x558e143bdb30 msgr2=0x558e143c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 shutdown_connections 2024-04-23T18:41:38.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 1 -- 172.21.15.33:0/1425514078 wait complete. 2024-04-23T18:41:38.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.434+0000 7fbb4800f740 20 client.17190 trim_cache size 0 max 16384 2024-04-23T18:41:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.449+0000 7fbb4800f740 1 Processor -- start 2024-04-23T18:41:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.449+0000 7fbb4800f740 1 -- start start 2024-04-23T18:41:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143d82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143c8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:38.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401afcf0 con 0x558e143cc800 2024-04-23T18:41:38.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b1830 con 0x558e143c1f40 2024-04-23T18:41:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb4800f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b03c0 con 0x558e143c1620 2024-04-23T18:41:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51878/0 (socket says 172.21.15.33:51878) 2024-04-23T18:41:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 -- 172.21.15.33:0/2005598898 learned_addr learned my addr 172.21.15.33:0/2005598898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143d82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:38.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb3effd640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 -- 172.21.15.33:0/2005598898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 msgr2=0x558e143d82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143d82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 -- 172.21.15.33:0/2005598898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 msgr2=0x558e143c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:38.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 -- 172.21.15.33:0/2005598898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e143c7a20 con 0x558e143cc800 2024-04-23T18:41:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb3effd640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.450+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800ba80 tx=0x7fbb28012480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b2750759790fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb34ff9640 1 -- 172.21.15.33:0/2005598898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28036070 con 0x558e143cc800 2024-04-23T18:41:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb34ff9640 1 -- 172.21.15.33:0/2005598898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28024070 con 0x558e143cc800 2024-04-23T18:41:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb34ff9640 1 -- 172.21.15.33:0/2005598898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28005370 con 0x558e143cc800 2024-04-23T18:41:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 -- 172.21.15.33:0/2005598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 msgr2=0x558e143c36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:39.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800ba80 tx=0x7fbb28012480 comp rx=0 tx=0).stop 2024-04-23T18:41:39.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 -- 172.21.15.33:0/2005598898 shutdown_connections 2024-04-23T18:41:39.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143cc800 0x558e143c36b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143c1f40 0x558e143c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2005598898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1620 0x558e143d82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:39.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 -- 172.21.15.33:0/2005598898 >> 172.21.15.33:0/2005598898 conn(0x558e143bdb30 msgr2=0x558e143c2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 -- 172.21.15.33:0/2005598898 shutdown_connections 2024-04-23T18:41:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.451+0000 7fbb4800f740 1 -- 172.21.15.33:0/2005598898 wait complete. 2024-04-23T18:41:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 -- start start 2024-04-23T18:41:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 Processor -- start 2024-04-23T18:41:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:39.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143d4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b1160 con 0x558e143c1620 2024-04-23T18:41:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401b0a90 con 0x558e143cc800 2024-04-23T18:41:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401adcf0 con 0x558e143c1f40 2024-04-23T18:41:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb4800f740 10 client.0 _ll_register_callbacks cb 0x7fbb280125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51894/0 (socket says 172.21.15.33:51894) 2024-04-23T18:41:39.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 -- 172.21.15.33:0/2064247959 learned_addr learned my addr 172.21.15.33:0/2064247959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:40.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb3effd640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143d4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 -- 172.21.15.33:0/2064247959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 msgr2=0x558e143d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143d4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 -- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 msgr2=0x558e143c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.452+0000 7fbb37fff640 1 -- 172.21.15.33:0/2064247959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e143c6120 con 0x558e143c1620 2024-04-23T18:41:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb37fff640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbb28003830 tx=0x7fbb28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dd238419198dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:40.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:41:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28045060 con 0x558e143c1620 2024-04-23T18:41:40.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8542+0+0 (secure 0 0 0) 0x7fbb2800fab0 con 0x558e143c1620 2024-04-23T18:41:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb28024070 con 0x558e143c1620 2024-04-23T18:41:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb28036070 con 0x558e143c1620 2024-04-23T18:41:40.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 10 client.17196 fetch_fsmap learned FSMap version 36 2024-04-23T18:41:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558e14442510 con 0x558e143c1620 2024-04-23T18:41:40.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28024070 con 0x558e143c1620 2024-04-23T18:41:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7fbb28004290 con 0x558e143c1620 2024-04-23T18:41:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 10 client.17196 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:41:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:41:40.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 20 client.17196 populate_metadata read hostname 'smithi033' 2024-04-23T18:41:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e14442ac0 0x558e14444ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 4 client.17196 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:41:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.453+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e1432e200 con 0x558e14442ac0 2024-04-23T18:41:41.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.454+0000 7fbb3effd640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e14442ac0 0x558e14444ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:41.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.454+0000 7fbb3effd640 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e14442ac0 0x558e14444ee0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53507b81447853fe server_cookie=f60dc72f2cecb0c2 in_seq=0 out_seq=0 2024-04-23T18:41:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:01.454+0000 7fbb267fc640 10 client.17196 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:41.155 INFO:tasks.workunit.client.0.smithi033.stdout:437/600: open fd = 3 2024-04-23T18:41:41.181 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.215 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.297 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:41.349 INFO:tasks.workunit.client.0.smithi033.stdout:438/600: open fd = 3 2024-04-23T18:41:41.382 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.415 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.515 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:41.549 INFO:tasks.workunit.client.0.smithi033.stdout:439/600: open fd = 3 2024-04-23T18:41:41.582 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.615 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.666 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:41.682 INFO:tasks.workunit.client.0.smithi033.stdout:440/600: open fd = 3 2024-04-23T18:41:41.699 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.799 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.815 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:41.840 INFO:tasks.workunit.client.0.smithi033.stdout:441/600: open fd = 3 2024-04-23T18:41:41.941 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.957 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:41.974 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:41.990 INFO:tasks.workunit.client.0.smithi033.stdout:442/600: open fd = 3 2024-04-23T18:41:42.021 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.058 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.082 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.099 INFO:tasks.workunit.client.0.smithi033.stdout:443/600: open fd = 3 2024-04-23T18:41:42.173 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.191 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.207 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.224 INFO:tasks.workunit.client.0.smithi033.stdout:444/600: open fd = 3 2024-04-23T18:41:42.308 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.330 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.349 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.366 INFO:tasks.workunit.client.0.smithi033.stdout:445/600: open fd = 3 2024-04-23T18:41:42.413 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.433 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.449 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.520 INFO:tasks.workunit.client.0.smithi033.stdout:446/600: open fd = 3 2024-04-23T18:41:42.558 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.583 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.617 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.733 INFO:tasks.workunit.client.0.smithi033.stdout:447/600: open fd = 3 2024-04-23T18:41:42.766 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.833 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.867 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:42.916 INFO:tasks.workunit.client.0.smithi033.stdout:448/600: open fd = 3 2024-04-23T18:41:42.950 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:42.983 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.017 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.092 INFO:tasks.workunit.client.0.smithi033.stdout:449/600: open fd = 3 2024-04-23T18:41:43.125 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.159 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.250 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.283 INFO:tasks.workunit.client.0.smithi033.stdout:450/600: open fd = 3 2024-04-23T18:41:43.317 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.350 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.414 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.450 INFO:tasks.workunit.client.0.smithi033.stdout:451/600: open fd = 3 2024-04-23T18:41:43.484 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.575 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.600 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.634 INFO:tasks.workunit.client.0.smithi033.stdout:452/600: open fd = 3 2024-04-23T18:41:43.675 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.742 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.778 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.809 INFO:tasks.workunit.client.0.smithi033.stdout:453/600: open fd = 3 2024-04-23T18:41:43.842 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.904 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:43.942 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:43.976 INFO:tasks.workunit.client.0.smithi033.stdout:454/600: open fd = 3 2024-04-23T18:41:44.009 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.095 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.126 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:44.159 INFO:tasks.workunit.client.0.smithi033.stdout:455/600: open fd = 3 2024-04-23T18:41:44.284 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.317 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.351 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:44.429 INFO:tasks.workunit.client.0.smithi033.stdout:456/600: open fd = 3 2024-04-23T18:41:44.484 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.589 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.626 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:44.659 INFO:tasks.workunit.client.0.smithi033.stdout:457/600: open fd = 3 2024-04-23T18:41:44.787 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.818 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:44.851 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:44.985 INFO:tasks.workunit.client.0.smithi033.stdout:458/600: open fd = 3 2024-04-23T18:41:45.044 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.285 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.318 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:45.351 INFO:tasks.workunit.client.0.smithi033.stdout:459/600: open fd = 3 2024-04-23T18:41:45.402 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.435 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.468 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:45.501 INFO:tasks.workunit.client.0.smithi033.stdout:460/600: open fd = 3 2024-04-23T18:41:45.577 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.610 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.651 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:45.685 INFO:tasks.workunit.client.0.smithi033.stdout:461/600: open fd = 3 2024-04-23T18:41:45.723 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.792 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.827 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:45.870 INFO:tasks.workunit.client.0.smithi033.stdout:462/600: open fd = 3 2024-04-23T18:41:45.893 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.927 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:45.961 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.028 INFO:tasks.workunit.client.0.smithi033.stdout:463/600: open fd = 3 2024-04-23T18:41:46.144 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.177 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.211 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.260 INFO:tasks.workunit.client.0.smithi033.stdout:464/600: open fd = 3 2024-04-23T18:41:46.294 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.327 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.387 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.411 INFO:tasks.workunit.client.0.smithi033.stdout:465/600: open fd = 3 2024-04-23T18:41:46.470 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.525 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.553 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.661 INFO:tasks.workunit.client.0.smithi033.stdout:466/600: open fd = 3 2024-04-23T18:41:46.694 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.785 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.819 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.853 INFO:tasks.workunit.client.0.smithi033.stdout:467/600: open fd = 3 2024-04-23T18:41:46.878 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.903 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:46.965 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:46.994 INFO:tasks.workunit.client.0.smithi033.stdout:468/600: open fd = 3 2024-04-23T18:41:47.020 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.045 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.154 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.212 INFO:tasks.workunit.client.0.smithi033.stdout:469/600: open fd = 3 2024-04-23T18:41:47.228 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.258 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.278 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.303 INFO:tasks.workunit.client.0.smithi033.stdout:470/600: open fd = 3 2024-04-23T18:41:47.328 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.399 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.437 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.453 INFO:tasks.workunit.client.0.smithi033.stdout:471/600: open fd = 3 2024-04-23T18:41:47.470 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.567 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.595 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.612 INFO:tasks.workunit.client.0.smithi033.stdout:472/600: open fd = 3 2024-04-23T18:41:47.628 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.645 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.729 INFO:tasks.workunit.client.0.smithi033.stdout:473/600: open fd = 3 2024-04-23T18:41:47.811 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.829 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.845 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.862 INFO:tasks.workunit.client.0.smithi033.stdout:474/600: open fd = 3 2024-04-23T18:41:47.879 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.962 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:47.979 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:47.995 INFO:tasks.workunit.client.0.smithi033.stdout:475/600: open fd = 3 2024-04-23T18:41:48.012 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.029 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.096 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:48.145 INFO:tasks.workunit.client.0.smithi033.stdout:476/600: open fd = 3 2024-04-23T18:41:48.179 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.218 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.246 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:48.282 INFO:tasks.workunit.client.0.smithi033.stdout:477/600: open fd = 3 2024-04-23T18:41:48.371 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.404 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.467 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:48.504 INFO:tasks.workunit.client.0.smithi033.stdout:478/600: open fd = 3 2024-04-23T18:41:48.538 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.571 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.619 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:48.698 INFO:tasks.workunit.client.0.smithi033.stdout:479/600: open fd = 3 2024-04-23T18:41:48.721 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.755 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.860 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:48.897 INFO:tasks.workunit.client.0.smithi033.stdout:480/600: open fd = 3 2024-04-23T18:41:48.930 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:48.963 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.001 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:49.030 INFO:tasks.workunit.client.0.smithi033.stdout:481/600: open fd = 3 2024-04-23T18:41:49.063 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.141 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.180 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:49.284 INFO:tasks.workunit.client.0.smithi033.stdout:482/600: open fd = 3 2024-04-23T18:41:49.313 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.347 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.422 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:49.455 INFO:tasks.workunit.client.0.smithi033.stdout:483/600: open fd = 3 2024-04-23T18:41:49.563 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.606 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.647 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:49.680 INFO:tasks.workunit.client.0.smithi033.stdout:484/600: open fd = 3 2024-04-23T18:41:49.720 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.747 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.824 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:49.864 INFO:tasks.workunit.client.0.smithi033.stdout:485/600: open fd = 3 2024-04-23T18:41:49.902 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.939 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:49.972 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:50.005 INFO:tasks.workunit.client.0.smithi033.stdout:486/600: open fd = 3 2024-04-23T18:41:50.039 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:50.073 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:50.198 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:51.047 INFO:tasks.workunit.client.0.smithi033.stdout:487/600: open fd = 3 2024-04-23T18:41:51.257 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.316 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.389 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:51.423 INFO:tasks.workunit.client.0.smithi033.stdout:488/600: open fd = 3 2024-04-23T18:41:51.456 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.489 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.577 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:51.606 INFO:tasks.workunit.client.0.smithi033.stdout:489/600: open fd = 3 2024-04-23T18:41:51.640 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.673 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.706 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:51.756 INFO:tasks.workunit.client.0.smithi033.stdout:490/600: open fd = 3 2024-04-23T18:41:51.790 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.828 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.856 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:51.890 INFO:tasks.workunit.client.0.smithi033.stdout:491/600: open fd = 3 2024-04-23T18:41:51.923 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:51.972 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.048 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.082 INFO:tasks.workunit.client.0.smithi033.stdout:492/600: open fd = 3 2024-04-23T18:41:52.143 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.174 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.207 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.240 INFO:tasks.workunit.client.0.smithi033.stdout:493/600: open fd = 3 2024-04-23T18:41:52.286 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.323 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.377 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.407 INFO:tasks.workunit.client.0.smithi033.stdout:494/600: open fd = 3 2024-04-23T18:41:52.496 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.526 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.557 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.590 INFO:tasks.workunit.client.0.smithi033.stdout:495/600: open fd = 3 2024-04-23T18:41:52.624 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.657 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.690 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.735 INFO:tasks.workunit.client.0.smithi033.stdout:496/600: open fd = 3 2024-04-23T18:41:52.802 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.824 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:52.857 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:52.901 INFO:tasks.workunit.client.0.smithi033.stdout:497/600: open fd = 3 2024-04-23T18:41:53.024 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:53.049 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:53.082 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:53.099 INFO:tasks.workunit.client.0.smithi033.stdout:498/600: open fd = 3 2024-04-23T18:41:53.116 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:53.132 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:41:53.177 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:41:53.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb267fc640 1 -- 172.21.15.33:0/2064247959 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e1432e200 con 0x558e14442ac0 2024-04-23T18:41:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb267fc640 10 client.17196 handle_command_reply: tid=0 2024-04-23T18:41:53.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 client.17196 shutdown 2024-04-23T18:41:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 msgr2=0x558e143d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbb28003830 tx=0x7fbb28000bc0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 shutdown_connections 2024-04-23T18:41:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558e14442ac0 0x558e14444ee0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558e143cc800 0x558e143c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e143c1f40 0x558e143d4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 --2- 172.21.15.33:0/2064247959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e143c1620 0x558e143d4800 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb267fc640 0 client.17196 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:41:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 >> 172.21.15.33:0/2064247959 conn(0x558e143bdb30 msgr2=0x558e143c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 shutdown_connections 2024-04-23T18:41:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 1 -- 172.21.15.33:0/2064247959 wait complete. 2024-04-23T18:41:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.180+0000 7fbb4800f740 20 client.17196 trim_cache size 0 max 16384 2024-04-23T18:41:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:41:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:41:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:41:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:41:53.652 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:32 smithi122 ceph-mon[32546]: pgmap v477: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:41:53.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:34 smithi122 ceph-mon[32546]: pgmap v478: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:53.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:36 smithi122 ceph-mon[32546]: pgmap v479: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T18:41:53.867 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:38 smithi122 ceph-mon[32546]: pgmap v480: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:53.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:40 smithi122 ceph-mon[32546]: pgmap v481: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:53.925 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:42 smithi122 ceph-mon[32546]: pgmap v482: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:41:53.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:44 smithi122 ceph-mon[32546]: pgmap v483: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:53.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:46 smithi122 ceph-mon[32546]: pgmap v484: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-23T18:41:54.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:48 smithi122 ceph-mon[32546]: pgmap v485: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:41:54.041 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:49 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:54.075 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:50 smithi122 ceph-mon[32546]: pgmap v486: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:41:54.108 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:50 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:54.142 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:50 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:54.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:50 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:41:54.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:52 smithi122 ceph-mon[32546]: pgmap v487: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:41:54.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:32 smithi106 ceph-mon[32716]: pgmap v477: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:41:54.367 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:34 smithi106 ceph-mon[32716]: pgmap v478: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:54.415 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:36 smithi106 ceph-mon[32716]: pgmap v479: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T18:41:54.450 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:38 smithi106 ceph-mon[32716]: pgmap v480: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:54.483 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:40 smithi106 ceph-mon[32716]: pgmap v481: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:41:54.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:42 smithi106 ceph-mon[32716]: pgmap v482: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:41:54.550 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:44 smithi106 ceph-mon[32716]: pgmap v483: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:41:54.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:46 smithi106 ceph-mon[32716]: pgmap v484: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-23T18:41:54.617 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:48 smithi106 ceph-mon[32716]: pgmap v485: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:41:54.698 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:49 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:41:54.784 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:50 smithi106 ceph-mon[32716]: pgmap v486: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:41:54.817 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:50 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:41:54.850 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:50 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:41:54.884 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:50 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:41:54.917 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:52 smithi106 ceph-mon[32716]: pgmap v487: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:41:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 msgr2=0x7fbb2c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005e30 tx=0x7fbb3808f040 comp rx=0 tx=0).stop 2024-04-23T18:41:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 msgr2=0x7fbb4019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009430 tx=0x7fbb380317c0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 shutdown_connections 2024-04-23T18:41:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbb2c0a3dc0 0x7fbb2c0a6280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb4019a680 0x7fbb4019ab00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbb40107290 0x7fbb401a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 --2- 172.21.15.33:0/1860870869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb401021c0 0x7fbb401a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 >> 172.21.15.33:0/1860870869 conn(0x7fbb400fda20 msgr2=0x7fbb4010eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:55.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 shutdown_connections 2024-04-23T18:41:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:03.181+0000 7fbb452fd640 1 -- 172.21.15.33:0/1860870869 wait complete. 2024-04-23T18:41:55.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:02 smithi033 ceph-mon[29274]: pgmap v428: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 27 op/s 2024-04-23T18:41:55.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:04 smithi033 ceph-mon[29274]: pgmap v429: 97 pgs: 97 active+clean; 11 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 41 KiB/s wr, 22 op/s 2024-04-23T18:41:55.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:04 smithi033 ceph-mon[29274]: osdmap e108: 12 total, 12 up, 12 in 2024-04-23T18:41:55.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:06 smithi033 ceph-mon[29274]: pgmap v431: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:41:55.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:08 smithi033 ceph-mon[29274]: pgmap v432: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:41:55.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:10 smithi033 ceph-mon[29274]: pgmap v433: 97 pgs: 97 active+clean; 11 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 90 KiB/s wr, 38 op/s 2024-04-23T18:41:55.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:10 smithi033 ceph-mon[29274]: osdmap e109: 12 total, 12 up, 12 in 2024-04-23T18:41:55.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:12 smithi033 ceph-mon[29274]: pgmap v435: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 79 KiB/s wr, 30 op/s 2024-04-23T18:41:55.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:14 smithi033 ceph-mon[29274]: pgmap v436: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-23T18:41:55.811 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:54 smithi106 ceph-mon[32716]: pgmap v488: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T18:41:55.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:54 smithi122 ceph-mon[32546]: pgmap v488: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T18:41:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 -- 172.21.15.33:0/1941406355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c102b70 msgr2=0x7f0b8c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 --2- 172.21.15.33:0/1941406355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c102b70 0x7f0b8c106bb0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009900 tx=0x7f0b8002a410 comp rx=0 tx=0).stop 2024-04-23T18:41:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 -- 172.21.15.33:0/1941406355 shutdown_connections 2024-04-23T18:41:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 --2- 172.21.15.33:0/1941406355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 --2- 172.21.15.33:0/1941406355 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c102b70 0x7f0b8c106bb0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 --2- 172.21.15.33:0/1941406355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 0x7f0b8c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 -- 172.21.15.33:0/1941406355 >> 172.21.15.33:0/1941406355 conn(0x7f0b8c0fda20 msgr2=0x7f0b8c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 -- 172.21.15.33:0/1941406355 shutdown_connections 2024-04-23T18:41:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.284+0000 7f0b908ef640 1 -- 172.21.15.33:0/1941406355 wait complete. 2024-04-23T18:41:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 Processor -- start 2024-04-23T18:41:56.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 -- start start 2024-04-23T18:41:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 0x7f0b8c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:56.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112200 con 0x7f0b8c074cd0 2024-04-23T18:41:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112080 con 0x7f0b8c107290 2024-04-23T18:41:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.285+0000 7f0b908ef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c112380 con 0x7f0b8c1021c0 2024-04-23T18:41:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44824/0 (socket says 172.21.15.33:44824) 2024-04-23T18:41:56.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 -- 172.21.15.33:0/2248802408 learned_addr learned my addr 172.21.15.33:0/2248802408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 0x7f0b8c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:56.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 -- 172.21.15.33:0/2248802408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 msgr2=0x7f0b8c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 0x7f0b8c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:57.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 -- 172.21.15.33:0/2248802408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 msgr2=0x7f0b8c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:57.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 -- 172.21.15.33:0/2248802408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c1a6eb0 con 0x7f0b8c074cd0 2024-04-23T18:41:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:41:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b8ad9d640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c970 tx=0x7f0b8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b7b7fe640 1 -- 172.21.15.33:0/2248802408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84007bb0 con 0x7f0b8c074cd0 2024-04-23T18:41:57.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c1a71a0 con 0x7f0b8c074cd0 2024-04-23T18:41:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b7b7fe640 1 -- 172.21.15.33:0/2248802408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b84007d50 con 0x7f0b8c074cd0 2024-04-23T18:41:57.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b7b7fe640 1 -- 172.21.15.33:0/2248802408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b84018770 con 0x7f0b8c074cd0 2024-04-23T18:41:57.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.286+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c1a76f0 con 0x7f0b8c074cd0 2024-04-23T18:41:57.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b7b7fe640 1 -- 172.21.15.33:0/2248802408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b8400f990 con 0x7f0b8c074cd0 2024-04-23T18:41:57.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 Processor -- start 2024-04-23T18:41:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 -- start start 2024-04-23T18:41:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1ca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:57.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560aef119970 con 0x560aef1b4e80 2024-04-23T18:41:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560aef0b64a0 con 0x560aef1c8010 2024-04-23T18:41:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b93608740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560aef115930 con 0x560aef1b4560 2024-04-23T18:41:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.289+0000 7f0b7b7fe640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 0x7f0b740a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b89d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:57.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44840/0 (socket says 172.21.15.33:44840) 2024-04-23T18:41:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3839520400 learned_addr learned my addr 172.21.15.33:0/3839520400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8ad9d640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 0x7f0b740a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:58.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b7b7fe640 1 -- 172.21.15.33:0/2248802408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f0b840d07c0 con 0x7f0b8c074cd0 2024-04-23T18:41:58.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3839520400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 msgr2=0x560aef1b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:58.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3839520400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 msgr2=0x560aef1ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:58.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3839520400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aeee2c210 con 0x560aef1b4e80 2024-04-23T18:41:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8ad9d640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 0x7f0b740a6330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b840cdfb0 tx=0x7f0b84087040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c007f90 tx=0x7f0b7c00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d501c4e77d158c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:41:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b78ff9640 1 -- 172.21.15.33:0/3839520400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004d20 con 0x560aef1b4e80 2024-04-23T18:41:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b78ff9640 1 -- 172.21.15.33:0/3839520400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c010af0 con 0x560aef1b4e80 2024-04-23T18:41:58.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.290+0000 7f0b78ff9640 1 -- 172.21.15.33:0/3839520400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c005620 con 0x560aef1b4e80 2024-04-23T18:41:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 -- 172.21.15.33:0/3839520400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 msgr2=0x560aef1c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:41:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c007f90 tx=0x7f0b7c00f850 comp rx=0 tx=0).stop 2024-04-23T18:41:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 -- 172.21.15.33:0/3839520400 shutdown_connections 2024-04-23T18:41:58.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c7ad0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 --2- 172.21.15.33:0/3839520400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:58.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 -- 172.21.15.33:0/3839520400 >> 172.21.15.33:0/3839520400 conn(0x560aef1b0b30 msgr2=0x560aef1b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:41:58.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 -- 172.21.15.33:0/3839520400 shutdown_connections 2024-04-23T18:41:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.291+0000 7f0b93608740 1 -- 172.21.15.33:0/3839520400 wait complete. 2024-04-23T18:41:58.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 -- start start 2024-04-23T18:41:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 Processor -- start 2024-04-23T18:41:58.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1c7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:58.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:59.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:41:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560aef1cabe0 con 0x560aef1b4e80 2024-04-23T18:41:59.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560aef0b40b0 con 0x560aef1c8010 2024-04-23T18:41:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560aef1cad60 con 0x560aef1b4560 2024-04-23T18:41:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b93608740 10 client.0 _ll_register_callbacks cb 0x560aef0a1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:41:59.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44852/0 (socket says 172.21.15.33:44852) 2024-04-23T18:41:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 -- 172.21.15.33:0/4067294451 learned_addr learned my addr 172.21.15.33:0/4067294451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:41:59.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:41:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 -- 172.21.15.33:0/4067294451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 msgr2=0x560aef1c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 -- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 msgr2=0x560aef1c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:41:59.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:41:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.292+0000 7f0b8a59c640 1 -- 172.21.15.33:0/4067294451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aeeef8620 con 0x560aef1b4e80 2024-04-23T18:41:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c005c40 tx=0x7f0b7c0108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff94ad5575b0b383 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c026020 con 0x560aef1b4e80 2024-04-23T18:42:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f0b7c005620 con 0x560aef1b4e80 2024-04-23T18:42:00.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c010af0 con 0x560aef1b4e80 2024-04-23T18:42:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c031050 con 0x560aef1b4e80 2024-04-23T18:42:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 10 client.17208 fetch_fsmap learned FSMap version 36 2024-04-23T18:42:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560aef1d6b70 con 0x560aef1b4e80 2024-04-23T18:42:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c010af0 con 0x560aef1b4e80 2024-04-23T18:42:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f0b7c022640 con 0x560aef1b4e80 2024-04-23T18:42:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 10 client.17208 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:42:00.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 10 client.17208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:00.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.293+0000 7f0b93608740 20 client.17208 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:00.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef1d7f70 0x560aef1da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b93608740 4 client.17208 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aef127390 con 0x560aef1d7f70 2024-04-23T18:42:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef1d7f70 0x560aef1da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef1d7f70 0x560aef1da360 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1726dd15965f23 server_cookie=44a1efc4b482294c in_seq=0 out_seq=0 2024-04-23T18:42:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.294+0000 7f0b71ffb640 10 client.17208 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b71ffb640 1 -- 172.21.15.33:0/4067294451 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560aef127390 con 0x560aef1d7f70 2024-04-23T18:42:01.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b71ffb640 10 client.17208 handle_command_reply: tid=0 2024-04-23T18:42:01.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 client.17208 shutdown 2024-04-23T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 msgr2=0x560aef1c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:01.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c005c40 tx=0x7f0b7c0108f0 comp rx=0 tx=0).stop 2024-04-23T18:42:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 shutdown_connections 2024-04-23T18:42:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef1d7f70 0x560aef1da360 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:01.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1c4c30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:01.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 --2- 172.21.15.33:0/4067294451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4560 0x560aef1c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b71ffb640 0 client.17208 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:01.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 >> 172.21.15.33:0/4067294451 conn(0x560aef1b0b30 msgr2=0x560aef1b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:01.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 shutdown_connections 2024-04-23T18:42:01.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 1 -- 172.21.15.33:0/4067294451 wait complete. 2024-04-23T18:42:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.295+0000 7f0b93608740 20 client.17208 trim_cache size 0 max 16384 2024-04-23T18:42:01.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.310+0000 7f0b93608740 1 Processor -- start 2024-04-23T18:42:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.310+0000 7f0b93608740 1 -- start start 2024-04-23T18:42:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.310+0000 7f0b93608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1b4560 0x560aef1d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b93608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b93608740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1c8010 0x560aef1c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b93608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1ab950 con 0x560aef1b4e80 2024-04-23T18:42:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b93608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1ad490 con 0x560aef1b4560 2024-04-23T18:42:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b93608740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1ac020 con 0x560aef1c8010 2024-04-23T18:42:02.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44860/0 (socket says 172.21.15.33:44860) 2024-04-23T18:42:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3861749503 learned_addr learned my addr 172.21.15.33:0/3861749503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1c8010 0x560aef1c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1b4560 0x560aef1d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3861749503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1c8010 msgr2=0x560aef1c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:02.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1c8010 0x560aef1c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:02.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3861749503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1b4560 msgr2=0x560aef1d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:02.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1b4560 0x560aef1d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 -- 172.21.15.33:0/3861749503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aef1c3c00 con 0x560aef1b4e80 2024-04-23T18:42:03.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c007eb0 tx=0x7f0b7c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1846baf7365670b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b73fff640 1 -- 172.21.15.33:0/3861749503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c026040 con 0x560aef1b4e80 2024-04-23T18:42:03.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b73fff640 1 -- 172.21.15.33:0/3861749503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c0049e0 con 0x560aef1b4e80 2024-04-23T18:42:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.311+0000 7f0b73fff640 1 -- 172.21.15.33:0/3861749503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c01d040 con 0x560aef1b4e80 2024-04-23T18:42:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- 172.21.15.33:0/3861749503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 msgr2=0x560aef1b0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c007eb0 tx=0x7f0b7c007720 comp rx=0 tx=0).stop 2024-04-23T18:42:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- 172.21.15.33:0/3861749503 shutdown_connections 2024-04-23T18:42:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1c8010 0x560aef1c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:03.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4e80 0x560aef1b0f10 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 --2- 172.21.15.33:0/3861749503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1b4560 0x560aef1d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- 172.21.15.33:0/3861749503 >> 172.21.15.33:0/3861749503 conn(0x560aef1b0b30 msgr2=0x560aef1c0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- 172.21.15.33:0/3861749503 shutdown_connections 2024-04-23T18:42:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- 172.21.15.33:0/3861749503 wait complete. 2024-04-23T18:42:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 -- start start 2024-04-23T18:42:03.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.312+0000 7f0b93608740 1 Processor -- start 2024-04-23T18:42:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:03.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 0x560aef1c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:03.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:03.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1acdc0 con 0x560aef1b4560 2024-04-23T18:42:03.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1ac6f0 con 0x560aef1c8010 2024-04-23T18:42:03.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c1a9950 con 0x560aef1b4e80 2024-04-23T18:42:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 10 client.0 _ll_register_callbacks cb 0x7f0b7c007de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44874/0 (socket says 172.21.15.33:44874) 2024-04-23T18:42:04.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 -- 172.21.15.33:0/3072378843 learned_addr learned my addr 172.21.15.33:0/3072378843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:04.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 0x560aef1c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 -- 172.21.15.33:0/3072378843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 msgr2=0x560aef1c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 0x560aef1c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 -- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 msgr2=0x560aef1c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:04.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:04.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 -- 172.21.15.33:0/3072378843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aef1c23f0 con 0x560aef1b4560 2024-04-23T18:42:04.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 0x560aef1c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:04.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b89d9b640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c06b5f0 tx=0x7f0b7c007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd1615c7959e00b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:04.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:04.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80035040 con 0x560aef1b4560 2024-04-23T18:42:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f0b800497e0 con 0x560aef1b4560 2024-04-23T18:42:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.313+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b7c0049e0 con 0x560aef1b4560 2024-04-23T18:42:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0b800315e0 con 0x560aef1b4560 2024-04-23T18:42:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 10 client.17214 fetch_fsmap learned FSMap version 36 2024-04-23T18:42:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560aef23e310 con 0x560aef1b4560 2024-04-23T18:42:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b7c0049e0 con 0x560aef1b4560 2024-04-23T18:42:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f0b8003a040 con 0x560aef1b4560 2024-04-23T18:42:04.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 10 client.17214 fetch_fsmap finished waiting for FSMap version 36 2024-04-23T18:42:05.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 20 client.17214 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef23ea70 0x560aef240e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 4 client.17214 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560aef121200 con 0x560aef23ea70 2024-04-23T18:42:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.314+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef23ea70 0x560aef240e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.315+0000 7f0b71ffb640 10 client.17214 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.315+0000 7f0b8a59c640 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef23ea70 0x560aef240e90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ef2a5bf9d320ff server_cookie=c51f0d445ec6fc57 in_seq=0 out_seq=0 2024-04-23T18:42:05.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.322+0000 7f0b71ffb640 1 -- 172.21.15.33:0/3072378843 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560aef121200 con 0x560aef23ea70 2024-04-23T18:42:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.322+0000 7f0b71ffb640 10 client.17214 handle_command_reply: tid=0 2024-04-23T18:42:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.322+0000 7f0b93608740 1 client.17214 shutdown 2024-04-23T18:42:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 msgr2=0x560aef1d0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c06b5f0 tx=0x7f0b7c007c20 comp rx=0 tx=0).stop 2024-04-23T18:42:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 shutdown_connections 2024-04-23T18:42:05.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560aef23ea70 0x560aef240e90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560aef1c8010 0x560aef1c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:05.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aef1b4e80 0x560aef1c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b71ffb640 0 client.17214 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 --2- 172.21.15.33:0/3072378843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560aef1b4560 0x560aef1d0450 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 >> 172.21.15.33:0/3072378843 conn(0x560aef1b0b30 msgr2=0x560aef1c0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 shutdown_connections 2024-04-23T18:42:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 1 -- 172.21.15.33:0/3072378843 wait complete. 2024-04-23T18:42:06.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b93608740 20 client.17214 trim_cache size 0 max 16384 2024-04-23T18:42:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:42:06.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:42:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:42:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:42:06.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 msgr2=0x7f0b740a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:06.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 0x7f0b740a6330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0b840cdfb0 tx=0x7f0b84087040 comp rx=0 tx=0).stop 2024-04-23T18:42:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 msgr2=0x7f0b8c1a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c970 tx=0x7f0b8400ce30 comp rx=0 tx=0).stop 2024-04-23T18:42:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 shutdown_connections 2024-04-23T18:42:06.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b740a3e70 0x7f0b740a6330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b8c074cd0 0x7f0b8c1a67b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b8c107290 0x7f0b8c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:06.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 --2- 172.21.15.33:0/2248802408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b8c1021c0 0x7f0b8c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 >> 172.21.15.33:0/2248802408 conn(0x7f0b8c0fda20 msgr2=0x7f0b8c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 shutdown_connections 2024-04-23T18:42:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:16.323+0000 7f0b908ef640 1 -- 172.21.15.33:0/2248802408 wait complete. 2024-04-23T18:42:07.256 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:56 smithi106 ceph-mon[32716]: pgmap v489: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-23T18:42:07.304 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:41:58 smithi106 ceph-mon[32716]: pgmap v490: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:42:07.354 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:00 smithi106 ceph-mon[32716]: pgmap v491: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:42:07.404 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:02 smithi106 ceph-mon[32716]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-23T18:42:07.490 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:04 smithi106 ceph-mon[32716]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:42:07.555 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:04 smithi106 ceph-mon[32716]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:42:07.605 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:56 smithi122 ceph-mon[32546]: pgmap v489: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-23T18:42:07.694 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:41:58 smithi122 ceph-mon[32546]: pgmap v490: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:42:07.730 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:00 smithi122 ceph-mon[32546]: pgmap v491: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:42:07.780 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:02 smithi122 ceph-mon[32546]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-23T18:42:07.830 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:04 smithi122 ceph-mon[32546]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:42:07.880 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:04 smithi122 ceph-mon[32546]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:42:07.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:06 smithi122 ceph-mon[32546]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:07.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:16 smithi033 ceph-mon[29274]: pgmap v437: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:42:08.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:18 smithi033 ceph-mon[29274]: pgmap v438: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:42:08.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:20 smithi033 ceph-mon[29274]: pgmap v439: 97 pgs: 97 active+clean; 12 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-23T18:42:08.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:20 smithi033 ceph-mon[29274]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-04-23T18:42:08.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:20 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:42:08.301 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:22 smithi033 ceph-mon[29274]: pgmap v440: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 13 op/s 2024-04-23T18:42:08.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:24 smithi033 ceph-mon[29274]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-23T18:42:08.405 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:26 smithi033 ceph-mon[29274]: pgmap v442: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 19 op/s 2024-04-23T18:42:08.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:28 smithi033 ceph-mon[29274]: pgmap v443: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:42:08.573 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:06 smithi106 ceph-mon[32716]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 -- 172.21.15.33:0/3285039631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac102b70 msgr2=0x7f4eac106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 --2- 172.21.15.33:0/3285039631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac102b70 0x7f4eac106bb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a410 comp rx=0 tx=0).stop 2024-04-23T18:42:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 -- 172.21.15.33:0/3285039631 shutdown_connections 2024-04-23T18:42:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 --2- 172.21.15.33:0/3285039631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac107290 0x7f4eac10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:08.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 --2- 172.21.15.33:0/3285039631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac102b70 0x7f4eac106bb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:08.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 --2- 172.21.15.33:0/3285039631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 0x7f4eac1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 -- 172.21.15.33:0/3285039631 >> 172.21.15.33:0/3285039631 conn(0x7f4eac0fda20 msgr2=0x7f4eac0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 -- 172.21.15.33:0/3285039631 shutdown_connections 2024-04-23T18:42:09.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.423+0000 7f4eb1610640 1 -- 172.21.15.33:0/3285039631 wait complete. 2024-04-23T18:42:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 Processor -- start 2024-04-23T18:42:09.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 -- start start 2024-04-23T18:42:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 0x7f4eac198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:09.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac19cf30 0x7f4eac19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:09.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1121d0 con 0x7f4eac107290 2024-04-23T18:42:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112050 con 0x7f4eac1021c0 2024-04-23T18:42:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eb1610640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac112350 con 0x7f4eac19cf30 2024-04-23T18:42:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37858/0 (socket says 172.21.15.33:37858) 2024-04-23T18:42:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/1584723109 learned_addr learned my addr 172.21.15.33:0/1584723109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.424+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 0x7f4eac198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:09.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac19cf30 0x7f4eac19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/1584723109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac19cf30 msgr2=0x7f4eac19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac19cf30 0x7f4eac19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/1584723109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 msgr2=0x7f4eac198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:09.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 0x7f4eac198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:10.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/1584723109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac19da20 con 0x7f4eac107290 2024-04-23T18:42:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006010 tx=0x7f4ea0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4e93fff640 1 -- 172.21.15.33:0/1584723109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea003f030 con 0x7f4eac107290 2024-04-23T18:42:10.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4e93fff640 1 -- 172.21.15.33:0/1584723109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ea0003aa0 con 0x7f4eac107290 2024-04-23T18:42:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac19dcb0 con 0x7f4eac107290 2024-04-23T18:42:10.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4e93fff640 1 -- 172.21.15.33:0/1584723109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0035070 con 0x7f4eac107290 2024-04-23T18:42:10.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.425+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac1a5470 con 0x7f4eac107290 2024-04-23T18:42:10.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.427+0000 7f4e93fff640 1 -- 172.21.15.33:0/1584723109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ea0042030 con 0x7f4eac107290 2024-04-23T18:42:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 Processor -- start 2024-04-23T18:42:10.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 -- start start 2024-04-23T18:42:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ed6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c230e3b970 con 0x55c230ed6e40 2024-04-23T18:42:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c230dd84a0 con 0x55c230ed6520 2024-04-23T18:42:10.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eb4404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c230e37930 con 0x55c230ee5c10 2024-04-23T18:42:10.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4e93fff640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 0x7f4e940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:10.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37864/0 (socket says 172.21.15.33:37864) 2024-04-23T18:42:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eaaffd640 1 -- 172.21.15.33:0/1092265203 learned_addr learned my addr 172.21.15.33:0/1092265203 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 0x7f4e940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:11.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4e93fff640 1 -- 172.21.15.33:0/1584723109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f4ea00f0c30 con 0x7f4eac107290 2024-04-23T18:42:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.428+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ed6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 -- 172.21.15.33:0/1092265203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 msgr2=0x55c230ee8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:11.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 -- 172.21.15.33:0/1092265203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 msgr2=0x55c230ed6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ed6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 -- 172.21.15.33:0/1092265203 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c230b4e210 con 0x55c230ed6e40 2024-04-23T18:42:11.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ed6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009cf0 tx=0x7f4e9c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3cd102c88b5fac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 0x7f4e940a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005c40 tx=0x7f4ea40094f0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4e917fa640 1 -- 172.21.15.33:0/1092265203 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c004230 con 0x55c230ed6e40 2024-04-23T18:42:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4e917fa640 1 -- 172.21.15.33:0/1092265203 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c0043d0 con 0x55c230ed6e40 2024-04-23T18:42:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.429+0000 7f4e917fa640 1 -- 172.21.15.33:0/1092265203 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c019880 con 0x55c230ed6e40 2024-04-23T18:42:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- 172.21.15.33:0/1092265203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 msgr2=0x55c230ee56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:11.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009cf0 tx=0x7f4e9c011810 comp rx=0 tx=0).stop 2024-04-23T18:42:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- 172.21.15.33:0/1092265203 shutdown_connections 2024-04-23T18:42:11.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee56d0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- 172.21.15.33:0/1092265203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ed6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- 172.21.15.33:0/1092265203 >> 172.21.15.33:0/1092265203 conn(0x55c230ed2b30 msgr2=0x55c230ed2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- 172.21.15.33:0/1092265203 shutdown_connections 2024-04-23T18:42:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- 172.21.15.33:0/1092265203 wait complete. 2024-04-23T18:42:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 -- start start 2024-04-23T18:42:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 Processor -- start 2024-04-23T18:42:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.430+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c230ee8840 con 0x55c230ed6520 2024-04-23T18:42:12.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c230dd60b0 con 0x55c230ed6e40 2024-04-23T18:42:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c230ee89c0 con 0x55c230ee5c10 2024-04-23T18:42:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 10 client.0 _ll_register_callbacks cb 0x55c230dc3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:12.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37878/0 (socket says 172.21.15.33:37878) 2024-04-23T18:42:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 -- 172.21.15.33:0/218198170 learned_addr learned my addr 172.21.15.33:0/218198170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:12.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:12.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 -- 172.21.15.33:0/218198170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 msgr2=0x55c230ee2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 -- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 msgr2=0x55c230ee2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:13.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 -- 172.21.15.33:0/218198170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c230c1a620 con 0x55c230ed6520 2024-04-23T18:42:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c004630 tx=0x7f4e9c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97df81ae45fa2f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:13.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4e8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c027070 con 0x55c230ed6520 2024-04-23T18:42:13.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f4e9c005240 con 0x55c230ed6520 2024-04-23T18:42:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.431+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c0043d0 con 0x55c230ed6520 2024-04-23T18:42:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c033050 con 0x55c230ed6520 2024-04-23T18:42:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 10 client.17226 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:13.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c230ef4710 con 0x55c230ed6520 2024-04-23T18:42:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c0043d0 con 0x55c230ed6520 2024-04-23T18:42:13.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f4e9c004b70 con 0x55c230ed6520 2024-04-23T18:42:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 10 client.17226 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 10 client.17226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:13.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 20 client.17226 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230ef5b20 0x55c230ef7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 4 client.17226 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c230e49390 con 0x55c230ef5b20 2024-04-23T18:42:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.432+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230ef5b20 0x55c230ef7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.433+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230ef5b20 0x55c230ef7f10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33852fd08bf83bea server_cookie=a860756887f9239c in_seq=0 out_seq=0 2024-04-23T18:42:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.433+0000 7f4e8eff5640 10 client.17226 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:14.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.433+0000 7f4e8eff5640 1 -- 172.21.15.33:0/218198170 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c230e49390 con 0x55c230ef5b20 2024-04-23T18:42:14.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.433+0000 7f4e8eff5640 10 client.17226 handle_command_reply: tid=0 2024-04-23T18:42:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.433+0000 7f4eb4404740 1 client.17226 shutdown 2024-04-23T18:42:14.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 msgr2=0x55c230ee5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:14.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c004630 tx=0x7f4e9c0038a0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 shutdown_connections 2024-04-23T18:42:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230ef5b20 0x55c230ef7f10 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ee2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4e8eff5640 0 client.17226 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 --2- 172.21.15.33:0/218198170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230ee5490 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 >> 172.21.15.33:0/218198170 conn(0x55c230ed2b30 msgr2=0x55c230ed89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 shutdown_connections 2024-04-23T18:42:15.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 1 -- 172.21.15.33:0/218198170 wait complete. 2024-04-23T18:42:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.434+0000 7f4eb4404740 20 client.17226 trim_cache size 0 max 16384 2024-04-23T18:42:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 Processor -- start 2024-04-23T18:42:15.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 -- start start 2024-04-23T18:42:15.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ed1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ad3a0 con 0x55c230ed6520 2024-04-23T18:42:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ab190 con 0x55c230ed6e40 2024-04-23T18:42:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eb4404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1accd0 con 0x55c230ee5c10 2024-04-23T18:42:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37894/0 (socket says 172.21.15.33:37894) 2024-04-23T18:42:15.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 -- 172.21.15.33:0/3843161037 learned_addr learned my addr 172.21.15.33:0/3843161037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 -- 172.21.15.33:0/3843161037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 msgr2=0x55c230ed1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ed1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 -- 172.21.15.33:0/3843161037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 msgr2=0x55c230ee2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:16.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 -- 172.21.15.33:0/3843161037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c230ed2960 con 0x55c230ed6520 2024-04-23T18:42:16.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.449+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000c00 tx=0x7f4e9c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a5dd7524473460 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4e90ff9640 1 -- 172.21.15.33:0/3843161037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c027070 con 0x55c230ed6520 2024-04-23T18:42:16.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4e90ff9640 1 -- 172.21.15.33:0/3843161037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c009910 con 0x55c230ed6520 2024-04-23T18:42:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4e90ff9640 1 -- 172.21.15.33:0/3843161037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0048e0 con 0x55c230ed6520 2024-04-23T18:42:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- 172.21.15.33:0/3843161037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 msgr2=0x55c230eddf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:16.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000c00 tx=0x7f4e9c005970 comp rx=0 tx=0).stop 2024-04-23T18:42:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- 172.21.15.33:0/3843161037 shutdown_connections 2024-04-23T18:42:16.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230ed1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6e40 0x55c230ee2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3843161037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6520 0x55c230eddf00 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:17.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- 172.21.15.33:0/3843161037 >> 172.21.15.33:0/3843161037 conn(0x55c230ed2b30 msgr2=0x55c230ed1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- 172.21.15.33:0/3843161037 shutdown_connections 2024-04-23T18:42:17.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- 172.21.15.33:0/3843161037 wait complete. 2024-04-23T18:42:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.450+0000 7f4eb4404740 1 -- start start 2024-04-23T18:42:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 Processor -- start 2024-04-23T18:42:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ee8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:17.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:17.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230f59f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1abf30 con 0x55c230ed6e40 2024-04-23T18:42:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ab860 con 0x55c230ed6520 2024-04-23T18:42:17.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac1ac600 con 0x55c230ee5c10 2024-04-23T18:42:17.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eb4404740 10 client.0 _ll_register_callbacks cb 0x7f4e9c004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:17.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:17.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37898/0 (socket says 172.21.15.33:37898) 2024-04-23T18:42:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/3312534646 learned_addr learned my addr 172.21.15.33:0/3312534646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eab7fe640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230f59f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ee8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:18.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/3312534646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 msgr2=0x55c230f59f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:18.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230f59f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 msgr2=0x55c230ee8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:18.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ee8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 -- 172.21.15.33:0/3312534646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c230ed7380 con 0x55c230ed6e40 2024-04-23T18:42:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4eaa7fc640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00b9020 tx=0x7f4ea0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef6a76deafa73ae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.451+0000 7f4e8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0049040 con 0x55c230ed6e40 2024-04-23T18:42:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f4ea00ee5f0 con 0x55c230ed6e40 2024-04-23T18:42:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c009910 con 0x55c230ed6e40 2024-04-23T18:42:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4ea00bd2c0 con 0x55c230ed6e40 2024-04-23T18:42:18.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 10 client.17232 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c230f5b690 con 0x55c230ed6e40 2024-04-23T18:42:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c009910 con 0x55c230ed6e40 2024-04-23T18:42:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f4ea00a1070 con 0x55c230ed6e40 2024-04-23T18:42:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 10 client.17232 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 20 client.17232 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230f5be10 0x55c230f5e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 4 client.17232 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c230ee3050 con 0x55c230f5be10 2024-04-23T18:42:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.452+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230f5be10 0x55c230f5e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.453+0000 7f4eaaffd640 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230f5be10 0x55c230f5e230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b9beaa92427a401 server_cookie=a871e44c78c1dd96 in_seq=0 out_seq=0 2024-04-23T18:42:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.453+0000 7f4e8eff5640 10 client.17232 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:19.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.460+0000 7f4e8eff5640 1 -- 172.21.15.33:0/3312534646 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c230ee3050 con 0x55c230f5be10 2024-04-23T18:42:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.460+0000 7f4e8eff5640 10 client.17232 handle_command_reply: tid=0 2024-04-23T18:42:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.460+0000 7f4eb4404740 1 client.17232 shutdown 2024-04-23T18:42:19.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 msgr2=0x55c230ee9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00b9020 tx=0x7f4ea0003f50 comp rx=0 tx=0).stop 2024-04-23T18:42:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 shutdown_connections 2024-04-23T18:42:19.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c230f5be10 0x55c230f5e230 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c230ee5c10 0x55c230f59f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c230ed6e40 0x55c230ee9300 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4e8eff5640 0 client.17232 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 --2- 172.21.15.33:0/3312534646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c230ed6520 0x55c230ee8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 >> 172.21.15.33:0/3312534646 conn(0x55c230ed2b30 msgr2=0x55c230ed1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 shutdown_connections 2024-04-23T18:42:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 1 -- 172.21.15.33:0/3312534646 wait complete. 2024-04-23T18:42:19.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb4404740 20 client.17232 trim_cache size 0 max 16384 2024-04-23T18:42:19.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:42:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:42:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:42:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:42:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 msgr2=0x7f4e940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 0x7f4e940a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005c40 tx=0x7f4ea40094f0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 msgr2=0x7f4eac198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:20.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0006010 tx=0x7f4ea0004060 comp rx=0 tx=0).stop 2024-04-23T18:42:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 shutdown_connections 2024-04-23T18:42:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f4e940a3dc0 0x7f4e940a6280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4eac19cf30 0x7f4eac19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4eac107290 0x7f4eac198bb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 --2- 172.21.15.33:0/1584723109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eac1021c0 0x7f4eac198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:20.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.461+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 >> 172.21.15.33:0/1584723109 conn(0x7f4eac0fda20 msgr2=0x7f4eac0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.462+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 shutdown_connections 2024-04-23T18:42:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:29.462+0000 7f4eb1610640 1 -- 172.21.15.33:0/1584723109 wait complete. 2024-04-23T18:42:20.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:08 smithi106 ceph-mon[32716]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:20.697 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:10 smithi106 ceph-mon[32716]: pgmap v497: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:20.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:10 smithi106 ceph-mon[32716]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:42:20.797 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:12 smithi106 ceph-mon[32716]: pgmap v499: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-23T18:42:20.848 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:14 smithi106 ceph-mon[32716]: pgmap v500: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-23T18:42:20.964 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:15 smithi106 ceph-mon[32716]: daemon mds.d restarted 2024-04-23T18:42:21.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:15 smithi106 ceph-mon[32716]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:42:21.064 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/2982854948,v1:172.21.15.33:6837/2982854948] up:boot 2024-04-23T18:42:21.114 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:15 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:42:21.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:42:21.214 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:16 smithi106 ceph-mon[32716]: pgmap v501: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:42:21.264 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:18 smithi106 ceph-mon[32716]: pgmap v503: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:42:21.314 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:08 smithi122 ceph-mon[32546]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:21.378 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:10 smithi122 ceph-mon[32546]: pgmap v497: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:42:21.427 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:10 smithi122 ceph-mon[32546]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:42:21.464 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:12 smithi122 ceph-mon[32546]: pgmap v499: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-23T18:42:21.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:14 smithi122 ceph-mon[32546]: pgmap v500: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-23T18:42:21.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:15 smithi122 ceph-mon[32546]: daemon mds.d restarted 2024-04-23T18:42:21.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:15 smithi122 ceph-mon[32546]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:42:21.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/2982854948,v1:172.21.15.33:6837/2982854948] up:boot 2024-04-23T18:42:21.779 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:15 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:42:21.829 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:42:21.946 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:16 smithi122 ceph-mon[32546]: pgmap v501: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:42:21.996 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:18 smithi122 ceph-mon[32546]: pgmap v503: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:42:22.048 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:20 smithi106 ceph-mon[32716]: pgmap v504: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:42:22.096 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:20 smithi122 ceph-mon[32546]: pgmap v504: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:42:22.147 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:30 smithi033 ceph-mon[29274]: pgmap v444: 97 pgs: 97 active+clean; 12 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-04-23T18:42:22.197 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:32 smithi033 ceph-mon[29274]: pgmap v445: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-23T18:42:22.247 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:34 smithi033 ceph-mon[29274]: pgmap v446: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-23T18:42:22.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:36 smithi033 ceph-mon[29274]: pgmap v447: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-23T18:42:22.347 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:38 smithi033 ceph-mon[29274]: pgmap v448: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:22.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:40 smithi033 ceph-mon[29274]: pgmap v449: 97 pgs: 97 active+clean; 12 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:22.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 -- 172.21.15.33:0/847600436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 msgr2=0x7f35141025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:22.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 --2- 172.21.15.33:0/847600436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f35141025a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3504009900 tx=0x7f350402a3f0 comp rx=0 tx=0).stop 2024-04-23T18:42:22.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 -- 172.21.15.33:0/847600436 shutdown_connections 2024-04-23T18:42:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 --2- 172.21.15.33:0/847600436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3514107290 0x7f351410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:22.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 --2- 172.21.15.33:0/847600436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514102b70 0x7f3514106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:22.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 --2- 172.21.15.33:0/847600436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f35141025a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 -- 172.21.15.33:0/847600436 >> 172.21.15.33:0/847600436 conn(0x7f35140fda20 msgr2=0x7f35140ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 -- 172.21.15.33:0/847600436 shutdown_connections 2024-04-23T18:42:22.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.561+0000 7f351a0e8640 1 -- 172.21.15.33:0/847600436 wait complete. 2024-04-23T18:42:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 Processor -- start 2024-04-23T18:42:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 -- start start 2024-04-23T18:42:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 0x7f3514198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f351419cef0 0x7f351419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112200 con 0x7f35141021c0 2024-04-23T18:42:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112080 con 0x7f3514107290 2024-04-23T18:42:23.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.562+0000 7f351a0e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3514112380 con 0x7f351419cef0 2024-04-23T18:42:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36110/0 (socket says 172.21.15.33:36110) 2024-04-23T18:42:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 -- 172.21.15.33:0/4150191286 learned_addr learned my addr 172.21.15.33:0/4150191286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3513fff640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f351419cef0 0x7f351419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:23.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3512ffd640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 0x7f3514198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:23.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 -- 172.21.15.33:0/4150191286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f351419cef0 msgr2=0x7f351419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:23.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f351419cef0 0x7f351419d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 -- 172.21.15.33:0/4150191286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 msgr2=0x7f3514198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 0x7f3514198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:23.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 -- 172.21.15.33:0/4150191286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351419d9e0 con 0x7f35141021c0 2024-04-23T18:42:23.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3512ffd640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 0x7f3514198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:42:23.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f35137fe640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3504006010 tx=0x7f35040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3510ff9640 1 -- 172.21.15.33:0/4150191286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350403f030 con 0x7f35141021c0 2024-04-23T18:42:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351419dc70 con 0x7f35141021c0 2024-04-23T18:42:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3510ff9640 1 -- 172.21.15.33:0/4150191286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3504003800 con 0x7f35141021c0 2024-04-23T18:42:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f3510ff9640 1 -- 172.21.15.33:0/4150191286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504035070 con 0x7f35141021c0 2024-04-23T18:42:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.563+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35141a5490 con 0x7f35141021c0 2024-04-23T18:42:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f3510ff9640 1 -- 172.21.15.33:0/4150191286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3504003cb0 con 0x7f35141021c0 2024-04-23T18:42:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 Processor -- start 2024-04-23T18:42:24.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 -- start start 2024-04-23T18:42:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33023a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b3302e800 0x555b33030bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:24.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b32f88970 con 0x555b33023f40 2024-04-23T18:42:24.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555b32f254a0 con 0x555b3302e800 2024-04-23T18:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.566+0000 7f351ce02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b32f84930 con 0x555b33023620 2024-04-23T18:42:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36118/0 (socket says 172.21.15.33:36118) 2024-04-23T18:42:24.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 -- 172.21.15.33:0/1735669123 learned_addr learned my addr 172.21.15.33:0/1735669123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:24.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3513fff640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33023a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3510ff9640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 0x7f34fc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3512ffd640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b3302e800 0x555b33030bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3513fff640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 0x7f34fc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:24.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3510ff9640 1 -- 172.21.15.33:0/4150191286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f35040f1260 con 0x7f35141021c0 2024-04-23T18:42:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 -- 172.21.15.33:0/1735669123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 msgr2=0x555b33023a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:24.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33023a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 -- 172.21.15.33:0/1735669123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b3302e800 msgr2=0x555b33030bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b3302e800 0x555b33030bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 -- 172.21.15.33:0/1735669123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b32c9b210 con 0x555b33023f40 2024-04-23T18:42:25.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3513fff640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33023a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:42:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f35137fe640 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35040bcef0 tx=0x7f350402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413c38d9ac8104e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:25.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f34f9ffb640 1 -- 172.21.15.33:0/1735669123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504049030 con 0x555b33023f40 2024-04-23T18:42:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f34f9ffb640 1 -- 172.21.15.33:0/1735669123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35040bd1f0 con 0x555b33023f40 2024-04-23T18:42:25.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f3513fff640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 0x7f34fc0a6280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f350c008fd0 tx=0x7f350c004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.567+0000 7f34f9ffb640 1 -- 172.21.15.33:0/1735669123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35040eec20 con 0x555b33023f40 2024-04-23T18:42:25.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 -- 172.21.15.33:0/1735669123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 msgr2=0x555b3302e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35040bcef0 tx=0x7f350402a3f0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 -- 172.21.15.33:0/1735669123 shutdown_connections 2024-04-23T18:42:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b3302e800 0x555b33030bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b33023f40 0x555b3302e2c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 --2- 172.21.15.33:0/1735669123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33023a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:25.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 -- 172.21.15.33:0/1735669123 >> 172.21.15.33:0/1735669123 conn(0x555b3301fb30 msgr2=0x555b3301ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 -- 172.21.15.33:0/1735669123 shutdown_connections 2024-04-23T18:42:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.568+0000 7f351ce02740 1 -- 172.21.15.33:0/1735669123 wait complete. 2024-04-23T18:42:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 -- start start 2024-04-23T18:42:25.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 Processor -- start 2024-04-23T18:42:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b3302b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b330311f0 con 0x555b3302e800 2024-04-23T18:42:26.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555b32f230b0 con 0x555b33023f40 2024-04-23T18:42:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b33031370 con 0x555b33023620 2024-04-23T18:42:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f351ce02740 10 client.0 _ll_register_callbacks cb 0x555b32f10d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:26.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36126/0 (socket says 172.21.15.33:36126) 2024-04-23T18:42:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f3512ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b3302b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:26.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f3513fff640 1 -- 172.21.15.33:0/1281461936 learned_addr learned my addr 172.21.15.33:0/1281461936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.569+0000 7f35137fe640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 -- 172.21.15.33:0/1281461936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 msgr2=0x555b3302e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:26.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:26.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 -- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 msgr2=0x555b3302b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b3302b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:26.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 -- 172.21.15.33:0/1281461936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b32d67620 con 0x555b3302e800 2024-04-23T18:42:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3512ffd640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b3302b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:42:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f3513fff640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f350c008d20 tx=0x7f350c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24edd0e65ea4bda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f34f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c018070 con 0x555b3302e800 2024-04-23T18:42:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f350c027400 con 0x555b3302e800 2024-04-23T18:42:27.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35040bd1f0 con 0x555b3302e800 2024-04-23T18:42:27.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f350c00fe90 con 0x555b3302e800 2024-04-23T18:42:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f351ce02740 10 client.17244 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:27.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.570+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555b3303d0b0 con 0x555b3302e800 2024-04-23T18:42:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35040bd1f0 con 0x555b3302e800 2024-04-23T18:42:27.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f350c02f3f0 con 0x555b3302e800 2024-04-23T18:42:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 10 client.17244 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 10 client.17244 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 20 client.17244 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b3303e590 0x555b330409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 4 client.17244 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b32f96390 con 0x555b3303e590 2024-04-23T18:42:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.571+0000 7f35137fe640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b3303e590 0x555b330409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:27.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f35137fe640 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b3303e590 0x555b330409c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2181c3bf16adf7f server_cookie=b2c33fbf96123069 in_seq=0 out_seq=0 2024-04-23T18:42:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f34f2ffd640 10 client.17244 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1281461936 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b32f96390 con 0x555b3303e590 2024-04-23T18:42:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f34f2ffd640 10 client.17244 handle_command_reply: tid=0 2024-04-23T18:42:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f351ce02740 1 client.17244 shutdown 2024-04-23T18:42:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 msgr2=0x555b3302b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:28.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.572+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f350c008d20 tx=0x7f350c004a40 comp rx=0 tx=0).stop 2024-04-23T18:42:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 shutdown_connections 2024-04-23T18:42:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b3303e590 0x555b330409c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:28.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b3302b9c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:28.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b3302b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 --2- 172.21.15.33:0/1281461936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f34f2ffd640 0 client.17244 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 >> 172.21.15.33:0/1281461936 conn(0x555b3301fb30 msgr2=0x555b33025860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 shutdown_connections 2024-04-23T18:42:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 1 -- 172.21.15.33:0/1281461936 wait complete. 2024-04-23T18:42:28.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.573+0000 7f351ce02740 20 client.17244 trim_cache size 0 max 16384 2024-04-23T18:42:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 Processor -- start 2024-04-23T18:42:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 -- start start 2024-04-23T18:42:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b33046320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:29.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35141af650 con 0x555b3302e800 2024-04-23T18:42:29.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35141ad440 con 0x555b33023f40 2024-04-23T18:42:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f351ce02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35141aef80 con 0x555b33023620 2024-04-23T18:42:29.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:29.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36132/0 (socket says 172.21.15.33:36132) 2024-04-23T18:42:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 -- 172.21.15.33:0/1070132165 learned_addr learned my addr 172.21.15.33:0/1070132165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3512ffd640 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b33046320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f35137fe640 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:29.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 -- 172.21.15.33:0/1070132165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 msgr2=0x555b3302dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:29.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:29.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 -- 172.21.15.33:0/1070132165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 msgr2=0x555b33046320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b33046320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.588+0000 7f3513fff640 1 -- 172.21.15.33:0/1070132165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3301b500 con 0x555b3302e800 2024-04-23T18:42:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f3513fff640 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f350c02f840 tx=0x7f350c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520240e45bdd0a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f34f97fa640 1 -- 172.21.15.33:0/1070132165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c035070 con 0x555b3302e800 2024-04-23T18:42:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f34f97fa640 1 -- 172.21.15.33:0/1070132165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c00fc90 con 0x555b3302e800 2024-04-23T18:42:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f34f97fa640 1 -- 172.21.15.33:0/1070132165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c005020 con 0x555b3302e800 2024-04-23T18:42:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 -- 172.21.15.33:0/1070132165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 msgr2=0x555b33046860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:30.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f350c02f840 tx=0x7f350c010820 comp rx=0 tx=0).stop 2024-04-23T18:42:30.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 -- 172.21.15.33:0/1070132165 shutdown_connections 2024-04-23T18:42:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b33046860 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b33046320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 --2- 172.21.15.33:0/1070132165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b3302dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.589+0000 7f351ce02740 1 -- 172.21.15.33:0/1070132165 >> 172.21.15.33:0/1070132165 conn(0x555b3301fb30 msgr2=0x555b3302b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- 172.21.15.33:0/1070132165 shutdown_connections 2024-04-23T18:42:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- 172.21.15.33:0/1070132165 wait complete. 2024-04-23T18:42:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- start start 2024-04-23T18:42:30.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 Processor -- start 2024-04-23T18:42:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33033f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:30.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b330344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:30.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35141ae1e0 con 0x555b3302e800 2024-04-23T18:42:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35141adb10 con 0x555b33023f40 2024-04-23T18:42:30.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35141ae8b0 con 0x555b33023620 2024-04-23T18:42:31.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f351ce02740 10 client.0 _ll_register_callbacks cb 0x7f350c010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:31.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.590+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:31.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36136/0 (socket says 172.21.15.33:36136) 2024-04-23T18:42:31.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 -- 172.21.15.33:0/1080665899 learned_addr learned my addr 172.21.15.33:0/1080665899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:31.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3512ffd640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b330344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f35137fe640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33033f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 -- 172.21.15.33:0/1080665899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 msgr2=0x555b33033f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33033f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 -- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 msgr2=0x555b330344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:32.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b330344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 -- 172.21.15.33:0/1080665899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b330a7fb0 con 0x555b3302e800 2024-04-23T18:42:32.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f3513fff640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f350c02f6f0 tx=0x7f350c02f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a62df8b18fa1214 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f34f2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350c035070 con 0x555b3302e800 2024-04-23T18:42:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f350c02fbb0 con 0x555b3302e800 2024-04-23T18:42:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f350c00fc90 con 0x555b3302e800 2024-04-23T18:42:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f350c00f810 con 0x555b3302e800 2024-04-23T18:42:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f351ce02740 10 client.17250 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:33.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.591+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555b330a92a0 con 0x555b3302e800 2024-04-23T18:42:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f350c00fc90 con 0x555b3302e800 2024-04-23T18:42:33.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f350c004200 con 0x555b3302e800 2024-04-23T18:42:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 10 client.17250 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 20 client.17250 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:33.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b330a9880 0x555b330abca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 4 client.17250 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b32f90200 con 0x555b330a9880 2024-04-23T18:42:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f35137fe640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b330a9880 0x555b330abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f34f2ffd640 10 client.17250 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.592+0000 7f35137fe640 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b330a9880 0x555b330abca0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4be89bd9f927f74 server_cookie=a22bdb10d1c977ce in_seq=0 out_seq=0 2024-04-23T18:42:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f34f2ffd640 1 -- 172.21.15.33:0/1080665899 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b32f90200 con 0x555b330a9880 2024-04-23T18:42:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f34f2ffd640 10 client.17250 handle_command_reply: tid=0 2024-04-23T18:42:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 client.17250 shutdown 2024-04-23T18:42:33.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 msgr2=0x555b330349e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f350c02f6f0 tx=0x7f350c02f870 comp rx=0 tx=0).stop 2024-04-23T18:42:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 shutdown_connections 2024-04-23T18:42:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555b330a9880 0x555b330abca0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b3302e800 0x555b330349e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b33023f40 0x555b330344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f34f2ffd640 0 client.17250 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 --2- 172.21.15.33:0/1080665899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b33023620 0x555b33033f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.600+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 >> 172.21.15.33:0/1080665899 conn(0x555b3301fb30 msgr2=0x555b3301e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 shutdown_connections 2024-04-23T18:42:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351ce02740 1 -- 172.21.15.33:0/1080665899 wait complete. 2024-04-23T18:42:34.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351ce02740 20 client.17250 trim_cache size 0 max 16384 2024-04-23T18:42:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:42:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:42:34.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:42:34.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:42:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 msgr2=0x7f34fc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 0x7f34fc0a6280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f350c008fd0 tx=0x7f350c004ab0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 msgr2=0x7f3514198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3504006010 tx=0x7f35040042f0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 shutdown_connections 2024-04-23T18:42:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34fc0a3dc0 0x7f34fc0a6280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f351419cef0 0x7f351419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3514107290 0x7f3514198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 --2- 172.21.15.33:0/4150191286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35141021c0 0x7f3514198680 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 >> 172.21.15.33:0/4150191286 conn(0x7f35140fda20 msgr2=0x7f35140fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 shutdown_connections 2024-04-23T18:42:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:42.601+0000 7f351a0e8640 1 -- 172.21.15.33:0/4150191286 wait complete. 2024-04-23T18:42:35.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:42 smithi033 ceph-mon[29274]: pgmap v450: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:42:35.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:44 smithi033 ceph-mon[29274]: pgmap v451: 97 pgs: 97 active+clean; 13 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:42:35.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:46 smithi033 ceph-mon[29274]: pgmap v452: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:42:35.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:48 smithi033 ceph-mon[29274]: pgmap v453: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:35.405 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:49 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:42:35.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:49 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:42:35.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:49 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:42:35.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:49 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:42:35.660 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:50 smithi033 ceph-mon[29274]: pgmap v454: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:35.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:52 smithi033 ceph-mon[29274]: pgmap v455: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:42:35.797 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:54 smithi033 ceph-mon[29274]: pgmap v456: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:35.858 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:22 smithi122 ceph-mon[32546]: pgmap v505: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:42:35.905 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:22 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1805597172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:42:35.955 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:24 smithi122 ceph-mon[32546]: pgmap v506: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:42:36.005 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:26 smithi122 ceph-mon[32546]: pgmap v507: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-04-23T18:42:36.055 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:28 smithi122 ceph-mon[32546]: pgmap v508: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:42:36.105 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:30 smithi122 ceph-mon[32546]: pgmap v509: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:42:36.155 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:32 smithi122 ceph-mon[32546]: pgmap v510: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:42:36.205 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:34 smithi122 ceph-mon[32546]: pgmap v511: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:42:36.262 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:22 smithi106 ceph-mon[32716]: pgmap v505: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:42:36.311 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:22 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1805597172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:42:36.389 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:24 smithi106 ceph-mon[32716]: pgmap v506: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:42:36.439 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:26 smithi106 ceph-mon[32716]: pgmap v507: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-04-23T18:42:36.489 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:28 smithi106 ceph-mon[32716]: pgmap v508: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:42:36.539 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:30 smithi106 ceph-mon[32716]: pgmap v509: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:42:36.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:32 smithi106 ceph-mon[32716]: pgmap v510: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:42:36.639 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:34 smithi106 ceph-mon[32716]: pgmap v511: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:42:36.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/1960829398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc10e930 msgr2=0x7fcbcc110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/1960829398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc10e930 0x7fcbcc110d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009ca0 tx=0x7fcbc402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:42:36.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/1960829398 shutdown_connections 2024-04-23T18:42:36.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/1960829398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc10e930 0x7fcbcc110d20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:36.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/1960829398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc1016a0 0x7fcbcc10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:37.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/1960829398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc100d80 0x7fcbcc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:37.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/1960829398 >> 172.21.15.33:0/1960829398 conn(0x7fcbcc077fd0 msgr2=0x7fcbcc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/1960829398 shutdown_connections 2024-04-23T18:42:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.699+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/1960829398 wait complete. 2024-04-23T18:42:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.700+0000 7fcbd1aa7640 1 Processor -- start 2024-04-23T18:42:37.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.700+0000 7fcbd1aa7640 1 -- start start 2024-04-23T18:42:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.700+0000 7fcbd1aa7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbd1aa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc1016a0 0x7fcbcc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbd1aa7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc10e930 0x7fcbcc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbd1aa7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc114390 con 0x7fcbcc100d80 2024-04-23T18:42:37.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbd1aa7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc114210 con 0x7fcbcc10e930 2024-04-23T18:42:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbd1aa7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc114510 con 0x7fcbcc1016a0 2024-04-23T18:42:37.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35036/0 (socket says 172.21.15.33:35036) 2024-04-23T18:42:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/2673263112 learned_addr learned my addr 172.21.15.33:0/2673263112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc10e930 0x7fcbcc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc1016a0 0x7fcbcc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:37.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/2673263112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc1016a0 msgr2=0x7fcbcc1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:37.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc1016a0 0x7fcbcc1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/2673263112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc10e930 msgr2=0x7fcbcc19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:38.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc10e930 0x7fcbcc19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:38.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/2673263112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc19afd0 con 0x7fcbcc100d80 2024-04-23T18:42:38.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00d950 tx=0x7fcbbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.701+0000 7fcbc8ff9640 1 -- 172.21.15.33:0/2673263112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc017030 con 0x7fcbcc100d80 2024-04-23T18:42:38.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.702+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc19b2c0 con 0x7fcbcc100d80 2024-04-23T18:42:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.702+0000 7fcbc8ff9640 1 -- 172.21.15.33:0/2673263112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbbc004020 con 0x7fcbcc100d80 2024-04-23T18:42:38.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.702+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc1a75d0 con 0x7fcbcc100d80 2024-04-23T18:42:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.702+0000 7fcbc8ff9640 1 -- 172.21.15.33:0/2673263112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc021060 con 0x7fcbcc100d80 2024-04-23T18:42:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 Processor -- start 2024-04-23T18:42:38.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 -- start start 2024-04-23T18:42:38.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d58979a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:38.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d589842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d588de970 con 0x563d58984800 2024-04-23T18:42:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbc8ff9640 1 -- 172.21.15.33:0/2673263112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbbc009a30 con 0x7fcbcc100d80 2024-04-23T18:42:38.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563d5887b4a0 con 0x563d58979f40 2024-04-23T18:42:38.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d588da930 con 0x563d58979620 2024-04-23T18:42:38.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.704+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35040/0 (socket says 172.21.15.33:35040) 2024-04-23T18:42:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 -- 172.21.15.33:0/3484990515 learned_addr learned my addr 172.21.15.33:0/3484990515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d58979a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d589842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 -- 172.21.15.33:0/3484990515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 msgr2=0x563d58979a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d58979a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 -- 172.21.15.33:0/3484990515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 msgr2=0x563d589842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d589842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 -- 172.21.15.33:0/3484990515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d585f1210 con 0x563d58984800 2024-04-23T18:42:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c10 tx=0x7fcbc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d99fad7b9faca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbc8ff9640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 0x7fcbb40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbb1ffb640 1 -- 172.21.15.33:0/3484990515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0004230 con 0x563d58984800 2024-04-23T18:42:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbb1ffb640 1 -- 172.21.15.33:0/3484990515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc00043d0 con 0x563d58984800 2024-04-23T18:42:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 0x7fcbb40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbb1ffb640 1 -- 172.21.15.33:0/3484990515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0004d30 con 0x563d58984800 2024-04-23T18:42:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.705+0000 7fcbc8ff9640 1 -- 172.21.15.33:0/2673263112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7fcbbc0cfb00 con 0x7fcbcc100d80 2024-04-23T18:42:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 0x7fcbb40a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006010 tx=0x7fcbc40099a0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3484990515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 msgr2=0x563d58986bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009c10 tx=0x7fcbc0011810 comp rx=0 tx=0).stop 2024-04-23T18:42:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3484990515 shutdown_connections 2024-04-23T18:42:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58984800 0x563d58986bf0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d589842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3484990515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d58979a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3484990515 >> 172.21.15.33:0/3484990515 conn(0x563d58975b30 msgr2=0x563d58975f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:39.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3484990515 shutdown_connections 2024-04-23T18:42:39.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3484990515 wait complete. 2024-04-23T18:42:39.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.706+0000 7fcbd47c1740 1 -- start start 2024-04-23T18:42:39.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 Processor -- start 2024-04-23T18:42:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 0x563d58984110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58981940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d58987430 con 0x563d58979f40 2024-04-23T18:42:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563d588790b0 con 0x563d58979620 2024-04-23T18:42:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d589875b0 con 0x563d58984800 2024-04-23T18:42:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbd47c1740 10 client.0 _ll_register_callbacks cb 0x563d58866d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:40.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35052/0 (socket says 172.21.15.33:35052) 2024-04-23T18:42:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1955903395 learned_addr learned my addr 172.21.15.33:0/1955903395 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:40.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 0x563d58984110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58981940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1955903395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 msgr2=0x563d58981940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58981940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 msgr2=0x563d58984110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:40.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 0x563d58984110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:40.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1955903395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d586bd620 con 0x563d58979f40 2024-04-23T18:42:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.707+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 0x563d58984110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:41.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0000c00 tx=0x7fcbc0005140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d6477e9889ced2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0024620 con 0x563d58979f40 2024-04-23T18:42:41.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7fcbc00247c0 con 0x563d58979f40 2024-04-23T18:42:41.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc00043d0 con 0x563d58979f40 2024-04-23T18:42:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcbc0032050 con 0x563d58979f40 2024-04-23T18:42:41.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 10 client.17262 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d58993310 con 0x563d58979f40 2024-04-23T18:42:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc00043d0 con 0x563d58979f40 2024-04-23T18:42:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fcbc0018070 con 0x563d58979f40 2024-04-23T18:42:41.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 10 client.17262 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:41.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 20 client.17262 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d58994710 0x563d58996b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:41.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 4 client.17262 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:42:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.708+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d588ec390 con 0x563d58994710 2024-04-23T18:42:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.709+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d58994710 0x563d58996b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.709+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d58994710 0x563d58996b00 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93842544c34777ce server_cookie=6a555caaeb19ffbf in_seq=0 out_seq=0 2024-04-23T18:42:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.709+0000 7fcbaaffd640 10 client.17262 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1955903395 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d588ec390 con 0x563d58994710 2024-04-23T18:42:42.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbaaffd640 10 client.17262 handle_command_reply: tid=0 2024-04-23T18:42:42.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 client.17262 shutdown 2024-04-23T18:42:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 msgr2=0x563d58981400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:42.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0000c00 tx=0x7fcbc0005140 comp rx=0 tx=0).stop 2024-04-23T18:42:42.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 shutdown_connections 2024-04-23T18:42:42.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d58994710 0x563d58996b00 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58981940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58981400 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1955903395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979620 0x563d58984110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbaaffd640 0 client.17262 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 >> 172.21.15.33:0/1955903395 conn(0x563d58975b30 msgr2=0x563d5897b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 shutdown_connections 2024-04-23T18:42:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1955903395 wait complete. 2024-04-23T18:42:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.710+0000 7fcbd47c1740 20 client.17262 trim_cache size 0 max 16384 2024-04-23T18:42:42.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.725+0000 7fcbd47c1740 1 Processor -- start 2024-04-23T18:42:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.725+0000 7fcbd47c1740 1 -- start start 2024-04-23T18:42:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d589902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58984800 0x563d5899c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:43.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b1890 con 0x563d58979f40 2024-04-23T18:42:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1af680 con 0x563d58984800 2024-04-23T18:42:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b11c0 con 0x563d58979620 2024-04-23T18:42:43.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35062/0 (socket says 172.21.15.33:35062) 2024-04-23T18:42:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1395168164 learned_addr learned my addr 172.21.15.33:0/1395168164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:43.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58984800 0x563d5899c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d589902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:43.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1395168164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 msgr2=0x563d589902f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d589902f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1395168164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58984800 msgr2=0x563d5899c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:43.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58984800 0x563d5899c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:43.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 -- 172.21.15.33:0/1395168164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d5897f990 con 0x563d58979f40 2024-04-23T18:42:43.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0011850 tx=0x7fcbc00096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8d03c953b3e6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbb17fa640 1 -- 172.21.15.33:0/1395168164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0009a90 con 0x563d58979f40 2024-04-23T18:42:44.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbb17fa640 1 -- 172.21.15.33:0/1395168164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc0029070 con 0x563d58979f40 2024-04-23T18:42:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.726+0000 7fcbb17fa640 1 -- 172.21.15.33:0/1395168164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc0019e20 con 0x563d58979f40 2024-04-23T18:42:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1395168164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 msgr2=0x563d58983c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:44.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0011850 tx=0x7fcbc00096f0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1395168164 shutdown_connections 2024-04-23T18:42:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58984800 0x563d5899c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979f40 0x563d58983c40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/1395168164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58979620 0x563d589902f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:44.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1395168164 >> 172.21.15.33:0/1395168164 conn(0x563d58975b30 msgr2=0x563d5897caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1395168164 shutdown_connections 2024-04-23T18:42:44.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- 172.21.15.33:0/1395168164 wait complete. 2024-04-23T18:42:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.727+0000 7fcbd47c1740 1 -- start start 2024-04-23T18:42:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 Processor -- start 2024-04-23T18:42:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d58987f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58988450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:44.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b0420 con 0x563d58979620 2024-04-23T18:42:44.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1afd50 con 0x563d58979f40 2024-04-23T18:42:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc1b0af0 con 0x563d58984800 2024-04-23T18:42:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbd47c1740 10 client.0 _ll_register_callbacks cb 0x7fcbc0004dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:42:45.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35070/0 (socket says 172.21.15.33:35070) 2024-04-23T18:42:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/3214779688 learned_addr learned my addr 172.21.15.33:0/3214779688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:42:45.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58988450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d58987f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/3214779688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 msgr2=0x563d58988450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58988450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:45.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 msgr2=0x563d58987f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:42:45.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d58987f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:45.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcb7fe640 1 -- 172.21.15.33:0/3214779688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d58988c80 con 0x563d58979620 2024-04-23T18:42:45.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.728+0000 7fcbcbfff640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58988450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:42:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbcb7fe640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09b840 tx=0x7fcbbc0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fcac1e1d1edaea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:42:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:42:45.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0a4700 con 0x563d58979620 2024-04-23T18:42:45.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7fcbbc0bc0b0 con 0x563d58979620 2024-04-23T18:42:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc0029070 con 0x563d58979620 2024-04-23T18:42:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fcbbc0041d0 con 0x563d58979620 2024-04-23T18:42:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 10 client.17268 fetch_fsmap learned FSMap version 37 2024-04-23T18:42:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563d589fab40 con 0x563d58979620 2024-04-23T18:42:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbc0029070 con 0x563d58979620 2024-04-23T18:42:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fcbbc0981b0 con 0x563d58979620 2024-04-23T18:42:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 10 client.17268 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:42:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 10 client.17268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:42:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 20 client.17268 populate_metadata read hostname 'smithi033' 2024-04-23T18:42:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d589fb2b0 0x563d589fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:42:46.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 4 client.17268 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:42:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.729+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d58981cd0 con 0x563d589fb2b0 2024-04-23T18:42:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.730+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d589fb2b0 0x563d589fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:42:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.730+0000 7fcbcaffd640 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d589fb2b0 0x563d589fd6a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e340e495e5be4ef4 server_cookie=5cff3c32fd96e5bb in_seq=0 out_seq=0 2024-04-23T18:42:46.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.730+0000 7fcbaaffd640 10 client.17268 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbaaffd640 1 -- 172.21.15.33:0/3214779688 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d58981cd0 con 0x563d589fb2b0 2024-04-23T18:42:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbaaffd640 10 client.17268 handle_command_reply: tid=0 2024-04-23T18:42:47.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 client.17268 shutdown 2024-04-23T18:42:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 msgr2=0x563d5898cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:47.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc09b840 tx=0x7fcbbc0bc040 comp rx=0 tx=0).stop 2024-04-23T18:42:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 shutdown_connections 2024-04-23T18:42:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563d589fb2b0 0x563d589fd6a0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d58984800 0x563d58988450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563d58979f40 0x563d58987f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbaaffd640 0 client.17268 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:42:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 --2- 172.21.15.33:0/3214779688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d58979620 0x563d5898cba0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 >> 172.21.15.33:0/3214779688 conn(0x563d58975b30 msgr2=0x563d5898f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 shutdown_connections 2024-04-23T18:42:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 1 -- 172.21.15.33:0/3214779688 wait complete. 2024-04-23T18:42:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.738+0000 7fcbd47c1740 20 client.17268 trim_cache size 0 max 16384 2024-04-23T18:42:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:42:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:42:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:42:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:42:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 msgr2=0x7fcbb40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 0x7fcbb40a6280 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4006010 tx=0x7fcbc40099a0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 msgr2=0x7fcbcc1a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:42:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00d950 tx=0x7fcbbc00de10 comp rx=0 tx=0).stop 2024-04-23T18:42:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 shutdown_connections 2024-04-23T18:42:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fcbb40a3dc0 0x7fcbb40a6280 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbcc10e930 0x7fcbcc19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbcc1016a0 0x7fcbcc1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 --2- 172.21.15.33:0/2673263112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbcc100d80 0x7fcbcc1a0650 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:42:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 >> 172.21.15.33:0/2673263112 conn(0x7fcbcc077fd0 msgr2=0x7fcbcc10fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:42:48.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 shutdown_connections 2024-04-23T18:42:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:40:55.739+0000 7fcbd1aa7640 1 -- 172.21.15.33:0/2673263112 wait complete. 2024-04-23T18:42:48.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:56 smithi033 ceph-mon[29274]: pgmap v457: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-23T18:42:48.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:40:58 smithi033 ceph-mon[29274]: pgmap v458: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:42:49.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:00 smithi033 ceph-mon[29274]: pgmap v459: 97 pgs: 97 active+clean; 13 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 KiB/s wr, 12 op/s 2024-04-23T18:42:49.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:02 smithi033 ceph-mon[29274]: pgmap v460: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-23T18:42:49.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:04 smithi033 ceph-mon[29274]: pgmap v461: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 14 op/s 2024-04-23T18:42:49.157 INFO:tasks.workunit.client.0.smithi033.stdout:499/600: open fd = 3 2024-04-23T18:42:49.248 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.299 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.425 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:49.482 INFO:tasks.workunit.client.0.smithi033.stdout:500/600: open fd = 3 2024-04-23T18:42:49.532 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.582 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.632 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:49.682 INFO:tasks.workunit.client.0.smithi033.stdout:501/600: open fd = 3 2024-04-23T18:42:49.732 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.782 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.832 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:49.882 INFO:tasks.workunit.client.0.smithi033.stdout:502/600: open fd = 3 2024-04-23T18:42:49.932 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:49.982 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.032 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:50.082 INFO:tasks.workunit.client.0.smithi033.stdout:503/600: open fd = 3 2024-04-23T18:42:50.210 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.257 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.307 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:50.357 INFO:tasks.workunit.client.0.smithi033.stdout:504/600: open fd = 3 2024-04-23T18:42:50.407 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.482 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.533 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:50.574 INFO:tasks.workunit.client.0.smithi033.stdout:505/600: open fd = 3 2024-04-23T18:42:50.608 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.641 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.674 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:50.708 INFO:tasks.workunit.client.0.smithi033.stdout:506/600: open fd = 3 2024-04-23T18:42:50.833 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.866 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:50.899 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:50.933 INFO:tasks.workunit.client.0.smithi033.stdout:507/600: open fd = 3 2024-04-23T18:42:51.057 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.091 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.131 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:51.183 INFO:tasks.workunit.client.0.smithi033.stdout:508/600: open fd = 3 2024-04-23T18:42:51.216 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.250 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.283 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:51.317 INFO:tasks.workunit.client.0.smithi033.stdout:509/600: open fd = 3 2024-04-23T18:42:51.408 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.441 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.475 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:51.572 INFO:tasks.workunit.client.0.smithi033.stdout:510/600: open fd = 3 2024-04-23T18:42:51.617 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.665 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.708 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:51.742 INFO:tasks.workunit.client.0.smithi033.stdout:511/600: open fd = 3 2024-04-23T18:42:51.775 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.808 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:51.892 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:51.946 INFO:tasks.workunit.client.0.smithi033.stdout:512/600: open fd = 3 2024-04-23T18:42:51.984 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.092 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.205 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:52.242 INFO:tasks.workunit.client.0.smithi033.stdout:513/600: open fd = 3 2024-04-23T18:42:52.275 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.313 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.342 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:52.375 INFO:tasks.workunit.client.0.smithi033.stdout:514/600: open fd = 3 2024-04-23T18:42:52.409 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.442 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.559 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:52.592 INFO:tasks.workunit.client.0.smithi033.stdout:515/600: open fd = 3 2024-04-23T18:42:52.626 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.783 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.817 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:52.851 INFO:tasks.workunit.client.0.smithi033.stdout:516/600: open fd = 3 2024-04-23T18:42:52.884 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.943 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:52.992 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:53.068 INFO:tasks.workunit.client.0.smithi033.stdout:517/600: open fd = 3 2024-04-23T18:42:53.118 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.168 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.218 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:53.268 INFO:tasks.workunit.client.0.smithi033.stdout:518/600: open fd = 3 2024-04-23T18:42:53.412 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.468 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.518 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:53.568 INFO:tasks.workunit.client.0.smithi033.stdout:519/600: open fd = 3 2024-04-23T18:42:53.618 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.668 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.776 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:53.827 INFO:tasks.workunit.client.0.smithi033.stdout:520/600: open fd = 3 2024-04-23T18:42:53.876 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.926 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:53.976 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:54.081 INFO:tasks.workunit.client.0.smithi033.stdout:521/600: open fd = 3 2024-04-23T18:42:54.126 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.176 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.226 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:54.276 INFO:tasks.workunit.client.0.smithi033.stdout:522/600: open fd = 3 2024-04-23T18:42:54.327 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.410 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.460 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:54.518 INFO:tasks.workunit.client.0.smithi033.stdout:523/600: open fd = 3 2024-04-23T18:42:54.579 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.618 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.722 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:54.768 INFO:tasks.workunit.client.0.smithi033.stdout:524/600: open fd = 3 2024-04-23T18:42:54.819 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.871 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:54.919 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:54.969 INFO:tasks.workunit.client.0.smithi033.stdout:525/600: open fd = 3 2024-04-23T18:42:55.019 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.110 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.160 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:55.210 INFO:tasks.workunit.client.0.smithi033.stdout:526/600: open fd = 3 2024-04-23T18:42:55.260 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.399 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.444 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:55.494 INFO:tasks.workunit.client.0.smithi033.stdout:527/600: open fd = 3 2024-04-23T18:42:55.544 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.619 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.669 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:55.719 INFO:tasks.workunit.client.0.smithi033.stdout:528/600: open fd = 3 2024-04-23T18:42:55.827 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.877 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:55.927 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:55.977 INFO:tasks.workunit.client.0.smithi033.stdout:529/600: open fd = 3 2024-04-23T18:42:56.076 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.111 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.152 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:56.202 INFO:tasks.workunit.client.0.smithi033.stdout:530/600: open fd = 3 2024-04-23T18:42:56.253 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.303 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.353 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:56.436 INFO:tasks.workunit.client.0.smithi033.stdout:531/600: open fd = 3 2024-04-23T18:42:56.486 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.536 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.586 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:56.636 INFO:tasks.workunit.client.0.smithi033.stdout:532/600: open fd = 3 2024-04-23T18:42:56.780 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.836 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:56.886 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:56.953 INFO:tasks.workunit.client.0.smithi033.stdout:533/600: open fd = 3 2024-04-23T18:42:57.066 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.111 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.153 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:57.246 INFO:tasks.workunit.client.0.smithi033.stdout:534/600: open fd = 3 2024-04-23T18:42:57.353 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.387 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.420 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:57.527 INFO:tasks.workunit.client.0.smithi033.stdout:535/600: open fd = 3 2024-04-23T18:42:57.579 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.620 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.662 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:57.696 INFO:tasks.workunit.client.0.smithi033.stdout:536/600: open fd = 3 2024-04-23T18:42:57.804 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.837 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:57.911 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:57.945 INFO:tasks.workunit.client.0.smithi033.stdout:537/600: open fd = 3 2024-04-23T18:42:57.987 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.029 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.148 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:58.187 INFO:tasks.workunit.client.0.smithi033.stdout:538/600: open fd = 3 2024-04-23T18:42:58.229 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.263 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.296 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:58.379 INFO:tasks.workunit.client.0.smithi033.stdout:539/600: open fd = 3 2024-04-23T18:42:58.413 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.446 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.479 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:58.512 INFO:tasks.workunit.client.0.smithi033.stdout:540/600: open fd = 3 2024-04-23T18:42:58.565 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.689 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.712 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:58.746 INFO:tasks.workunit.client.0.smithi033.stdout:541/600: open fd = 3 2024-04-23T18:42:58.788 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.822 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.888 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:58.921 INFO:tasks.workunit.client.0.smithi033.stdout:542/600: open fd = 3 2024-04-23T18:42:58.954 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:58.988 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.021 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:59.096 INFO:tasks.workunit.client.0.smithi033.stdout:543/600: open fd = 3 2024-04-23T18:42:59.246 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.271 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.305 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:59.341 INFO:tasks.workunit.client.0.smithi033.stdout:544/600: open fd = 3 2024-04-23T18:42:59.372 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.513 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.546 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:59.580 INFO:tasks.workunit.client.0.smithi033.stdout:545/600: open fd = 3 2024-04-23T18:42:59.613 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.647 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.680 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:59.758 INFO:tasks.workunit.client.0.smithi033.stdout:546/600: open fd = 3 2024-04-23T18:42:59.823 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.855 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:42:59.888 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:42:59.980 INFO:tasks.workunit.client.0.smithi033.stdout:547/600: open fd = 3 2024-04-23T18:43:00.022 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.047 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.080 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:00.114 INFO:tasks.workunit.client.0.smithi033.stdout:548/600: open fd = 3 2024-04-23T18:43:00.164 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.214 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.379 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:00.422 INFO:tasks.workunit.client.0.smithi033.stdout:549/600: open fd = 3 2024-04-23T18:43:00.472 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.639 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:00.689 INFO:tasks.workunit.client.0.smithi033.stdout:550/600: open fd = 3 2024-04-23T18:43:00.739 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.789 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:00.839 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:00.889 INFO:tasks.workunit.client.0.smithi033.stdout:551/600: open fd = 3 2024-04-23T18:43:00.972 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.030 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.107 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:01.156 INFO:tasks.workunit.client.0.smithi033.stdout:552/600: open fd = 3 2024-04-23T18:43:01.206 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.323 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.373 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:01.423 INFO:tasks.workunit.client.0.smithi033.stdout:553/600: open fd = 3 2024-04-23T18:43:01.473 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.573 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:01.623 INFO:tasks.workunit.client.0.smithi033.stdout:554/600: open fd = 3 2024-04-23T18:43:01.756 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.798 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:01.848 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:01.948 INFO:tasks.workunit.client.0.smithi033.stdout:555/600: open fd = 3 2024-04-23T18:43:02.023 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.073 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.123 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:02.181 INFO:tasks.workunit.client.0.smithi033.stdout:556/600: open fd = 3 2024-04-23T18:43:02.231 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.281 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.331 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:02.468 INFO:tasks.workunit.client.0.smithi033.stdout:557/600: open fd = 3 2024-04-23T18:43:02.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.632 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.682 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:02.807 INFO:tasks.workunit.client.0.smithi033.stdout:558/600: open fd = 3 2024-04-23T18:43:02.857 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.907 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:02.957 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:03.032 INFO:tasks.workunit.client.0.smithi033.stdout:559/600: open fd = 3 2024-04-23T18:43:03.082 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:03.132 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:03.182 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:03.300 INFO:tasks.workunit.client.0.smithi033.stdout:560/600: open fd = 3 2024-04-23T18:43:03.332 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:03.382 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:03.482 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:03.533 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:06 smithi033 ceph-mon[29274]: pgmap v462: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 86 KiB/s wr, 37 op/s 2024-04-23T18:43:03.587 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:36 smithi122 ceph-mon[32546]: pgmap v512: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:03.649 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:38 smithi122 ceph-mon[32546]: pgmap v513: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:03.699 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:41 smithi122 ceph-mon[32546]: pgmap v514: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:03.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:43 smithi122 ceph-mon[32546]: pgmap v515: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:03.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:45 smithi122 ceph-mon[32546]: pgmap v516: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:03.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:47 smithi122 ceph-mon[32546]: pgmap v517: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:03.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:49 smithi122 ceph-mon[32546]: pgmap v518: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:03.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:51 smithi122 ceph-mon[32546]: pgmap v519: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:04.032 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:51 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:04.157 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:04.207 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:04.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:43:04.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:53 smithi122 ceph-mon[32546]: pgmap v520: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:04.357 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:55 smithi122 ceph-mon[32546]: pgmap v521: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:04.407 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:57 smithi122 ceph-mon[32546]: pgmap v522: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:04.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:42:59 smithi122 ceph-mon[32546]: pgmap v523: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:04.507 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:01 smithi122 ceph-mon[32546]: pgmap v524: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:04.628 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:03 smithi122 ceph-mon[32546]: pgmap v525: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:04.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:36 smithi106 ceph-mon[32716]: pgmap v512: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:04.747 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:38 smithi106 ceph-mon[32716]: pgmap v513: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:04.833 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:41 smithi106 ceph-mon[32716]: pgmap v514: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:04.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:43 smithi106 ceph-mon[32716]: pgmap v515: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:04.933 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:45 smithi106 ceph-mon[32716]: pgmap v516: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:04.966 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:47 smithi106 ceph-mon[32716]: pgmap v517: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:05.008 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:49 smithi106 ceph-mon[32716]: pgmap v518: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:05.041 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:51 smithi106 ceph-mon[32716]: pgmap v519: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:05.092 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:51 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:43:05.124 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:43:05.176 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:43:05.228 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:43:05.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:53 smithi106 ceph-mon[32716]: pgmap v520: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:05.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:55 smithi106 ceph-mon[32716]: pgmap v521: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:05.366 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:57 smithi106 ceph-mon[32716]: pgmap v522: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:05.405 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:42:59 smithi106 ceph-mon[32716]: pgmap v523: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:05.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:01 smithi106 ceph-mon[32716]: pgmap v524: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:43:05.483 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:03 smithi106 ceph-mon[32716]: pgmap v525: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:05.518 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:05 smithi106 ceph-mon[32716]: pgmap v526: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:05.550 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:05 smithi122 ceph-mon[32546]: pgmap v526: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 -- 172.21.15.33:0/2879285568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc102b70 msgr2=0x7f73dc106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 --2- 172.21.15.33:0/2879285568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc102b70 0x7f73dc106bb0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009900 tx=0x7f73cc02a410 comp rx=0 tx=0).stop 2024-04-23T18:43:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 -- 172.21.15.33:0/2879285568 shutdown_connections 2024-04-23T18:43:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 --2- 172.21.15.33:0/2879285568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc107290 0x7f73dc10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 --2- 172.21.15.33:0/2879285568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc102b70 0x7f73dc106bb0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 --2- 172.21.15.33:0/2879285568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 0x7f73dc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:05.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 -- 172.21.15.33:0/2879285568 >> 172.21.15.33:0/2879285568 conn(0x7f73dc0fda20 msgr2=0x7f73dc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 -- 172.21.15.33:0/2879285568 shutdown_connections 2024-04-23T18:43:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.839+0000 7f73e3619640 1 -- 172.21.15.33:0/2879285568 wait complete. 2024-04-23T18:43:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 Processor -- start 2024-04-23T18:43:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 -- start start 2024-04-23T18:43:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 0x7f73dc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc19cf90 0x7f73dc19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1121b0 con 0x7f73dc107290 2024-04-23T18:43:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc112030 con 0x7f73dc1021c0 2024-04-23T18:43:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e3619640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc112330 con 0x7f73dc19cf90 2024-04-23T18:43:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49374/0 (socket says 172.21.15.33:49374) 2024-04-23T18:43:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.840+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2705427199 learned_addr learned my addr 172.21.15.33:0/2705427199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:06.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e13b5640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 0x7f73dc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc19cf90 0x7f73dc19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2705427199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc19cf90 msgr2=0x7f73dc19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc19cf90 0x7f73dc19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2705427199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 msgr2=0x7f73dc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 0x7f73dc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2705427199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc19da80 con 0x7f73dc107290 2024-04-23T18:43:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006010 tx=0x7f73cc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:06.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73d27fc640 1 -- 172.21.15.33:0/2705427199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc03f030 con 0x7f73dc107290 2024-04-23T18:43:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73d27fc640 1 -- 172.21.15.33:0/2705427199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc003aa0 con 0x7f73dc107290 2024-04-23T18:43:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc19dd10 con 0x7f73dc107290 2024-04-23T18:43:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73d27fc640 1 -- 172.21.15.33:0/2705427199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc035070 con 0x7f73dc107290 2024-04-23T18:43:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.841+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc1a54d0 con 0x7f73dc107290 2024-04-23T18:43:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73d27fc640 1 -- 172.21.15.33:0/2705427199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f73cc042030 con 0x7f73dc107290 2024-04-23T18:43:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 Processor -- start 2024-04-23T18:43:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 -- start start 2024-04-23T18:43:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0620 0x557d402e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:07.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402eb800 0x557d402edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557d40245970 con 0x557d402e0f40 2024-04-23T18:43:07.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d401e24a0 con 0x557d402eb800 2024-04-23T18:43:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73e6405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557d40241930 con 0x557d402e0620 2024-04-23T18:43:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.844+0000 7f73d27fc640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 0x7f73c80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:07.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 0x7f73c80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:08.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49380/0 (socket says 172.21.15.33:49380) 2024-04-23T18:43:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 -- 172.21.15.33:0/1428499668 learned_addr learned my addr 172.21.15.33:0/1428499668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:08.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0620 0x557d402e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73d27fc640 1 -- 172.21.15.33:0/2705427199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f73cc0f0c10 con 0x7f73dc107290 2024-04-23T18:43:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402eb800 0x557d402edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 -- 172.21.15.33:0/1428499668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0620 msgr2=0x557d402e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0620 0x557d402e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 -- 172.21.15.33:0/1428499668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402eb800 msgr2=0x557d402edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:08.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402eb800 0x557d402edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 -- 172.21.15.33:0/1428499668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d3ff58210 con 0x557d402e0f40 2024-04-23T18:43:08.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 0x7f73c80a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009fd0 tx=0x7f73d4009d60 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009cf0 tx=0x7f73d8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b1b8a14edab975 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73c77fe640 1 -- 172.21.15.33:0/1428499668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8004220 con 0x557d402e0f40 2024-04-23T18:43:08.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73c77fe640 1 -- 172.21.15.33:0/1428499668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d80043c0 con 0x557d402e0f40 2024-04-23T18:43:08.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.845+0000 7f73c77fe640 1 -- 172.21.15.33:0/1428499668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8018770 con 0x557d402e0f40 2024-04-23T18:43:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 -- 172.21.15.33:0/1428499668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 msgr2=0x557d402eb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009cf0 tx=0x7f73d8011810 comp rx=0 tx=0).stop 2024-04-23T18:43:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 -- 172.21.15.33:0/1428499668 shutdown_connections 2024-04-23T18:43:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402eb800 0x557d402edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:09.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402eb2c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:09.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 --2- 172.21.15.33:0/1428499668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0620 0x557d402e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 -- 172.21.15.33:0/1428499668 >> 172.21.15.33:0/1428499668 conn(0x557d402dcb30 msgr2=0x557d402dcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 -- 172.21.15.33:0/1428499668 shutdown_connections 2024-04-23T18:43:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.846+0000 7f73e6405740 1 -- 172.21.15.33:0/1428499668 wait complete. 2024-04-23T18:43:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 -- start start 2024-04-23T18:43:09.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 Processor -- start 2024-04-23T18:43:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402eb800 0x557d402e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:09.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557d402ee430 con 0x557d402e0f40 2024-04-23T18:43:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557d401e00b0 con 0x557d402e0620 2024-04-23T18:43:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557d402ee5b0 con 0x557d402eb800 2024-04-23T18:43:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e6405740 10 client.0 _ll_register_callbacks cb 0x557d401cdd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:10.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:10.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49392/0 (socket says 172.21.15.33:49392) 2024-04-23T18:43:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2874588916 learned_addr learned my addr 172.21.15.33:0/2874588916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:10.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e13b5640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.847+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402eb800 0x557d402e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2874588916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402eb800 msgr2=0x557d402e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402eb800 0x557d402e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 msgr2=0x557d402eb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:10.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402eb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:10.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 -- 172.21.15.33:0/2874588916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d40024620 con 0x557d402e0f40 2024-04-23T18:43:10.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0092a0 tx=0x7f73cc0bca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25738b6ef3f7225 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc049040 con 0x557d402e0f40 2024-04-23T18:43:10.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f73cc0b9260 con 0x557d402e0f40 2024-04-23T18:43:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d80043c0 con 0x557d402e0f40 2024-04-23T18:43:10.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73cc03b460 con 0x557d402e0f40 2024-04-23T18:43:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 10 client.17280 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d402fe720 con 0x557d402e0f40 2024-04-23T18:43:10.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d80043c0 con 0x557d402e0f40 2024-04-23T18:43:11.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f73cc0ee570 con 0x557d402e0f40 2024-04-23T18:43:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 10 client.17280 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:11.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 10 client.17280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.848+0000 7f73e6405740 20 client.17280 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402ffc20 0x557d40302050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73e6405740 4 client.17280 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d40253390 con 0x557d402ffc20 2024-04-23T18:43:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73e13b5640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402ffc20 0x557d40302050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73c4ff9640 10 client.17280 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.849+0000 7f73e13b5640 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402ffc20 0x557d40302050 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ec3299822b52beb server_cookie=40f5848fe9cf0935 in_seq=0 out_seq=0 2024-04-23T18:43:11.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73c4ff9640 1 -- 172.21.15.33:0/2874588916 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d40253390 con 0x557d402ffc20 2024-04-23T18:43:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73c4ff9640 10 client.17280 handle_command_reply: tid=0 2024-04-23T18:43:11.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 client.17280 shutdown 2024-04-23T18:43:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 msgr2=0x557d402e8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:11.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0092a0 tx=0x7f73cc0bca80 comp rx=0 tx=0).stop 2024-04-23T18:43:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 shutdown_connections 2024-04-23T18:43:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402ffc20 0x557d40302050 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402eb800 0x557d402e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0f40 0x557d402e8410 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 --2- 172.21.15.33:0/2874588916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73c4ff9640 0 client.17280 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 >> 172.21.15.33:0/2874588916 conn(0x557d402dcb30 msgr2=0x557d402e2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 shutdown_connections 2024-04-23T18:43:11.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 1 -- 172.21.15.33:0/2874588916 wait complete. 2024-04-23T18:43:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.850+0000 7f73e6405740 20 client.17280 trim_cache size 0 max 16384 2024-04-23T18:43:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.865+0000 7f73e6405740 1 Processor -- start 2024-04-23T18:43:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.865+0000 7f73e6405740 1 -- start start 2024-04-23T18:43:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402dcb30 0x557d402dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 0x557d40307e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:12.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1ad380 con 0x557d402e0620 2024-04-23T18:43:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1ab170 con 0x557d402e0f40 2024-04-23T18:43:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e6405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1accb0 con 0x557d402dcb30 2024-04-23T18:43:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49396/0 (socket says 172.21.15.33:49396) 2024-04-23T18:43:12.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 -- 172.21.15.33:0/3834622700 learned_addr learned my addr 172.21.15.33:0/3834622700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:12.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e13b5640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402dcb30 0x557d402dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:12.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 0x557d40307e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 -- 172.21.15.33:0/3834622700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402dcb30 msgr2=0x557d402dbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:12.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402dcb30 0x557d402dbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 -- 172.21.15.33:0/3834622700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 msgr2=0x557d40307e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:12.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 0x557d40307e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 -- 172.21.15.33:0/3834622700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d402e0d70 con 0x557d402e0620 2024-04-23T18:43:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 0x557d40307e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0b95c0 tx=0x7f73cc02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceac2e01d97b34f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.866+0000 7f73c6ffd640 1 -- 172.21.15.33:0/3834622700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc049040 con 0x557d402e0620 2024-04-23T18:43:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73c6ffd640 1 -- 172.21.15.33:0/3834622700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc0bcbf0 con 0x557d402e0620 2024-04-23T18:43:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73c6ffd640 1 -- 172.21.15.33:0/3834622700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc003d60 con 0x557d402e0620 2024-04-23T18:43:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- 172.21.15.33:0/3834622700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 msgr2=0x557d40307910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0b95c0 tx=0x7f73cc02a410 comp rx=0 tx=0).stop 2024-04-23T18:43:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- 172.21.15.33:0/3834622700 shutdown_connections 2024-04-23T18:43:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0f40 0x557d40307e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402e0620 0x557d40307910 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 --2- 172.21.15.33:0/3834622700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402dcb30 0x557d402dbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:13.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- 172.21.15.33:0/3834622700 >> 172.21.15.33:0/3834622700 conn(0x557d402e8ab0 msgr2=0x557d402db420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- 172.21.15.33:0/3834622700 shutdown_connections 2024-04-23T18:43:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- 172.21.15.33:0/3834622700 wait complete. 2024-04-23T18:43:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.867+0000 7f73e6405740 1 -- start start 2024-04-23T18:43:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 Processor -- start 2024-04-23T18:43:13.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:13.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0f40 0x557d402f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1abf10 con 0x557d402dcb30 2024-04-23T18:43:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1ab840 con 0x557d402e0620 2024-04-23T18:43:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1ac5e0 con 0x557d402e0f40 2024-04-23T18:43:13.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e6405740 10 client.0 _ll_register_callbacks cb 0x7f73cc0b8f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:13.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49404/0 (socket says 172.21.15.33:49404) 2024-04-23T18:43:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 -- 172.21.15.33:0/1570596085 learned_addr learned my addr 172.21.15.33:0/1570596085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e1bb6640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0f40 0x557d402f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 -- 172.21.15.33:0/1570596085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0f40 msgr2=0x557d402f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0f40 0x557d402f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 -- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 msgr2=0x557d402f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:14.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e13b5640 1 -- 172.21.15.33:0/1570596085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d403651e0 con 0x557d402dcb30 2024-04-23T18:43:14.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.868+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:43:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73e13b5640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x557d402e1bc0 tx=0x7f73d8009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce42bd0fc75f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:14.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d80035f0 con 0x557d402dcb30 2024-04-23T18:43:14.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8542+0+0 (secure 0 0 0) 0x7f73d8003790 con 0x557d402dcb30 2024-04-23T18:43:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73cc0bcbf0 con 0x557d402dcb30 2024-04-23T18:43:14.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f73d8011d40 con 0x557d402dcb30 2024-04-23T18:43:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73e6405740 10 client.17286 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:14.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x557d40366460 con 0x557d402dcb30 2024-04-23T18:43:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc0bcbf0 con 0x557d402dcb30 2024-04-23T18:43:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f73d8005140 con 0x557d402dcb30 2024-04-23T18:43:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.869+0000 7f73e6405740 10 client.17286 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e6405740 10 client.17286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e6405740 20 client.17286 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402eb800 0x557d40368a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e6405740 4 client.17286 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:15.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d4024d200 con 0x557d402eb800 2024-04-23T18:43:15.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402eb800 0x557d40368a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:15.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73e0bb4640 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402eb800 0x557d40368a00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351e86a8f9bc21a3 server_cookie=8196640beaad1e4a in_seq=0 out_seq=0 2024-04-23T18:43:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.870+0000 7f73c4ff9640 10 client.17286 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.878+0000 7f73c4ff9640 1 -- 172.21.15.33:0/1570596085 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d4024d200 con 0x557d402eb800 2024-04-23T18:43:15.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.878+0000 7f73c4ff9640 10 client.17286 handle_command_reply: tid=0 2024-04-23T18:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 client.17286 shutdown 2024-04-23T18:43:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 msgr2=0x557d402f56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x557d402e1bc0 tx=0x7f73d8009c60 comp rx=0 tx=0).stop 2024-04-23T18:43:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 shutdown_connections 2024-04-23T18:43:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557d402eb800 0x557d40368a00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557d402e0f40 0x557d402f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557d402e0620 0x557d402f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73c4ff9640 0 client.17286 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:15.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 --2- 172.21.15.33:0/1570596085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d402dcb30 0x557d402f56f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:15.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 >> 172.21.15.33:0/1570596085 conn(0x557d402e8ab0 msgr2=0x557d402dbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 shutdown_connections 2024-04-23T18:43:15.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 1 -- 172.21.15.33:0/1570596085 wait complete. 2024-04-23T18:43:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.879+0000 7f73e6405740 20 client.17286 trim_cache size 0 max 16384 2024-04-23T18:43:16.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:43:16.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:43:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:43:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:43:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 msgr2=0x7f73c80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:16.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 0x7f73c80a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009fd0 tx=0x7f73d4009d60 comp rx=0 tx=0).stop 2024-04-23T18:43:16.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 msgr2=0x7f73dc198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:16.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f73cc006010 tx=0x7f73cc004060 comp rx=0 tx=0).stop 2024-04-23T18:43:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 shutdown_connections 2024-04-23T18:43:16.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73c80a3dc0 0x7f73c80a6280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73dc19cf90 0x7f73dc19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:16.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73dc107290 0x7f73dc198c10 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:16.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 --2- 172.21.15.33:0/2705427199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73dc1021c0 0x7f73dc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 >> 172.21.15.33:0/2705427199 conn(0x7f73dc0fda20 msgr2=0x7f73dc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 shutdown_connections 2024-04-23T18:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:08.880+0000 7f73e3619640 1 -- 172.21.15.33:0/2705427199 wait complete. 2024-04-23T18:43:16.783 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:07 smithi122 ceph-mon[32546]: pgmap v527: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:43:16.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:09 smithi122 ceph-mon[32546]: pgmap v528: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:43:16.849 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:10 smithi122 ceph-mon[32546]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:43:16.883 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:11 smithi122 ceph-mon[32546]: pgmap v530: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:43:16.916 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:13 smithi122 ceph-mon[32546]: pgmap v531: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:43:16.959 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:15 smithi122 ceph-mon[32546]: pgmap v532: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:43:17.037 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:07 smithi106 ceph-mon[32716]: pgmap v527: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:43:17.075 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:09 smithi106 ceph-mon[32716]: pgmap v528: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:43:17.108 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:10 smithi106 ceph-mon[32716]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:43:17.141 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:11 smithi106 ceph-mon[32716]: pgmap v530: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:43:17.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:13 smithi106 ceph-mon[32716]: pgmap v531: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:43:17.258 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:15 smithi106 ceph-mon[32716]: pgmap v532: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:43:17.293 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:09 smithi033 ceph-mon[29274]: pgmap v463: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-04-23T18:43:17.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:10 smithi033 ceph-mon[29274]: osdmap e110: 12 total, 12 up, 12 in 2024-04-23T18:43:17.358 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:11 smithi033 ceph-mon[29274]: pgmap v465: 97 pgs: 97 active+clean; 13 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 37 op/s 2024-04-23T18:43:17.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:13 smithi033 ceph-mon[29274]: pgmap v466: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:43:17.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:15 smithi033 ceph-mon[29274]: pgmap v467: 97 pgs: 97 active+clean; 13 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 91 KiB/s wr, 38 op/s 2024-04-23T18:43:17.523 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:16 smithi033 ceph-mon[29274]: pgmap v468: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:43:17.600 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:16 smithi033 ceph-mon[29274]: osdmap e111: 12 total, 12 up, 12 in 2024-04-23T18:43:17.633 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:18 smithi033 ceph-mon[29274]: pgmap v470: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 20 op/s 2024-04-23T18:43:17.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:20 smithi033 ceph-mon[29274]: pgmap v471: 97 pgs: 97 active+clean; 14 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:43:17.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:16 smithi122 ceph-mon[32546]: pgmap v533: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:43:17.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:16 smithi122 ceph-mon[32546]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:43:17.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:16 smithi106 ceph-mon[32716]: pgmap v533: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:43:17.828 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:16 smithi106 ceph-mon[32716]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:43:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 -- 172.21.15.33:0/179284072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c101aa0 msgr2=0x7f3c5c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:17.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 --2- 172.21.15.33:0/179284072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c101aa0 0x7f3c5c105a90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009900 tx=0x7f3c4402a410 comp rx=0 tx=0).stop 2024-04-23T18:43:18.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 -- 172.21.15.33:0/179284072 shutdown_connections 2024-04-23T18:43:18.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 --2- 172.21.15.33:0/179284072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c106170 0x7f3c5c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 --2- 172.21.15.33:0/179284072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c101aa0 0x7f3c5c105a90 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 --2- 172.21.15.33:0/179284072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 -- 172.21.15.33:0/179284072 >> 172.21.15.33:0/179284072 conn(0x7f3c5c077fd0 msgr2=0x7f3c5c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 -- 172.21.15.33:0/179284072 shutdown_connections 2024-04-23T18:43:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 -- 172.21.15.33:0/179284072 wait complete. 2024-04-23T18:43:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.979+0000 7f3c613fe640 1 Processor -- start 2024-04-23T18:43:18.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 -- start start 2024-04-23T18:43:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c106170 0x7f3c5c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c19cf90 0x7f3c5c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:18.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1121b0 con 0x7f3c5c1010f0 2024-04-23T18:43:18.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c112030 con 0x7f3c5c106170 2024-04-23T18:43:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c613fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c112330 con 0x7f3c5c19cf90 2024-04-23T18:43:18.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39476/0 (socket says 172.21.15.33:39476) 2024-04-23T18:43:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 -- 172.21.15.33:0/3123226363 learned_addr learned my addr 172.21.15.33:0/3123226363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c106170 0x7f3c5c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:18.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c19cf90 0x7f3c5c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 -- 172.21.15.33:0/3123226363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c19cf90 msgr2=0x7f3c5c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:18.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c19cf90 0x7f3c5c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 -- 172.21.15.33:0/3123226363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c106170 msgr2=0x7f3c5c198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c106170 0x7f3c5c198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:19.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.980+0000 7f3c5affd640 1 -- 172.21.15.33:0/3123226363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c19da80 con 0x7f3c5c1010f0 2024-04-23T18:43:19.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c5affd640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00ea60 tx=0x7f3c4c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c53fff640 1 -- 172.21.15.33:0/3123226363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00cde0 con 0x7f3c5c1010f0 2024-04-23T18:43:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c53fff640 1 -- 172.21.15.33:0/3123226363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c0035f0 con 0x7f3c5c1010f0 2024-04-23T18:43:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c19dd70 con 0x7f3c5c1010f0 2024-04-23T18:43:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c53fff640 1 -- 172.21.15.33:0/3123226363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c010690 con 0x7f3c5c1010f0 2024-04-23T18:43:19.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.981+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c1a5530 con 0x7f3c5c1010f0 2024-04-23T18:43:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.983+0000 7f3c53fff640 1 -- 172.21.15.33:0/3123226363 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c4c0037a0 con 0x7f3c5c1010f0 2024-04-23T18:43:19.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 Processor -- start 2024-04-23T18:43:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 -- start start 2024-04-23T18:43:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c53fff640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 0x7f3c480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c53fff640 1 -- 172.21.15.33:0/3123226363 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f3c4c0d0ab0 con 0x7f3c5c1010f0 2024-04-23T18:43:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 0x7f3c480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513520 0x5580cd513900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:19.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd525000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:20.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580cd478970 con 0x5580cd513e40 2024-04-23T18:43:20.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5580cd4154a0 con 0x5580cd513520 2024-04-23T18:43:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c641ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5580cd474930 con 0x5580cd522c10 2024-04-23T18:43:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39492/0 (socket says 172.21.15.33:39492) 2024-04-23T18:43:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5a7fc640 1 -- 172.21.15.33:0/65394696 learned_addr learned my addr 172.21.15.33:0/65394696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5affd640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513520 0x5580cd513900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.984+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd525000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 0x7f3c480a6280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005fd0 tx=0x7f3c54011810 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 -- 172.21.15.33:0/65394696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 msgr2=0x5580cd525000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:20.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd525000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 -- 172.21.15.33:0/65394696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513520 msgr2=0x5580cd513900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:20.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513520 0x5580cd513900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:20.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 -- 172.21.15.33:0/65394696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd18b210 con 0x5580cd513e40 2024-04-23T18:43:20.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd525000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009900 tx=0x7f3c44005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d189deda9ee7a215 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c517fa640 1 -- 172.21.15.33:0/65394696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44040070 con 0x5580cd513e40 2024-04-23T18:43:20.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c517fa640 1 -- 172.21.15.33:0/65394696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c44004030 con 0x5580cd513e40 2024-04-23T18:43:20.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.985+0000 7f3c517fa640 1 -- 172.21.15.33:0/65394696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44035040 con 0x5580cd513e40 2024-04-23T18:43:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- 172.21.15.33:0/65394696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 msgr2=0x5580cd5226d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009900 tx=0x7f3c44005e40 comp rx=0 tx=0).stop 2024-04-23T18:43:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- 172.21.15.33:0/65394696 shutdown_connections 2024-04-23T18:43:20.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd525000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513e40 0x5580cd5226d0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 --2- 172.21.15.33:0/65394696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513520 0x5580cd513900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- 172.21.15.33:0/65394696 >> 172.21.15.33:0/65394696 conn(0x5580cd50fb30 msgr2=0x5580cd50ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- 172.21.15.33:0/65394696 shutdown_connections 2024-04-23T18:43:21.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- 172.21.15.33:0/65394696 wait complete. 2024-04-23T18:43:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 -- start start 2024-04-23T18:43:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.986+0000 7f3c641ef740 1 Processor -- start 2024-04-23T18:43:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580cd5257e0 con 0x5580cd513520 2024-04-23T18:43:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5580cd4130b0 con 0x5580cd513e40 2024-04-23T18:43:21.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5580cd525960 con 0x5580cd522c10 2024-04-23T18:43:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 10 client.0 _ll_register_callbacks cb 0x5580cd400d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39494/0 (socket says 172.21.15.33:39494) 2024-04-23T18:43:21.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 -- 172.21.15.33:0/1685501996 learned_addr learned my addr 172.21.15.33:0/1685501996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 -- 172.21.15.33:0/1685501996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 msgr2=0x5580cd51fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 -- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 msgr2=0x5580cd51f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 -- 172.21.15.33:0/1685501996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cd257620 con 0x5580cd513520 2024-04-23T18:43:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c5affd640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0ce250 tx=0x7f3c4c09c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b23811ac38e95b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c427fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0a0df0 con 0x5580cd513520 2024-04-23T18:43:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f3c4c0ce470 con 0x5580cd513520 2024-04-23T18:43:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.987+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c44004030 con 0x5580cd513520 2024-04-23T18:43:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c0a56b0 con 0x5580cd513520 2024-04-23T18:43:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 10 client.17298 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5580cd531740 con 0x5580cd513520 2024-04-23T18:43:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c44004030 con 0x5580cd513520 2024-04-23T18:43:22.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f3c4c0b9020 con 0x5580cd513520 2024-04-23T18:43:22.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 10 client.17298 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:22.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 20 client.17298 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd532c00 0x5580cd535050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:22.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 4 client.17298 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580cd486390 con 0x5580cd532c00 2024-04-23T18:43:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.988+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd532c00 0x5580cd535050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.989+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd532c00 0x5580cd535050 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f49ed2f27bdedea7 server_cookie=f7cec319ef9e83d7 in_seq=0 out_seq=0 2024-04-23T18:43:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.989+0000 7f3c427fc640 10 client.17298 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c427fc640 1 -- 172.21.15.33:0/1685501996 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580cd486390 con 0x5580cd532c00 2024-04-23T18:43:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c427fc640 10 client.17298 handle_command_reply: tid=0 2024-04-23T18:43:22.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 client.17298 shutdown 2024-04-23T18:43:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 msgr2=0x5580cd522520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0ce250 tx=0x7f3c4c09c3a0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 shutdown_connections 2024-04-23T18:43:23.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd532c00 0x5580cd535050 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c427fc640 0 client.17298 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 --2- 172.21.15.33:0/1685501996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd522520 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 >> 172.21.15.33:0/1685501996 conn(0x5580cd50fb30 msgr2=0x5580cd5159a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:23.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 shutdown_connections 2024-04-23T18:43:23.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 1 -- 172.21.15.33:0/1685501996 wait complete. 2024-04-23T18:43:23.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:21.990+0000 7f3c641ef740 20 client.17298 trim_cache size 0 max 16384 2024-04-23T18:43:23.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 Processor -- start 2024-04-23T18:43:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 -- start start 2024-04-23T18:43:23.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:23.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd513e40 0x5580cd51ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:23.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd522c10 0x5580cd522050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:23.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ad3e0 con 0x5580cd513520 2024-04-23T18:43:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ab1d0 con 0x5580cd522c10 2024-04-23T18:43:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c641ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1acd10 con 0x5580cd513e40 2024-04-23T18:43:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:24.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.005+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39506/0 (socket says 172.21.15.33:39506) 2024-04-23T18:43:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 -- 172.21.15.33:0/508397345 learned_addr learned my addr 172.21.15.33:0/508397345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd513e40 0x5580cd51ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 -- 172.21.15.33:0/508397345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd513e40 msgr2=0x5580cd51ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:24.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd513e40 0x5580cd51ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 -- 172.21.15.33:0/508397345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd522c10 msgr2=0x5580cd522050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:24.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd522c10 0x5580cd522050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 -- 172.21.15.33:0/508397345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd513c70 con 0x5580cd513520 2024-04-23T18:43:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c5affd640 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00d010 tx=0x7f3c4c09cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e44cc98d497ccb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c50ff9640 1 -- 172.21.15.33:0/508397345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0bd0c0 con 0x5580cd513520 2024-04-23T18:43:24.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c50ff9640 1 -- 172.21.15.33:0/508397345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c09d140 con 0x5580cd513520 2024-04-23T18:43:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.006+0000 7f3c50ff9640 1 -- 172.21.15.33:0/508397345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c087150 con 0x5580cd513520 2024-04-23T18:43:24.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- 172.21.15.33:0/508397345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 msgr2=0x5580cd52e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00d010 tx=0x7f3c4c09cbc0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- 172.21.15.33:0/508397345 shutdown_connections 2024-04-23T18:43:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd522c10 0x5580cd522050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd513e40 0x5580cd51ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- 172.21.15.33:0/508397345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52e6a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- 172.21.15.33:0/508397345 >> 172.21.15.33:0/508397345 conn(0x5580cd50fb30 msgr2=0x5580cd51a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- 172.21.15.33:0/508397345 shutdown_connections 2024-04-23T18:43:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- 172.21.15.33:0/508397345 wait complete. 2024-04-23T18:43:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- start start 2024-04-23T18:43:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 Processor -- start 2024-04-23T18:43:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:25.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1abf70 con 0x5580cd513520 2024-04-23T18:43:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ab8a0 con 0x5580cd513e40 2024-04-23T18:43:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.007+0000 7f3c641ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1ac640 con 0x5580cd522c10 2024-04-23T18:43:25.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c641ef740 10 client.0 _ll_register_callbacks cb 0x7f3c4c09c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:25.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39514/0 (socket says 172.21.15.33:39514) 2024-04-23T18:43:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 -- 172.21.15.33:0/2492399211 learned_addr learned my addr 172.21.15.33:0/2492399211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 -- 172.21.15.33:0/2492399211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 msgr2=0x5580cd51c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 -- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 msgr2=0x5580cd51c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:25.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:25.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 -- 172.21.15.33:0/2492399211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cd51cff0 con 0x5580cd513520 2024-04-23T18:43:25.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0ce7f0 tx=0x7f3c4c09c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f07cf6d57695c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c427fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:26.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c09c090 con 0x5580cd513520 2024-04-23T18:43:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f3c4c09b900 con 0x5580cd513520 2024-04-23T18:43:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4c09d140 con 0x5580cd513520 2024-04-23T18:43:26.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c09bb00 con 0x5580cd513520 2024-04-23T18:43:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c641ef740 10 client.17304 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.008+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5580cd5992c0 con 0x5580cd513520 2024-04-23T18:43:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c4c09d140 con 0x5580cd513520 2024-04-23T18:43:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f3c4c0b9020 con 0x5580cd513520 2024-04-23T18:43:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 10 client.17304 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 20 client.17304 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd5996f0 0x5580cd59bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 4 client.17304 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580cd5200e0 con 0x5580cd5996f0 2024-04-23T18:43:26.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd5996f0 0x5580cd59bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c427fc640 10 client.17304 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.009+0000 7f3c5a7fc640 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd5996f0 0x5580cd59bb00 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=187d97d8590208db server_cookie=401fba47457f75e5 in_seq=0 out_seq=0 2024-04-23T18:43:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.017+0000 7f3c427fc640 1 -- 172.21.15.33:0/2492399211 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580cd5200e0 con 0x5580cd5996f0 2024-04-23T18:43:26.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.017+0000 7f3c427fc640 10 client.17304 handle_command_reply: tid=0 2024-04-23T18:43:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.017+0000 7f3c641ef740 1 client.17304 shutdown 2024-04-23T18:43:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 msgr2=0x5580cd52b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:27.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c0ce7f0 tx=0x7f3c4c09c4d0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 shutdown_connections 2024-04-23T18:43:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5580cd5996f0 0x5580cd59bb00 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580cd522c10 0x5580cd51c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580cd513e40 0x5580cd51c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c427fc640 0 client.17304 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 --2- 172.21.15.33:0/2492399211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5580cd513520 0x5580cd52b020 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 >> 172.21.15.33:0/2492399211 conn(0x5580cd50fb30 msgr2=0x5580cd515110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 shutdown_connections 2024-04-23T18:43:27.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 1 -- 172.21.15.33:0/2492399211 wait complete. 2024-04-23T18:43:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c641ef740 20 client.17304 trim_cache size 0 max 16384 2024-04-23T18:43:27.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:43:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:43:27.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:43:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:43:27.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 msgr2=0x7f3c480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:27.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 0x7f3c480a6280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005fd0 tx=0x7f3c54011810 comp rx=0 tx=0).stop 2024-04-23T18:43:27.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 msgr2=0x7f3c5c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:27.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00ea60 tx=0x7f3c4c00ef20 comp rx=0 tx=0).stop 2024-04-23T18:43:27.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 shutdown_connections 2024-04-23T18:43:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3c480a3dc0 0x7f3c480a6280 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:27.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c5c19cf90 0x7f3c5c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3c5c106170 0x7f3c5c198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 --2- 172.21.15.33:0/3123226363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c1010f0 0x7f3c5c1986d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.018+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 >> 172.21.15.33:0/3123226363 conn(0x7f3c5c077fd0 msgr2=0x7f3c5c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:28.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.019+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 shutdown_connections 2024-04-23T18:43:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:22.019+0000 7f3c613fe640 1 -- 172.21.15.33:0/3123226363 wait complete. 2024-04-23T18:43:28.325 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:18 smithi106 ceph-mon[32716]: pgmap v535: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-23T18:43:28.357 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:20 smithi106 ceph-mon[32716]: pgmap v536: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:43:28.391 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:22 smithi106 ceph-mon[32716]: pgmap v537: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:43:28.424 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:24 smithi106 ceph-mon[32716]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:43:28.457 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:26 smithi106 ceph-mon[32716]: pgmap v539: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:43:28.505 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:18 smithi122 ceph-mon[32546]: pgmap v535: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-23T18:43:28.601 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:20 smithi122 ceph-mon[32546]: pgmap v536: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:43:28.641 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:22 smithi122 ceph-mon[32546]: pgmap v537: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:43:28.674 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:24 smithi122 ceph-mon[32546]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:43:28.788 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:26 smithi122 ceph-mon[32546]: pgmap v539: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:43:28.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:22 smithi033 ceph-mon[29274]: pgmap v472: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:43:28.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:24 smithi033 ceph-mon[29274]: pgmap v473: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-23T18:43:28.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:26 smithi033 ceph-mon[29274]: pgmap v474: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-23T18:43:28.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:28 smithi033 ceph-mon[29274]: pgmap v475: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:43:28.999 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:30 smithi033 ceph-mon[29274]: pgmap v476: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:43:29.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:32 smithi033 ceph-mon[29274]: pgmap v477: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:43:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 -- 172.21.15.33:0/4143579399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103b20 msgr2=0x7fe7d010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:29.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/4143579399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103b20 0x7fe7d010c000 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009900 tx=0x7fe7c002a480 comp rx=0 tx=0).stop 2024-04-23T18:43:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 -- 172.21.15.33:0/4143579399 shutdown_connections 2024-04-23T18:43:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/4143579399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/4143579399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103b20 0x7fe7d010c000 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/4143579399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103200 0x7fe7d01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 -- 172.21.15.33:0/4143579399 >> 172.21.15.33:0/4143579399 conn(0x7fe7d00771d0 msgr2=0x7fe7d00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 -- 172.21.15.33:0/4143579399 shutdown_connections 2024-04-23T18:43:29.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 -- 172.21.15.33:0/4143579399 wait complete. 2024-04-23T18:43:29.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.118+0000 7fe7d83f7640 1 Processor -- start 2024-04-23T18:43:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 -- start start 2024-04-23T18:43:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:29.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103b20 0x7fe7d0198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:29.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01121f0 con 0x7fe7d0103200 2024-04-23T18:43:29.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112070 con 0x7fe7d0103b20 2024-04-23T18:43:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d83f7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112370 con 0x7fe7d010c5d0 2024-04-23T18:43:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:29.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38544/0 (socket says 172.21.15.33:38544) 2024-04-23T18:43:29.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 -- 172.21.15.33:0/1192457251 learned_addr learned my addr 172.21.15.33:0/1192457251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d5992640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103b20 0x7fe7d0198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 -- 172.21.15.33:0/1192457251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 msgr2=0x7fe7d019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:30.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 -- 172.21.15.33:0/1192457251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103b20 msgr2=0x7fe7d0198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103b20 0x7fe7d0198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6193640 1 -- 172.21.15.33:0/1192457251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d019d680 con 0x7fe7d0103200 2024-04-23T18:43:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.119+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7d6193640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7c77fe640 1 -- 172.21.15.33:0/1192457251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc017030 con 0x7fe7d0103200 2024-04-23T18:43:30.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7c77fe640 1 -- 172.21.15.33:0/1192457251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7cc004020 con 0x7fe7d0103200 2024-04-23T18:43:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d019d970 con 0x7fe7d0103200 2024-04-23T18:43:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7c77fe640 1 -- 172.21.15.33:0/1192457251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0047a0 con 0x7fe7d0103200 2024-04-23T18:43:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.120+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a5200 con 0x7fe7d0103200 2024-04-23T18:43:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.122+0000 7fe7c77fe640 1 -- 172.21.15.33:0/1192457251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe7cc009a30 con 0x7fe7d0103200 2024-04-23T18:43:30.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 Processor -- start 2024-04-23T18:43:30.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 -- start start 2024-04-23T18:43:30.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:30.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7e60 0x55a86e9d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9d6c10 0x55a86e9d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:31.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a86e92c970 con 0x55a86e9c7540 2024-04-23T18:43:31.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a86e8c94a0 con 0x55a86e9c7e60 2024-04-23T18:43:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a86e928930 con 0x55a86e9d6c10 2024-04-23T18:43:31.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7d6994640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7d6994640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38552/0 (socket says 172.21.15.33:38552) 2024-04-23T18:43:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7d6994640 1 -- 172.21.15.33:0/3832423511 learned_addr learned my addr 172.21.15.33:0/3832423511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7c77fe640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 0x7fe7bc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7d5992640 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9d6c10 0x55a86e9d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.123+0000 7fe7d6193640 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7e60 0x55a86e9d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 0x7fe7bc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:31.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7c77fe640 1 -- 172.21.15.33:0/1192457251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fe7cc0d0380 con 0x7fe7d0103200 2024-04-23T18:43:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 -- 172.21.15.33:0/3832423511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9d6c10 msgr2=0x55a86e9d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9d6c10 0x55a86e9d9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:31.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 -- 172.21.15.33:0/3832423511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7e60 msgr2=0x55a86e9d66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:31.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7e60 0x55a86e9d66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 -- 172.21.15.33:0/3832423511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e63f210 con 0x55a86e9c7540 2024-04-23T18:43:31.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80149d0 tx=0x7fe7c8014e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bd85af4431cb2f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7c4bfe640 1 -- 172.21.15.33:0/3832423511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8014470 con 0x55a86e9c7540 2024-04-23T18:43:32.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 0x7fe7bc0a6280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8022000 tx=0x7fe7c8020000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7c4bfe640 1 -- 172.21.15.33:0/3832423511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8023070 con 0x55a86e9c7540 2024-04-23T18:43:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.124+0000 7fe7c4bfe640 1 -- 172.21.15.33:0/3832423511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c800cc90 con 0x55a86e9c7540 2024-04-23T18:43:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- 172.21.15.33:0/3832423511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 msgr2=0x55a86e9c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80149d0 tx=0x7fe7c8014e90 comp rx=0 tx=0).stop 2024-04-23T18:43:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- 172.21.15.33:0/3832423511 shutdown_connections 2024-04-23T18:43:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9d6c10 0x55a86e9d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:32.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7e60 0x55a86e9d66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/3832423511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7540 0x55a86e9c7920 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- 172.21.15.33:0/3832423511 >> 172.21.15.33:0/3832423511 conn(0x55a86e9c3b30 msgr2=0x55a86e9c3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- 172.21.15.33:0/3832423511 shutdown_connections 2024-04-23T18:43:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- 172.21.15.33:0/3832423511 wait complete. 2024-04-23T18:43:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.125+0000 7fe7db1e3740 1 -- start start 2024-04-23T18:43:32.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 Processor -- start 2024-04-23T18:43:32.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a86e9d9840 con 0x55a86e9c7e60 2024-04-23T18:43:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a86e8c70b0 con 0x55a86e9d6c10 2024-04-23T18:43:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a86e9d99c0 con 0x55a86e9c7540 2024-04-23T18:43:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7db1e3740 10 client.0 _ll_register_callbacks cb 0x55a86e8b4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38560/0 (socket says 172.21.15.33:38560) 2024-04-23T18:43:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 -- 172.21.15.33:0/2659660561 learned_addr learned my addr 172.21.15.33:0/2659660561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:33.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d6994640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:33.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 -- 172.21.15.33:0/2659660561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 msgr2=0x55a86e9d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 -- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 msgr2=0x55a86e9d3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9d3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:33.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.126+0000 7fe7d5992640 1 -- 172.21.15.33:0/2659660561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e70b620 con 0x55a86e9c7e60 2024-04-23T18:43:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7d5992640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a9a0 tx=0x7fe7c00317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f5aa50a72c29e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:33.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:33.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0035040 con 0x55a86e9c7e60 2024-04-23T18:43:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fe7c0004040 con 0x55a86e9c7e60 2024-04-23T18:43:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c8023070 con 0x55a86e9c7e60 2024-04-23T18:43:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7c0004240 con 0x55a86e9c7e60 2024-04-23T18:43:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 10 client.17316 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a86e9e57d0 con 0x55a86e9c7e60 2024-04-23T18:43:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8023070 con 0x55a86e9c7e60 2024-04-23T18:43:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fe7c0004aa0 con 0x55a86e9c7e60 2024-04-23T18:43:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 10 client.17316 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 20 client.17316 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86e9e6c90 0x55a86e9e9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:34.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 4 client.17316 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.127+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86e93a390 con 0x55a86e9e6c90 2024-04-23T18:43:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.128+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86e9e6c90 0x55a86e9e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.128+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86e9e6c90 0x55a86e9e9080 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e476a589c104f2ba server_cookie=a6b0c386a329108d in_seq=0 out_seq=0 2024-04-23T18:43:34.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.128+0000 7fe7b9ffb640 10 client.17316 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2659660561 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a86e93a390 con 0x55a86e9e6c90 2024-04-23T18:43:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7b9ffb640 10 client.17316 handle_command_reply: tid=0 2024-04-23T18:43:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 client.17316 shutdown 2024-04-23T18:43:34.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 msgr2=0x55a86e9d38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:34.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a9a0 tx=0x7fe7c00317a0 comp rx=0 tx=0).stop 2024-04-23T18:43:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 shutdown_connections 2024-04-23T18:43:34.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86e9e6c90 0x55a86e9e9080 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9d3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d38d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7b9ffb640 0 client.17316 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2659660561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 >> 172.21.15.33:0/2659660561 conn(0x55a86e9c3b30 msgr2=0x55a86e9c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 shutdown_connections 2024-04-23T18:43:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2659660561 wait complete. 2024-04-23T18:43:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.129+0000 7fe7db1e3740 20 client.17316 trim_cache size 0 max 16384 2024-04-23T18:43:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.144+0000 7fe7db1e3740 1 Processor -- start 2024-04-23T18:43:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.144+0000 7fe7db1e3740 1 -- start start 2024-04-23T18:43:35.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.144+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9cf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:35.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.144+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:35.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.144+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9ee970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01ad450 con 0x55a86e9c7e60 2024-04-23T18:43:35.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01ab240 con 0x55a86e9d6c10 2024-04-23T18:43:35.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01acd80 con 0x55a86e9c7540 2024-04-23T18:43:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38566/0 (socket says 172.21.15.33:38566) 2024-04-23T18:43:36.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 -- 172.21.15.33:0/1219286688 learned_addr learned my addr 172.21.15.33:0/1219286688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d6193640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 -- 172.21.15.33:0/1219286688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 msgr2=0x55a86e9cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:36.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9cf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 -- 172.21.15.33:0/1219286688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 msgr2=0x55a86e9ee970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:36.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9ee970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:36.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 -- 172.21.15.33:0/1219286688 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86e9d44e0 con 0x55a86e9c7e60 2024-04-23T18:43:36.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d6994640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d6193640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:43:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7d5992640 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009850 tx=0x7fe7c002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd9661d6041402 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7bbfff640 1 -- 172.21.15.33:0/1219286688 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0035050 con 0x55a86e9c7e60 2024-04-23T18:43:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7bbfff640 1 -- 172.21.15.33:0/1219286688 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0003f70 con 0x55a86e9c7e60 2024-04-23T18:43:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.145+0000 7fe7bbfff640 1 -- 172.21.15.33:0/1219286688 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0051070 con 0x55a86e9c7e60 2024-04-23T18:43:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- 172.21.15.33:0/1219286688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 msgr2=0x55a86e9d6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009850 tx=0x7fe7c002a480 comp rx=0 tx=0).stop 2024-04-23T18:43:36.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- 172.21.15.33:0/1219286688 shutdown_connections 2024-04-23T18:43:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9d6c10 0x55a86e9ee970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:37.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9c7e60 0x55a86e9d6110 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:37.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/1219286688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7540 0x55a86e9cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- 172.21.15.33:0/1219286688 >> 172.21.15.33:0/1219286688 conn(0x55a86e9c3b30 msgr2=0x55a86e9c2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:37.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- 172.21.15.33:0/1219286688 shutdown_connections 2024-04-23T18:43:37.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- 172.21.15.33:0/1219286688 wait complete. 2024-04-23T18:43:37.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 -- start start 2024-04-23T18:43:37.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.146+0000 7fe7db1e3740 1 Processor -- start 2024-04-23T18:43:37.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7540 0x55a86ea46560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7e60 0x55a86ea46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01abfe0 con 0x55a86e9d6c10 2024-04-23T18:43:37.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01ab910 con 0x55a86e9c7540 2024-04-23T18:43:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01ac6b0 con 0x55a86e9c7e60 2024-04-23T18:43:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 10 client.0 _ll_register_callbacks cb 0x7fe7c00314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38582/0 (socket says 172.21.15.33:38582) 2024-04-23T18:43:37.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 -- 172.21.15.33:0/2482234351 learned_addr learned my addr 172.21.15.33:0/2482234351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d5992640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7e60 0x55a86ea46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:38.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7540 0x55a86ea46560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:38.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 -- 172.21.15.33:0/2482234351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7e60 msgr2=0x55a86ea46aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7e60 0x55a86ea46aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 -- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7540 msgr2=0x55a86ea46560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7540 0x55a86ea46560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 -- 172.21.15.33:0/2482234351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86e9c3960 con 0x55a86e9d6c10 2024-04-23T18:43:38.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7d6994640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80149d0 tx=0x7fe7c8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f9f0b44e6c44be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:38.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8009840 con 0x55a86e9d6c10 2024-04-23T18:43:38.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fe7c800c950 con 0x55a86e9d6c10 2024-04-23T18:43:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.147+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7c0003f70 con 0x55a86e9d6c10 2024-04-23T18:43:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe7c800cb50 con 0x55a86e9d6c10 2024-04-23T18:43:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 10 client.17322 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55a86ea4da70 con 0x55a86e9d6c10 2024-04-23T18:43:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c0003f70 con 0x55a86e9d6c10 2024-04-23T18:43:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fe7c8031070 con 0x55a86e9d6c10 2024-04-23T18:43:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 10 client.17322 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 20 client.17322 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:39.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86ea4e120 0x55a86ea50540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:39.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 4 client.17322 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:43:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a86e934200 con 0x55a86ea4e120 2024-04-23T18:43:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.148+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86ea4e120 0x55a86ea50540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.149+0000 7fe7d6193640 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86ea4e120 0x55a86ea50540 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06d159b5199939c server_cookie=c52c12d88c6596d2 in_seq=0 out_seq=0 2024-04-23T18:43:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.149+0000 7fe7b9ffb640 10 client.17322 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7b9ffb640 1 -- 172.21.15.33:0/2482234351 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a86e934200 con 0x55a86ea4e120 2024-04-23T18:43:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7b9ffb640 10 client.17322 handle_command_reply: tid=0 2024-04-23T18:43:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 client.17322 shutdown 2024-04-23T18:43:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 msgr2=0x55a86e9dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80149d0 tx=0x7fe7c8004a90 comp rx=0 tx=0).stop 2024-04-23T18:43:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 shutdown_connections 2024-04-23T18:43:39.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a86ea4e120 0x55a86ea50540 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:39.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a86e9d6c10 0x55a86e9dae70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7b9ffb640 0 client.17322 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:40.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a86e9c7e60 0x55a86ea46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 --2- 172.21.15.33:0/2482234351 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a86e9c7540 0x55a86ea46560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 >> 172.21.15.33:0/2482234351 conn(0x55a86e9c3b30 msgr2=0x55a86e9c2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 shutdown_connections 2024-04-23T18:43:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 1 -- 172.21.15.33:0/2482234351 wait complete. 2024-04-23T18:43:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.157+0000 7fe7db1e3740 20 client.17322 trim_cache size 0 max 16384 2024-04-23T18:43:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:43:40.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:43:40.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:43:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:43:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 msgr2=0x7fe7bc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 0x7fe7bc0a6280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8022000 tx=0x7fe7c8020000 comp rx=0 tx=0).stop 2024-04-23T18:43:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 msgr2=0x7fe7d0198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:40.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00d950 tx=0x7fe7cc00de10 comp rx=0 tx=0).stop 2024-04-23T18:43:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 shutdown_connections 2024-04-23T18:43:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7bc0a3dc0 0x7fe7bc0a6280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d010c5d0 0x7fe7d019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7d0103b20 0x7fe7d0198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 --2- 172.21.15.33:0/1192457251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d0103200 0x7fe7d0198690 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 >> 172.21.15.33:0/1192457251 conn(0x7fe7d00771d0 msgr2=0x7fe7d00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 shutdown_connections 2024-04-23T18:43:41.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:35.158+0000 7fe7d83f7640 1 -- 172.21.15.33:0/1192457251 wait complete. 2024-04-23T18:43:41.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:28 smithi106 ceph-mon[32716]: pgmap v540: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:30 smithi106 ceph-mon[32716]: pgmap v541: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:32 smithi106 ceph-mon[32716]: pgmap v542: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:43:41.561 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:34 smithi106 ceph-mon[32716]: pgmap v543: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:36 smithi106 ceph-mon[32716]: pgmap v544: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:43:41.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:38 smithi106 ceph-mon[32716]: pgmap v545: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:41.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:28 smithi122 ceph-mon[32546]: pgmap v540: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.762 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:30 smithi122 ceph-mon[32546]: pgmap v541: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.802 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:32 smithi122 ceph-mon[32546]: pgmap v542: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:43:41.836 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:34 smithi122 ceph-mon[32546]: pgmap v543: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:43:41.907 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:36 smithi122 ceph-mon[32546]: pgmap v544: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:43:41.981 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:38 smithi122 ceph-mon[32546]: pgmap v545: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:42.012 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:40 smithi122 ceph-mon[32546]: pgmap v546: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:42.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:34 smithi033 ceph-mon[29274]: pgmap v478: 97 pgs: 97 active+clean; 14 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:43:42.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:36 smithi033 ceph-mon[29274]: pgmap v479: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-23T18:43:42.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:38 smithi033 ceph-mon[29274]: pgmap v480: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:43:42.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:40 smithi033 ceph-mon[29274]: pgmap v481: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 KiB/s wr, 14 op/s 2024-04-23T18:43:42.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:42 smithi033 ceph-mon[29274]: pgmap v482: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 46 KiB/s wr, 21 op/s 2024-04-23T18:43:42.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:44 smithi033 ceph-mon[29274]: pgmap v483: 97 pgs: 97 active+clean; 14 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-23T18:43:42.337 INFO:tasks.workunit.client.0.smithi033.stdout:561/600: open fd = 3 2024-04-23T18:43:42.399 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.469 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.510 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:42.545 INFO:tasks.workunit.client.0.smithi033.stdout:562/600: open fd = 3 2024-04-23T18:43:42.579 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.670 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.704 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:42.737 INFO:tasks.workunit.client.0.smithi033.stdout:563/600: open fd = 3 2024-04-23T18:43:42.770 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.804 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.837 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:42.871 INFO:tasks.workunit.client.0.smithi033.stdout:564/600: open fd = 3 2024-04-23T18:43:42.955 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:42.996 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.029 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:43.075 INFO:tasks.workunit.client.0.smithi033.stdout:565/600: open fd = 3 2024-04-23T18:43:43.154 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.188 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.221 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:43.254 INFO:tasks.workunit.client.0.smithi033.stdout:566/600: open fd = 3 2024-04-23T18:43:43.296 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.346 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.396 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:43.489 INFO:tasks.workunit.client.0.smithi033.stdout:567/600: open fd = 3 2024-04-23T18:43:43.538 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.580 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.621 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:43.672 INFO:tasks.workunit.client.0.smithi033.stdout:568/600: open fd = 3 2024-04-23T18:43:43.763 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.813 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:43.863 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:43.930 INFO:tasks.workunit.client.0.smithi033.stdout:569/600: open fd = 3 2024-04-23T18:43:43.980 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.030 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.097 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:44.143 INFO:tasks.workunit.client.0.smithi033.stdout:570/600: open fd = 3 2024-04-23T18:43:44.197 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.247 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.354 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:44.405 INFO:tasks.workunit.client.0.smithi033.stdout:571/600: open fd = 3 2024-04-23T18:43:44.455 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.505 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.555 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:44.605 INFO:tasks.workunit.client.0.smithi033.stdout:572/600: open fd = 3 2024-04-23T18:43:44.655 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.705 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.822 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:44.872 INFO:tasks.workunit.client.0.smithi033.stdout:573/600: open fd = 3 2024-04-23T18:43:44.922 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:44.997 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.047 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:45.097 INFO:tasks.workunit.client.0.smithi033.stdout:574/600: open fd = 3 2024-04-23T18:43:45.156 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.231 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.280 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:45.330 INFO:tasks.workunit.client.0.smithi033.stdout:575/600: open fd = 3 2024-04-23T18:43:45.381 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.480 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.531 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:45.581 INFO:tasks.workunit.client.0.smithi033.stdout:576/600: open fd = 3 2024-04-23T18:43:45.691 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.739 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.789 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:45.839 INFO:tasks.workunit.client.0.smithi033.stdout:577/600: open fd = 3 2024-04-23T18:43:45.889 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.939 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:45.989 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:46.039 INFO:tasks.workunit.client.0.smithi033.stdout:578/600: open fd = 3 2024-04-23T18:43:46.130 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.173 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.223 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:46.273 INFO:tasks.workunit.client.0.smithi033.stdout:579/600: open fd = 3 2024-04-23T18:43:46.389 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.439 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.489 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:46.539 INFO:tasks.workunit.client.0.smithi033.stdout:580/600: open fd = 3 2024-04-23T18:43:46.590 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.640 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.690 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:46.773 INFO:tasks.workunit.client.0.smithi033.stdout:581/600: open fd = 3 2024-04-23T18:43:46.823 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.873 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:46.923 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:47.040 INFO:tasks.workunit.client.0.smithi033.stdout:582/600: open fd = 3 2024-04-23T18:43:47.090 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.140 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.190 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:47.240 INFO:tasks.workunit.client.0.smithi033.stdout:583/600: open fd = 3 2024-04-23T18:43:47.298 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.348 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.417 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:47.465 INFO:tasks.workunit.client.0.smithi033.stdout:584/600: open fd = 3 2024-04-23T18:43:47.507 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.557 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.640 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:47.682 INFO:tasks.workunit.client.0.smithi033.stdout:585/600: open fd = 3 2024-04-23T18:43:47.723 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.765 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:47.807 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:47.849 INFO:tasks.workunit.client.0.smithi033.stdout:586/600: open fd = 3 2024-04-23T18:43:47.899 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.028 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.065 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:48.167 INFO:tasks.workunit.client.0.smithi033.stdout:587/600: open fd = 3 2024-04-23T18:43:48.232 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.266 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.299 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:48.332 INFO:tasks.workunit.client.0.smithi033.stdout:588/600: open fd = 3 2024-04-23T18:43:48.366 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.399 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.433 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:48.466 INFO:tasks.workunit.client.0.smithi033.stdout:589/600: open fd = 3 2024-04-23T18:43:48.551 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.583 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.691 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:48.724 INFO:tasks.workunit.client.0.smithi033.stdout:590/600: open fd = 3 2024-04-23T18:43:48.758 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.791 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.824 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:48.872 INFO:tasks.workunit.client.0.smithi033.stdout:591/600: open fd = 3 2024-04-23T18:43:48.908 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.941 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:48.975 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.057 INFO:tasks.workunit.client.0.smithi033.stdout:592/600: open fd = 3 2024-04-23T18:43:49.091 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.125 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.191 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.225 INFO:tasks.workunit.client.0.smithi033.stdout:593/600: open fd = 3 2024-04-23T18:43:49.258 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.291 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.325 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.358 INFO:tasks.workunit.client.0.smithi033.stdout:594/600: open fd = 3 2024-04-23T18:43:49.392 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.425 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.516 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.542 INFO:tasks.workunit.client.0.smithi033.stdout:595/600: open fd = 3 2024-04-23T18:43:49.629 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.667 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.700 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.734 INFO:tasks.workunit.client.0.smithi033.stdout:596/600: open fd = 3 2024-04-23T18:43:49.767 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.800 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.834 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:49.867 INFO:tasks.workunit.client.0.smithi033.stdout:597/600: open fd = 3 2024-04-23T18:43:49.900 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:49.975 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.009 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:50.084 INFO:tasks.workunit.client.0.smithi033.stdout:598/600: open fd = 3 2024-04-23T18:43:50.134 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.192 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.242 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:50.292 INFO:tasks.workunit.client.0.smithi033.stdout:599/600: open fd = 3 2024-04-23T18:43:50.351 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.401 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.451 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:50.501 INFO:tasks.workunit.client.0.smithi033.stdout:600/600: open fd = 3 2024-04-23T18:43:50.579 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.626 INFO:tasks.workunit.client.0.smithi033.stdout:write ret = 32 2024-04-23T18:43:50.676 INFO:tasks.workunit.client.0.smithi033.stdout:pread ret = 64 2024-04-23T18:43:50.727 INFO:tasks.workunit.client.0.smithi033.stderr:+ echo OK 2024-04-23T18:43:50.778 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:42 smithi122 ceph-mon[32546]: pgmap v547: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:50.826 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:44 smithi122 ceph-mon[32546]: pgmap v548: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:50.876 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:46 smithi122 ceph-mon[32546]: pgmap v549: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:50.926 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:48 smithi122 ceph-mon[32546]: pgmap v550: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:43:50.976 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:40 smithi106 ceph-mon[32716]: pgmap v546: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:51.026 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:42 smithi106 ceph-mon[32716]: pgmap v547: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:51.134 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:44 smithi106 ceph-mon[32716]: pgmap v548: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:43:51.185 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:46 smithi106 ceph-mon[32716]: pgmap v549: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:43:51.235 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:48 smithi106 ceph-mon[32716]: pgmap v550: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:43:51.285 INFO:tasks.workunit.client.0.smithi033.stdout:OK 2024-04-23T18:43:51.335 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-23T18:43:51.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-23T18:43:51.435 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:46 smithi033 ceph-mon[29274]: pgmap v484: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-23T18:43:51.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:50 smithi106 ceph-mon[32716]: pgmap v551: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:43:51.535 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:50 smithi122 ceph-mon[32546]: pgmap v551: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:43:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.257+0000 7fc658b95640 1 -- 172.21.15.33:0/2238235485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65410c5d0 msgr2=0x7fc65410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:51.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.257+0000 7fc658b95640 1 --2- 172.21.15.33:0/2238235485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65410c5d0 0x7fc65410ea00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc64c009ca0 tx=0x7fc64c02a950 comp rx=0 tx=0).stop 2024-04-23T18:43:51.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.257+0000 7fc658b95640 1 -- 172.21.15.33:0/2238235485 shutdown_connections 2024-04-23T18:43:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.257+0000 7fc658b95640 1 --2- 172.21.15.33:0/2238235485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc65410c5d0 0x7fc65410ea00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:51.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 --2- 172.21.15.33:0/2238235485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc654103b20 0x7fc65410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 --2- 172.21.15.33:0/2238235485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 0x7fc6541035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:52.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 -- 172.21.15.33:0/2238235485 >> 172.21.15.33:0/2238235485 conn(0x7fc6540771d0 msgr2=0x7fc6540775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:52.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 -- 172.21.15.33:0/2238235485 shutdown_connections 2024-04-23T18:43:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 -- 172.21.15.33:0/2238235485 wait complete. 2024-04-23T18:43:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 Processor -- start 2024-04-23T18:43:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.258+0000 7fc658b95640 1 -- start start 2024-04-23T18:43:52.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 0x7fc654198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc65410c5d0 0x7fc65419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541121f0 con 0x7fc654103b20 2024-04-23T18:43:52.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc654112070 con 0x7fc65410c5d0 2024-04-23T18:43:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc658b95640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc654112370 con 0x7fc654103200 2024-04-23T18:43:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:52.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38742/0 (socket says 172.21.15.33:38742) 2024-04-23T18:43:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 -- 172.21.15.33:0/1031056120 learned_addr learned my addr 172.21.15.33:0/1031056120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc65259c640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 0x7fc654198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:52.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc652d9d640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc65410c5d0 0x7fc65419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 -- 172.21.15.33:0/1031056120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 msgr2=0x7fc654198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 0x7fc654198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 -- 172.21.15.33:0/1031056120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc65410c5d0 msgr2=0x7fc65419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc65410c5d0 0x7fc65419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 -- 172.21.15.33:0/1031056120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65419d630 con 0x7fc654103b20 2024-04-23T18:43:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.259+0000 7fc651d9b640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc64800e9f0 tx=0x7fc64800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.260+0000 7fc6437fe640 1 -- 172.21.15.33:0/1031056120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64800cd80 con 0x7fc654103b20 2024-04-23T18:43:53.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.260+0000 7fc6437fe640 1 -- 172.21.15.33:0/1031056120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6480108e0 con 0x7fc654103b20 2024-04-23T18:43:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.260+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65419d920 con 0x7fc654103b20 2024-04-23T18:43:53.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.260+0000 7fc6437fe640 1 -- 172.21.15.33:0/1031056120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648018980 con 0x7fc654103b20 2024-04-23T18:43:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.260+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6541a51b0 con 0x7fc654103b20 2024-04-23T18:43:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc6437fe640 1 -- 172.21.15.33:0/1031056120 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc648010430 con 0x7fc654103b20 2024-04-23T18:43:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 Processor -- start 2024-04-23T18:43:53.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 -- start start 2024-04-23T18:43:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759235a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c7592402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:53.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c75919a970 con 0x55c759240800 2024-04-23T18:43:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c7591374a0 con 0x55c759235f40 2024-04-23T18:43:53.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc65b984740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c759196930 con 0x55c759235620 2024-04-23T18:43:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.263+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:53.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38756/0 (socket says 172.21.15.33:38756) 2024-04-23T18:43:53.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759235a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:53.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 -- 172.21.15.33:0/573865729 learned_addr learned my addr 172.21.15.33:0/573865729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:54.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc65259c640 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c7592402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc6437fe640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 0x7fc63c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 -- 172.21.15.33:0/573865729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 msgr2=0x55c759235a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759235a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 -- 172.21.15.33:0/573865729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 msgr2=0x55c7592402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c7592402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 -- 172.21.15.33:0/573865729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c758ead210 con 0x55c759240800 2024-04-23T18:43:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc652d9d640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 0x7fc63c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc6437fe640 1 -- 172.21.15.33:0/1031056120 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fc6480d09e0 con 0x7fc654103b20 2024-04-23T18:43:54.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc651d9b640 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ce090 tx=0x7fc648067190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75282a296b5a34b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:54.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc652d9d640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 0x7fc63c0a6300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005e30 tx=0x7fc64c03c040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.264+0000 7fc640ff9640 1 -- 172.21.15.33:0/573865729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64809c560 con 0x55c759240800 2024-04-23T18:43:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc640ff9640 1 -- 172.21.15.33:0/573865729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648026050 con 0x55c759240800 2024-04-23T18:43:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc640ff9640 1 -- 172.21.15.33:0/573865729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64809c380 con 0x55c759240800 2024-04-23T18:43:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 -- 172.21.15.33:0/573865729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 msgr2=0x55c759242bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc6480ce090 tx=0x7fc648067190 comp rx=0 tx=0).stop 2024-04-23T18:43:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 -- 172.21.15.33:0/573865729 shutdown_connections 2024-04-23T18:43:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c759242bf0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c7592402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 --2- 172.21.15.33:0/573865729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759235a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 -- 172.21.15.33:0/573865729 >> 172.21.15.33:0/573865729 conn(0x55c759231b30 msgr2=0x55c759231f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 -- 172.21.15.33:0/573865729 shutdown_connections 2024-04-23T18:43:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.265+0000 7fc65b984740 1 -- 172.21.15.33:0/573865729 wait complete. 2024-04-23T18:43:55.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 -- start start 2024-04-23T18:43:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 Processor -- start 2024-04-23T18:43:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759240100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:55.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75923d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c759243370 con 0x55c759240800 2024-04-23T18:43:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c7591350b0 con 0x55c759235f40 2024-04-23T18:43:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7592434f0 con 0x55c759235620 2024-04-23T18:43:55.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65b984740 10 client.0 _ll_register_callbacks cb 0x55c759122d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:43:55.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38760/0 (socket says 172.21.15.33:38760) 2024-04-23T18:43:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc652d9d640 1 -- 172.21.15.33:0/2748147736 learned_addr learned my addr 172.21.15.33:0/2748147736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc651d9b640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75923d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.266+0000 7fc65259c640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759240100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 -- 172.21.15.33:0/2748147736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 msgr2=0x55c759240100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:56.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759240100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 -- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 msgr2=0x55c75923d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75923d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 -- 172.21.15.33:0/2748147736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c758f79620 con 0x55c759240800 2024-04-23T18:43:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc64c035000 tx=0x7fc64c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eb7a1dd5900567 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:43:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c034040 con 0x55c759240800 2024-04-23T18:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fc64c003d80 con 0x55c759240800 2024-04-23T18:43:56.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc648026050 con 0x55c759240800 2024-04-23T18:43:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc64c004050 con 0x55c759240800 2024-04-23T18:43:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc65b984740 10 client.17334 fetch_fsmap learned FSMap version 37 2024-04-23T18:43:56.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c759253690 con 0x55c759240800 2024-04-23T18:43:56.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.267+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648026050 con 0x55c759240800 2024-04-23T18:43:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fc64c030940 con 0x55c759240800 2024-04-23T18:43:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 10 client.17334 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:43:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:43:57.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 20 client.17334 populate_metadata read hostname 'smithi033' 2024-04-23T18:43:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c759254be0 0x55c759256ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 4 client.17334 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:43:57.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7591a8390 con 0x55c759254be0 2024-04-23T18:43:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65259c640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c759254be0 0x55c759256ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc639ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:57.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.268+0000 7fc65259c640 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c759254be0 0x55c759256ff0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4184a129cf03a7b server_cookie=605b4d2a12e27739 in_seq=0 out_seq=0 2024-04-23T18:43:57.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc639ffb640 1 -- 172.21.15.33:0/2748147736 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7591a8390 con 0x55c759254be0 2024-04-23T18:43:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc639ffb640 10 client.17334 handle_command_reply: tid=0 2024-04-23T18:43:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 client.17334 shutdown 2024-04-23T18:43:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 msgr2=0x55c75923d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc64c035000 tx=0x7fc64c005c50 comp rx=0 tx=0).stop 2024-04-23T18:43:57.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 shutdown_connections 2024-04-23T18:43:57.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c759254be0 0x55c759256ff0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75923d930 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc639ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:43:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75923d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 --2- 172.21.15.33:0/2748147736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759240100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 >> 172.21.15.33:0/2748147736 conn(0x55c759231b30 msgr2=0x55c7592378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:57.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 shutdown_connections 2024-04-23T18:43:57.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 1 -- 172.21.15.33:0/2748147736 wait complete. 2024-04-23T18:43:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.269+0000 7fc65b984740 20 client.17334 trim_cache size 0 max 16384 2024-04-23T18:43:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.284+0000 7fc65b984740 1 Processor -- start 2024-04-23T18:43:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 -- start start 2024-04-23T18:43:58.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759230b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:58.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759240800 0x55c75925ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:58.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541ad3a0 con 0x55c759235f40 2024-04-23T18:43:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541ab190 con 0x55c759240800 2024-04-23T18:43:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65b984740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541accd0 con 0x55c759235620 2024-04-23T18:43:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38772/0 (socket says 172.21.15.33:38772) 2024-04-23T18:43:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 -- 172.21.15.33:0/4270529945 learned_addr learned my addr 172.21.15.33:0/4270529945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:43:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc652d9d640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759240800 0x55c75925ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65259c640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759230b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:43:58.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 -- 172.21.15.33:0/4270529945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 msgr2=0x55c759230b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759230b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 -- 172.21.15.33:0/4270529945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759240800 msgr2=0x55c75925ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:43:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759240800 0x55c75925ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:58.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 -- 172.21.15.33:0/4270529945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c759235d70 con 0x55c759235f40 2024-04-23T18:43:58.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc65259c640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759230b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:43:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc651d9b640 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc64809acd0 tx=0x7fc648000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54997414fafa8b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:43:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc63bfff640 1 -- 172.21.15.33:0/4270529945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64809c560 con 0x55c759235f40 2024-04-23T18:43:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.285+0000 7fc63bfff640 1 -- 172.21.15.33:0/4270529945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc648010720 con 0x55c759235f40 2024-04-23T18:43:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc63bfff640 1 -- 172.21.15.33:0/4270529945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc648010a90 con 0x55c759235f40 2024-04-23T18:43:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- 172.21.15.33:0/4270529945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 msgr2=0x55c75925c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:43:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc64809acd0 tx=0x7fc648000bc0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- 172.21.15.33:0/4270529945 shutdown_connections 2024-04-23T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759240800 0x55c75925ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759235f40 0x55c75925c900 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 --2- 172.21.15.33:0/4270529945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c759230b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:43:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- 172.21.15.33:0/4270529945 >> 172.21.15.33:0/4270529945 conn(0x55c759231b30 msgr2=0x55c759230420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:43:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- 172.21.15.33:0/4270529945 shutdown_connections 2024-04-23T18:43:59.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- 172.21.15.33:0/4270529945 wait complete. 2024-04-23T18:43:59.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 -- start start 2024-04-23T18:43:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.286+0000 7fc65b984740 1 Processor -- start 2024-04-23T18:43:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c75924a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75924aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:43:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541abf30 con 0x55c759240800 2024-04-23T18:44:00.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541ab860 con 0x55c759235f40 2024-04-23T18:44:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6541ac600 con 0x55c759235620 2024-04-23T18:44:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 10 client.0 _ll_register_callbacks cb 0x7fc64c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38776/0 (socket says 172.21.15.33:38776) 2024-04-23T18:44:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 -- 172.21.15.33:0/2908170251 learned_addr learned my addr 172.21.15.33:0/2908170251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc651d9b640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75924aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 -- 172.21.15.33:0/2908170251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 msgr2=0x55c75924a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65259c640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c75924a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c75924a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 -- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 msgr2=0x55c75924aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75924aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 -- 172.21.15.33:0/2908170251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7592ba180 con 0x55c759240800 2024-04-23T18:44:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc652d9d640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc64c030d80 tx=0x7fc64c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1586c59b3014092f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc639ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:01.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64c004600 con 0x55c759240800 2024-04-23T18:44:01.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7fc64c0047a0 con 0x55c759240800 2024-04-23T18:44:01.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.287+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7591a2200 con 0x55c759240800 2024-04-23T18:44:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc64c030440 con 0x55c759240800 2024-04-23T18:44:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 10 client.17340 fetch_fsmap learned FSMap version 37 2024-04-23T18:44:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7592bb310 con 0x55c759240800 2024-04-23T18:44:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c7591a2200 con 0x55c759240800 2024-04-23T18:44:01.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7fc64c061030 con 0x55c759240800 2024-04-23T18:44:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 10 client.17340 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:44:01.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 20 client.17340 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c7592bba90 0x55c7592bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 4 client.17340 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c75911e5f0 con 0x55c7592bba90 2024-04-23T18:44:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.288+0000 7fc65259c640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c7592bba90 0x55c7592bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.289+0000 7fc65259c640 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c7592bba90 0x55c7592bdeb0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a556b30028ab74c9 server_cookie=fd626e291cd6ea8b in_seq=0 out_seq=0 2024-04-23T18:44:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.289+0000 7fc639ffb640 10 client.17340 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:01.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.296+0000 7fc639ffb640 1 -- 172.21.15.33:0/2908170251 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c75911e5f0 con 0x55c7592bba90 2024-04-23T18:44:01.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.296+0000 7fc639ffb640 10 client.17340 handle_command_reply: tid=0 2024-04-23T18:44:01.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.296+0000 7fc65b984740 1 client.17340 shutdown 2024-04-23T18:44:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 msgr2=0x55c75924afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc64c030d80 tx=0x7fc64c003c00 comp rx=0 tx=0).stop 2024-04-23T18:44:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 shutdown_connections 2024-04-23T18:44:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c7592bba90 0x55c7592bdeb0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c759240800 0x55c75924afb0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:02.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c759235f40 0x55c75924aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:02.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc639ffb640 0 client.17340 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 --2- 172.21.15.33:0/2908170251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c759235620 0x55c75924a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:02.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 >> 172.21.15.33:0/2908170251 conn(0x55c759231b30 msgr2=0x55c759230cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:02.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 shutdown_connections 2024-04-23T18:44:02.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 1 -- 172.21.15.33:0/2908170251 wait complete. 2024-04-23T18:44:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc65b984740 20 client.17340 trim_cache size 0 max 16384 2024-04-23T18:44:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:44:02.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:44:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:44:02.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:44:02.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 msgr2=0x7fc63c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 0x7fc63c0a6300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc64c005e30 tx=0x7fc64c03c040 comp rx=0 tx=0).stop 2024-04-23T18:44:02.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 msgr2=0x7fc654198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc64800e9f0 tx=0x7fc64800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:44:02.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 shutdown_connections 2024-04-23T18:44:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc63c0a3e40 0x7fc63c0a6300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc65410c5d0 0x7fc65419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc654103b20 0x7fc654198bd0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 --2- 172.21.15.33:0/1031056120 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc654103200 0x7fc654198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:03.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.297+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 >> 172.21.15.33:0/1031056120 conn(0x7fc6540771d0 msgr2=0x7fc6540fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:03.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.298+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 shutdown_connections 2024-04-23T18:44:03.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:41:48.298+0000 7fc658b95640 1 -- 172.21.15.33:0/1031056120 wait complete. 2024-04-23T18:44:03.402 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:51 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:03.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:52 smithi122 ceph-mon[32546]: pgmap v552: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:03.500 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:03.550 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:03.606 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:52 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:44:03.731 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:54 smithi122 ceph-mon[32546]: pgmap v553: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:03.834 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:56 smithi122 ceph-mon[32546]: pgmap v554: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:03.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:43:58 smithi122 ceph-mon[32546]: pgmap v555: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:03.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:00 smithi122 ceph-mon[32546]: pgmap v556: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:04.009 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:51 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:04.059 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:52 smithi106 ceph-mon[32716]: pgmap v552: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:04.109 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:04.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:04.209 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:52 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:44:04.259 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:54 smithi106 ceph-mon[32716]: pgmap v553: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:04.309 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:56 smithi106 ceph-mon[32716]: pgmap v554: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:04.402 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:43:58 smithi106 ceph-mon[32716]: pgmap v555: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:04.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:00 smithi106 ceph-mon[32716]: pgmap v556: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:04.543 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:02 smithi106 ceph-mon[32716]: pgmap v557: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:04.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:48 smithi033 ceph-mon[29274]: pgmap v485: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:44:04.643 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:49 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:44:04.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:50 smithi033 ceph-mon[29274]: pgmap v486: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 12 op/s 2024-04-23T18:44:04.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:50 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:44:04.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:50 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:44:04.843 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:50 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:44:04.893 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:52 smithi033 ceph-mon[29274]: pgmap v487: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-04-23T18:44:04.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:54 smithi033 ceph-mon[29274]: pgmap v488: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 18 KiB/s wr, 12 op/s 2024-04-23T18:44:05.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:56 smithi033 ceph-mon[29274]: pgmap v489: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-23T18:44:05.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:41:58 smithi033 ceph-mon[29274]: pgmap v490: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:44:05.193 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:00 smithi033 ceph-mon[29274]: pgmap v491: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.8 KiB/s wr, 10 op/s 2024-04-23T18:44:05.243 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:02 smithi122 ceph-mon[32546]: pgmap v557: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:05.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.396+0000 7f01baee8640 1 -- 172.21.15.33:0/2811351621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b4102b70 msgr2=0x7f01b4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- 172.21.15.33:0/2811351621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b4102b70 0x7f01b4106bb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a802a410 comp rx=0 tx=0).stop 2024-04-23T18:44:05.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 -- 172.21.15.33:0/2811351621 shutdown_connections 2024-04-23T18:44:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- 172.21.15.33:0/2811351621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- 172.21.15.33:0/2811351621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b4102b70 0x7f01b4106bb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- 172.21.15.33:0/2811351621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b41021c0 0x7f01b41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 -- 172.21.15.33:0/2811351621 >> 172.21.15.33:0/2811351621 conn(0x7f01b40fda20 msgr2=0x7f01b40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 -- 172.21.15.33:0/2811351621 shutdown_connections 2024-04-23T18:44:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 -- 172.21.15.33:0/2811351621 wait complete. 2024-04-23T18:44:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 Processor -- start 2024-04-23T18:44:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 -- start start 2024-04-23T18:44:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.397+0000 7f01baee8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 0x7f01b41a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01baee8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4112200 con 0x7f01b41021c0 2024-04-23T18:44:06.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01baee8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4112080 con 0x7f01b4107290 2024-04-23T18:44:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01baee8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01b4112380 con 0x7f01b4074cd0 2024-04-23T18:44:06.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57294/0 (socket says 172.21.15.33:57294) 2024-04-23T18:44:06.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 -- 172.21.15.33:0/3984874976 learned_addr learned my addr 172.21.15.33:0/3984874976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b3fff640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b9485640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 0x7f01b41a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 -- 172.21.15.33:0/3984874976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 msgr2=0x7f01b41a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 0x7f01b41a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 -- 172.21.15.33:0/3984874976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 msgr2=0x7f01b4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 -- 172.21.15.33:0/3984874976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b41a6f10 con 0x7f01b41021c0 2024-04-23T18:44:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b3fff640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:06.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b9485640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 0x7f01b41a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:06.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c920 tx=0x7f01a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b1ffb640 1 -- 172.21.15.33:0/3984874976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4007c70 con 0x7f01b41021c0 2024-04-23T18:44:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b1ffb640 1 -- 172.21.15.33:0/3984874976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a4007e10 con 0x7f01b41021c0 2024-04-23T18:44:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b41a7200 con 0x7f01b41021c0 2024-04-23T18:44:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.398+0000 7f01b1ffb640 1 -- 172.21.15.33:0/3984874976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4018820 con 0x7f01b41021c0 2024-04-23T18:44:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.399+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b41a77a0 con 0x7f01b41021c0 2024-04-23T18:44:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.401+0000 7f01b1ffb640 1 -- 172.21.15.33:0/3984874976 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f01a400f450 con 0x7f01b41021c0 2024-04-23T18:44:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.401+0000 7f01bdbff740 1 Processor -- start 2024-04-23T18:44:07.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.401+0000 7f01bdbff740 1 -- start start 2024-04-23T18:44:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.401+0000 7f01b1ffb640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 0x7f019c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.401+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be7f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:07.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b9485640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 0x7f019c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be801bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01bdbff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9be759970 con 0x55e9be7f4f40 2024-04-23T18:44:07.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01bdbff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e9be6f64a0 con 0x55e9be7f4620 2024-04-23T18:44:07.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01bdbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e9be755930 con 0x55e9be7ff800 2024-04-23T18:44:07.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b1ffb640 1 -- 172.21.15.33:0/3984874976 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f01a40d0980 con 0x7f01b41021c0 2024-04-23T18:44:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57296/0 (socket says 172.21.15.33:57296) 2024-04-23T18:44:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 -- 172.21.15.33:0/2043330385 learned_addr learned my addr 172.21.15.33:0/2043330385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b8c84640 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be7f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b9485640 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be801bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:08.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 -- 172.21.15.33:0/2043330385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 msgr2=0x55e9be801bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be801bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 -- 172.21.15.33:0/2043330385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 msgr2=0x55e9be7f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be7f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 -- 172.21.15.33:0/2043330385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9be46c210 con 0x55e9be7f4f40 2024-04-23T18:44:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b9485640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 0x7f019c0a6330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac00e6c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f01b3fff640 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f01a802a930 tx=0x7f01a8004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977cb6401b51d56e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:08.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.402+0000 7f019affd640 1 -- 172.21.15.33:0/2043330385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a803f070 con 0x55e9be7f4f40 2024-04-23T18:44:08.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f019affd640 1 -- 172.21.15.33:0/2043330385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80054f0 con 0x55e9be7f4f40 2024-04-23T18:44:08.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f019affd640 1 -- 172.21.15.33:0/2043330385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8035030 con 0x55e9be7f4f40 2024-04-23T18:44:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 -- 172.21.15.33:0/2043330385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 msgr2=0x55e9be7ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:08.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f01a802a930 tx=0x7f01a8004220 comp rx=0 tx=0).stop 2024-04-23T18:44:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 -- 172.21.15.33:0/2043330385 shutdown_connections 2024-04-23T18:44:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be801bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be7ff2c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2043330385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be7f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 -- 172.21.15.33:0/2043330385 >> 172.21.15.33:0/2043330385 conn(0x55e9be7f0b30 msgr2=0x55e9be7f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:09.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 -- 172.21.15.33:0/2043330385 shutdown_connections 2024-04-23T18:44:09.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.403+0000 7f01bdbff740 1 -- 172.21.15.33:0/2043330385 wait complete. 2024-04-23T18:44:09.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 -- start start 2024-04-23T18:44:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 Processor -- start 2024-04-23T18:44:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be7ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4f40 0x55e9be7fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e9be8023d0 con 0x55e9be7ff800 2024-04-23T18:44:09.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e9be6f40b0 con 0x55e9be7f4f40 2024-04-23T18:44:09.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e9be802550 con 0x55e9be7f4620 2024-04-23T18:44:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01bdbff740 10 client.0 _ll_register_callbacks cb 0x55e9be6e1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01b9485640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:09.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01b9485640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57310/0 (socket says 172.21.15.33:57310) 2024-04-23T18:44:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01b9485640 1 -- 172.21.15.33:0/2167072486 learned_addr learned my addr 172.21.15.33:0/2167072486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.404+0000 7f01b3fff640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4f40 0x55e9be7fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b8c84640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be7ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:10.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 -- 172.21.15.33:0/2167072486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 msgr2=0x55e9be7ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:10.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be7ff100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 -- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4f40 msgr2=0x55e9be7fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:10.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4f40 0x55e9be7fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:10.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 -- 172.21.15.33:0/2167072486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9be538620 con 0x55e9be7ff800 2024-04-23T18:44:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01b9485640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f01ac012b90 tx=0x7f01ac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c74cc2a7e3d22a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f0193fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac011d10 con 0x55e9be7ff800 2024-04-23T18:44:10.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f01ac003970 con 0x55e9be7ff800 2024-04-23T18:44:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a80054f0 con 0x55e9be7ff800 2024-04-23T18:44:10.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f01ac003b70 con 0x55e9be7ff800 2024-04-23T18:44:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01bdbff740 10 client.17352 fetch_fsmap learned FSMap version 37 2024-04-23T18:44:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e9be80e300 con 0x55e9be7ff800 2024-04-23T18:44:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.405+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80054f0 con 0x55e9be7ff800 2024-04-23T18:44:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f01ac00f040 con 0x55e9be7ff800 2024-04-23T18:44:11.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 10 client.17352 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:44:11.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:11.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 20 client.17352 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be80f7c0 0x55e9be811bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:11.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 4 client.17352 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:11.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9be767390 con 0x55e9be80f7c0 2024-04-23T18:44:11.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01b8c84640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be80f7c0 0x55e9be811bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:11.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f0193fff640 10 client.17352 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.406+0000 7f01b8c84640 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be80f7c0 0x55e9be811bd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93736474abf1258 server_cookie=d85655eab6ef856 in_seq=0 out_seq=0 2024-04-23T18:44:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f0193fff640 1 -- 172.21.15.33:0/2167072486 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9be767390 con 0x55e9be80f7c0 2024-04-23T18:44:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f0193fff640 10 client.17352 handle_command_reply: tid=0 2024-04-23T18:44:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 client.17352 shutdown 2024-04-23T18:44:11.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 msgr2=0x55e9be7fc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f01ac012b90 tx=0x7f01ac004ab0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 shutdown_connections 2024-04-23T18:44:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be80f7c0 0x55e9be811bd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7ff800 0x55e9be7fc930 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4f40 0x55e9be7fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f0193fff640 0 client.17352 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 --2- 172.21.15.33:0/2167072486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be7ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:11.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 >> 172.21.15.33:0/2167072486 conn(0x55e9be7f0b30 msgr2=0x55e9be7f68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 shutdown_connections 2024-04-23T18:44:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 1 -- 172.21.15.33:0/2167072486 wait complete. 2024-04-23T18:44:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.407+0000 7f01bdbff740 20 client.17352 trim_cache size 0 max 16384 2024-04-23T18:44:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.422+0000 7f01bdbff740 1 Processor -- start 2024-04-23T18:44:12.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.422+0000 7f01bdbff740 1 -- start start 2024-04-23T18:44:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be80b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 0x55e9be817a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41b1790 con 0x55e9be7f4f40 2024-04-23T18:44:12.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41af580 con 0x55e9be7ff800 2024-04-23T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01bdbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41b10c0 con 0x55e9be7f4620 2024-04-23T18:44:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57326/0 (socket says 172.21.15.33:57326) 2024-04-23T18:44:12.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 -- 172.21.15.33:0/1058303552 learned_addr learned my addr 172.21.15.33:0/1058303552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b8c84640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be80b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:12.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b9485640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 0x55e9be817a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 -- 172.21.15.33:0/1058303552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 msgr2=0x55e9be80b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:12.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be80b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 -- 172.21.15.33:0/1058303552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 msgr2=0x55e9be817a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 0x55e9be817a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 -- 172.21.15.33:0/1058303552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9be7fcf30 con 0x55e9be7f4f40 2024-04-23T18:44:13.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b8c84640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be80b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b9485640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 0x55e9be817a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f01b3fff640 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005ab0 tx=0x7f01a80045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b596a35c95cd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f019a7fc640 1 -- 172.21.15.33:0/1058303552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a803f040 con 0x55e9be7f4f40 2024-04-23T18:44:13.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f019a7fc640 1 -- 172.21.15.33:0/1058303552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80035f0 con 0x55e9be7f4f40 2024-04-23T18:44:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.423+0000 7f019a7fc640 1 -- 172.21.15.33:0/1058303552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8035040 con 0x55e9be7f4f40 2024-04-23T18:44:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- 172.21.15.33:0/1058303552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 msgr2=0x55e9be8174e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005ab0 tx=0x7f01a80045a0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- 172.21.15.33:0/1058303552 shutdown_connections 2024-04-23T18:44:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7ff800 0x55e9be817a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be8174e0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 --2- 172.21.15.33:0/1058303552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7f4620 0x55e9be80b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- 172.21.15.33:0/1058303552 >> 172.21.15.33:0/1058303552 conn(0x55e9be7f0b30 msgr2=0x55e9be7ef420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- 172.21.15.33:0/1058303552 shutdown_connections 2024-04-23T18:44:13.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- 172.21.15.33:0/1058303552 wait complete. 2024-04-23T18:44:13.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 -- start start 2024-04-23T18:44:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.424+0000 7f01bdbff740 1 Processor -- start 2024-04-23T18:44:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be807a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:13.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be805340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41b0320 con 0x55e9be7f4f40 2024-04-23T18:44:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41afc50 con 0x55e9be7f4620 2024-04-23T18:44:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01b41b09f0 con 0x55e9be7ff800 2024-04-23T18:44:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01bdbff740 10 client.0 _ll_register_callbacks cb 0x7f01ac012860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57334/0 (socket says 172.21.15.33:57334) 2024-04-23T18:44:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 -- 172.21.15.33:0/3888306030 learned_addr learned my addr 172.21.15.33:0/3888306030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be807a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b9485640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be805340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 -- 172.21.15.33:0/3888306030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 msgr2=0x55e9be805340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:14.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be805340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 -- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 msgr2=0x55e9be807a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:14.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be807a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.425+0000 7f01b3fff640 1 -- 172.21.15.33:0/3888306030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9be805a30 con 0x55e9be7f4f40 2024-04-23T18:44:14.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01b3fff640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d79445413076b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:14.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8035030 con 0x55e9be7f4f40 2024-04-23T18:44:14.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8542+0+0 (secure 0 0 0) 0x7f01a8004a10 con 0x55e9be7f4f40 2024-04-23T18:44:14.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01a80035f0 con 0x55e9be7f4f40 2024-04-23T18:44:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f01a803f040 con 0x55e9be7f4f40 2024-04-23T18:44:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 10 client.17358 fetch_fsmap learned FSMap version 37 2024-04-23T18:44:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e9be875be0 con 0x55e9be7f4f40 2024-04-23T18:44:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80035f0 con 0x55e9be7f4f40 2024-04-23T18:44:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 7749+0+0 (secure 0 0 0) 0x7f01a80053c0 con 0x55e9be7f4f40 2024-04-23T18:44:15.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 10 client.17358 fetch_fsmap finished waiting for FSMap version 37 2024-04-23T18:44:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 20 client.17358 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be875f00 0x55e9be8782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 4 client.17358 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.426+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9be7fccc0 con 0x55e9be875f00 2024-04-23T18:44:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.427+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be875f00 0x55e9be8782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.427+0000 7f01b8c84640 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be875f00 0x55e9be8782f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44336fc289102247 server_cookie=b74f348204c67ccc in_seq=0 out_seq=0 2024-04-23T18:44:15.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:01.427+0000 7f0193fff640 10 client.17358 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f0193fff640 1 -- 172.21.15.33:0/3888306030 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9be7fccc0 con 0x55e9be875f00 2024-04-23T18:44:15.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f0193fff640 10 client.17358 handle_command_reply: tid=0 2024-04-23T18:44:15.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 client.17358 shutdown 2024-04-23T18:44:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 msgr2=0x55e9be804e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009900 tx=0x7f01a8000bc0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 shutdown_connections 2024-04-23T18:44:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e9be875f00 0x55e9be8782f0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e9be7ff800 0x55e9be805340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e9be7f4f40 0x55e9be804e00 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 --2- 172.21.15.33:0/3888306030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e9be7f4620 0x55e9be807a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:16.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f0193fff640 0 client.17358 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.182+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 >> 172.21.15.33:0/3888306030 conn(0x55e9be7f0b30 msgr2=0x55e9be7ef420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:16.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 shutdown_connections 2024-04-23T18:44:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01bdbff740 1 -- 172.21.15.33:0/3888306030 wait complete. 2024-04-23T18:44:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01bdbff740 20 client.17358 trim_cache size 0 max 16384 2024-04-23T18:44:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:44:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:44:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:44:16.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:44:16.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 msgr2=0x7f019c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:17.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 0x7f019c0a6330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f01ac005fd0 tx=0x7f01ac00e6c0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 msgr2=0x7f01b4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.183+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c920 tx=0x7f01a400cde0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 shutdown_connections 2024-04-23T18:44:17.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f019c0a3e70 0x7f019c0a6330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01b4074cd0 0x7f01b41a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b4107290 0x7f01b4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 --2- 172.21.15.33:0/3984874976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01b41021c0 0x7f01b4074250 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:17.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 >> 172.21.15.33:0/3984874976 conn(0x7f01b40fda20 msgr2=0x7f01b40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:17.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 shutdown_connections 2024-04-23T18:44:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:03.184+0000 7f01baee8640 1 -- 172.21.15.33:0/3984874976 wait complete. 2024-04-23T18:44:17.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:02 smithi033 ceph-mon[29274]: pgmap v492: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-23T18:44:17.869 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:04 smithi122 ceph-mon[32546]: pgmap v558: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:17.941 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:04 smithi122 ceph-mon[32546]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:44:18.000 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:06 smithi122 ceph-mon[32546]: pgmap v560: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.051 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:08 smithi122 ceph-mon[32546]: pgmap v561: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.101 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:10 smithi122 ceph-mon[32546]: pgmap v562: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:10 smithi122 ceph-mon[32546]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:44:18.201 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:12 smithi122 ceph-mon[32546]: pgmap v564: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-23T18:44:18.251 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:14 smithi122 ceph-mon[32546]: pgmap v565: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T18:44:18.301 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:15 smithi122 ceph-mon[32546]: daemon mds.d restarted 2024-04-23T18:44:18.351 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:15 smithi122 ceph-mon[32546]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:44:18.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:15 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/1425237468,v1:172.21.15.33:6837/1425237468] up:boot 2024-04-23T18:44:18.493 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:15 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:44:18.543 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:44:18.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:16 smithi122 ceph-mon[32546]: pgmap v566: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:44:18.626 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:04 smithi106 ceph-mon[32716]: pgmap v558: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:18.668 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:04 smithi106 ceph-mon[32716]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:44:18.701 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:06 smithi106 ceph-mon[32716]: pgmap v560: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:08 smithi106 ceph-mon[32716]: pgmap v561: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:10 smithi106 ceph-mon[32716]: pgmap v562: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:44:18.809 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:10 smithi106 ceph-mon[32716]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:44:18.843 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:12 smithi106 ceph-mon[32716]: pgmap v564: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-23T18:44:18.918 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:14 smithi106 ceph-mon[32716]: pgmap v565: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T18:44:18.982 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:15 smithi106 ceph-mon[32716]: daemon mds.d restarted 2024-04-23T18:44:19.074 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:15 smithi106 ceph-mon[32716]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:44:19.147 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:15 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/1425237468,v1:172.21.15.33:6837/1425237468] up:boot 2024-04-23T18:44:19.184 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:15 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:44:19.218 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:44:19.251 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:16 smithi106 ceph-mon[32716]: pgmap v566: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:44:19.285 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:44:19.318 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: in thread 7ff2d5569700 thread_name:md_log_replay 2024-04-23T18:44:19.351 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:44:19.385 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2e692ad20] 2024-04-23T18:44:19.485 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5645d356438f] 2024-04-23T18:44:19.518 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5645d3567b23] 2024-04-23T18:44:19.551 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 4: (MDLog::_replay_thread()+0x617) [0x5645d35c2647] 2024-04-23T18:44:19.585 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5645d35cbd31] 2024-04-23T18:44:19.618 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2e69201ca] 2024-04-23T18:44:19.652 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 7: clone() 2024-04-23T18:44:19.685 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:42:04.056+0000 7ff2d5569700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:44:19.718 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: in thread 7ff2d5569700 thread_name:md_log_replay 2024-04-23T18:44:19.752 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:19.787 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:44:19.933 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2e692ad20] 2024-04-23T18:44:19.968 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5645d356438f] 2024-04-23T18:44:20.002 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5645d3567b23] 2024-04-23T18:44:20.035 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 4: (MDLog::_replay_thread()+0x617) [0x5645d35c2647] 2024-04-23T18:44:20.069 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5645d35cbd31] 2024-04-23T18:44:20.102 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2e69201ca] 2024-04-23T18:44:20.135 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 7: clone() 2024-04-23T18:44:20.168 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:44:20.202 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:20.235 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 0> 2024-04-23T18:42:04.056+0000 7ff2d5569700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:44:20.281 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: in thread 7ff2d5569700 thread_name:md_log_replay 2024-04-23T18:44:20.346 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:20.386 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:44:20.452 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2e692ad20] 2024-04-23T18:44:20.544 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5645d356438f] 2024-04-23T18:44:20.577 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5645d3567b23] 2024-04-23T18:44:20.628 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 4: (MDLog::_replay_thread()+0x617) [0x5645d35c2647] 2024-04-23T18:44:20.661 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5645d35cbd31] 2024-04-23T18:44:20.694 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2e69201ca] 2024-04-23T18:44:20.727 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 7: clone() 2024-04-23T18:44:20.761 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:44:20.820 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:20.899 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 0> 2024-04-23T18:42:04.056+0000 7ff2d5569700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:44:20.969 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: in thread 7ff2d5569700 thread_name:md_log_replay 2024-04-23T18:44:21.003 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:21.098 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:44:21.128 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff2e692ad20] 2024-04-23T18:44:21.161 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5645d356438f] 2024-04-23T18:44:21.202 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5645d3567b23] 2024-04-23T18:44:21.253 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 4: (MDLog::_replay_thread()+0x617) [0x5645d35c2647] 2024-04-23T18:44:21.303 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5645d35cbd31] 2024-04-23T18:44:21.353 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff2e69201ca] 2024-04-23T18:44:21.424 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 7: clone() 2024-04-23T18:44:21.495 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:44:21.551 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[69918]: 2024-04-23T18:44:21.603 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 podman[92437]: 2024-04-23 18:42:04.123405304 +0000 UTC m=+0.022922135 container died 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:44:21.683 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-mon[29274]: pgmap v493: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:44:21.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:04 smithi033 ceph-mon[29274]: osdmap e112: 12 total, 12 up, 12 in 2024-04-23T18:44:21.812 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 podman[92437]: 2024-04-23 18:42:04.834296407 +0000 UTC m=+0.733813221 container remove 76a9367cdeb38b2d6b9253829e534bd28f8fb21588cc4ec35c5ffbacdffa352a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:44:21.861 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:44:21.911 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:05 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:44:21.961 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:05 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.087s CPU time. 2024-04-23T18:44:22.012 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:06 smithi033 ceph-mon[29274]: pgmap v495: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:44:22.061 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:08 smithi033 ceph-mon[29274]: pgmap v496: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:44:22.128 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:10 smithi033 ceph-mon[29274]: pgmap v497: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:44:22.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:10 smithi033 ceph-mon[29274]: osdmap e113: 12 total, 12 up, 12 in 2024-04-23T18:44:22.273 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:12 smithi033 ceph-mon[29274]: pgmap v499: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-04-23T18:44:22.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:14 smithi033 ceph-mon[29274]: pgmap v500: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-04-23T18:44:22.437 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-04-23T18:44:22.487 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:44:22.537 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Consumed 1.087s CPU time. 2024-04-23T18:44:22.587 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:44:22.637 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 podman[92533]: 2024-04-23 18:42:15.385154044 +0000 UTC m=+0.019134240 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:44:22.687 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 podman[92533]: 2024-04-23 18:42:15.526571371 +0000 UTC m=+0.160551552 container create c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T18:44:22.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mon[29274]: daemon mds.d restarted 2024-04-23T18:44:22.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mon[29274]: osdmap e114: 12 total, 12 up, 12 in 2024-04-23T18:44:22.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/2982854948,v1:172.21.15.33:6837/2982854948] up:boot 2024-04-23T18:44:22.887 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:44:22.996 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:44:23.096 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 podman[92533]: 2024-04-23 18:42:15.788258886 +0000 UTC m=+0.422239069 container init c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T18:44:23.146 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 podman[92533]: 2024-04-23 18:42:15.791065146 +0000 UTC m=+0.425045329 container start c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T18:44:23.196 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-mds[92552]: -- 172.21.15.33:0/4086344027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb1094fa40 con 0x55eb10922800 2024-04-23T18:44:23.246 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: starting mds.d at 2024-04-23T18:44:23.296 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 bash[92533]: c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 2024-04-23T18:44:23.346 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:42:15 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:44:23.398 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:18 smithi122 ceph-mon[32546]: pgmap v568: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:44:23.446 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:20 smithi122 ceph-mon[32546]: pgmap v569: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:44:23.539 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:18 smithi106 ceph-mon[32716]: pgmap v568: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:44:23.610 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:20 smithi106 ceph-mon[32716]: pgmap v569: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:44:23.696 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:22 smithi106 ceph-mon[32716]: pgmap v570: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:44:23.792 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:22 smithi122 ceph-mon[32546]: pgmap v570: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:44:23.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 -- 172.21.15.33:0/3204669432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c102b70 msgr2=0x7f636c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 --2- 172.21.15.33:0/3204669432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c102b70 0x7f636c106bb0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6358009900 tx=0x7f635802a410 comp rx=0 tx=0).stop 2024-04-23T18:44:23.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 -- 172.21.15.33:0/3204669432 shutdown_connections 2024-04-23T18:44:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 --2- 172.21.15.33:0/3204669432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c107290 0x7f636c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 --2- 172.21.15.33:0/3204669432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c102b70 0x7f636c106bb0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 --2- 172.21.15.33:0/3204669432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c1021c0 0x7f636c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 -- 172.21.15.33:0/3204669432 >> 172.21.15.33:0/3204669432 conn(0x7f636c0fda20 msgr2=0x7f636c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:24.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 -- 172.21.15.33:0/3204669432 shutdown_connections 2024-04-23T18:44:24.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.279+0000 7f6372ca3640 1 -- 172.21.15.33:0/3204669432 wait complete. 2024-04-23T18:44:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 Processor -- start 2024-04-23T18:44:24.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 -- start start 2024-04-23T18:44:24.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 0x7f636c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c19cf70 0x7f636c19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:24.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1121b0 con 0x7f636c107290 2024-04-23T18:44:24.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112030 con 0x7f636c1021c0 2024-04-23T18:44:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f6372ca3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f636c112330 con 0x7f636c19cf70 2024-04-23T18:44:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f636bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f636bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41320/0 (socket says 172.21.15.33:41320) 2024-04-23T18:44:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.280+0000 7f636bfff640 1 -- 172.21.15.33:0/346393826 learned_addr learned my addr 172.21.15.33:0/346393826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:24.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6370a3f640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 0x7f636c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6371240640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c19cf70 0x7f636c19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:24.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 -- 172.21.15.33:0/346393826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c19cf70 msgr2=0x7f636c19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c19cf70 0x7f636c19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 -- 172.21.15.33:0/346393826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 msgr2=0x7f636c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 0x7f636c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:24.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 -- 172.21.15.33:0/346393826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c19da60 con 0x7f636c107290 2024-04-23T18:44:25.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6370a3f640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 0x7f636c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f636bfff640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6358032fd0 tx=0x7f6358004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:25.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6369ffb640 1 -- 172.21.15.33:0/346393826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635803f030 con 0x7f636c107290 2024-04-23T18:44:25.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6369ffb640 1 -- 172.21.15.33:0/346393826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6358004490 con 0x7f636c107290 2024-04-23T18:44:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c19dcf0 con 0x7f636c107290 2024-04-23T18:44:25.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6369ffb640 1 -- 172.21.15.33:0/346393826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6358035070 con 0x7f636c107290 2024-04-23T18:44:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.281+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a54b0 con 0x7f636c107290 2024-04-23T18:44:25.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6369ffb640 1 -- 172.21.15.33:0/346393826 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6358042030 con 0x7f636c107290 2024-04-23T18:44:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 Processor -- start 2024-04-23T18:44:25.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 -- start start 2024-04-23T18:44:25.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a2658196d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a26581c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a26576f970 con 0x55a26580a500 2024-04-23T18:44:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a26570c4a0 con 0x55a265819c10 2024-04-23T18:44:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f63759b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a26576b930 con 0x55a26580ae20 2024-04-23T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41330/0 (socket says 172.21.15.33:41330) 2024-04-23T18:44:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 -- 172.21.15.33:0/3981581206 learned_addr learned my addr 172.21.15.33:0/3981581206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6370a3f640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a2658196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6369ffb640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 0x7f635c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f636bfff640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a26581c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:25.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 -- 172.21.15.33:0/3981581206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 msgr2=0x55a2658196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:26.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a2658196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 -- 172.21.15.33:0/3981581206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 msgr2=0x55a26581c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a26581c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.284+0000 7f6371240640 1 -- 172.21.15.33:0/3981581206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a265482210 con 0x55a26580a500 2024-04-23T18:44:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f636bfff640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a26581c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:26.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6371240640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 0x7f635c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6369ffb640 1 -- 172.21.15.33:0/346393826 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f63580f0d00 con 0x7f636c107290 2024-04-23T18:44:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6371240640 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6364010470 tx=0x7f6364010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298bcf9f0da90ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6356ffd640 1 -- 172.21.15.33:0/3981581206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364022070 con 0x55a26580a500 2024-04-23T18:44:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6356ffd640 1 -- 172.21.15.33:0/3981581206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636401ddb0 con 0x55a26580a500 2024-04-23T18:44:26.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.285+0000 7f6356ffd640 1 -- 172.21.15.33:0/3981581206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636401ccb0 con 0x55a26580a500 2024-04-23T18:44:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- 172.21.15.33:0/3981581206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 msgr2=0x55a26580a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6364010470 tx=0x7f6364010930 comp rx=0 tx=0).stop 2024-04-23T18:44:26.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f6371240640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 0x7f635c0a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f636401f000 tx=0x7f636401d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:26.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- 172.21.15.33:0/3981581206 shutdown_connections 2024-04-23T18:44:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a26581c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a2658196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 --2- 172.21.15.33:0/3981581206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a26580a8e0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- 172.21.15.33:0/3981581206 >> 172.21.15.33:0/3981581206 conn(0x55a265806b30 msgr2=0x55a265806f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:26.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- 172.21.15.33:0/3981581206 shutdown_connections 2024-04-23T18:44:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- 172.21.15.33:0/3981581206 wait complete. 2024-04-23T18:44:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 -- start start 2024-04-23T18:44:26.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.286+0000 7f63759b7740 1 Processor -- start 2024-04-23T18:44:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a265816780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a265816cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a26581c840 con 0x55a26580a500 2024-04-23T18:44:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a26570a0b0 con 0x55a265819c10 2024-04-23T18:44:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a26581c9c0 con 0x55a26580ae20 2024-04-23T18:44:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 10 client.0 _ll_register_callbacks cb 0x55a2656f7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41332/0 (socket says 172.21.15.33:41332) 2024-04-23T18:44:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 -- 172.21.15.33:0/3483104620 learned_addr learned my addr 172.21.15.33:0/3483104620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6371240640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a265816cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f636bfff640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a265816780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 -- 172.21.15.33:0/3483104620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 msgr2=0x55a265816780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a265816780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 -- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 msgr2=0x55a265816cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a265816cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:27.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 -- 172.21.15.33:0/3483104620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a26554e620 con 0x55a26580a500 2024-04-23T18:44:27.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f6370a3f640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f636000bf70 tx=0x7f63600128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9643de23aa4d0816 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f634bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636001f060 con 0x55a26580a500 2024-04-23T18:44:27.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f6360010960 con 0x55a26580a500 2024-04-23T18:44:27.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.287+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f636401ddb0 con 0x55a26580a500 2024-04-23T18:44:27.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6360010b60 con 0x55a26580a500 2024-04-23T18:44:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 10 client.17376 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a265828710 con 0x55a26580a500 2024-04-23T18:44:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f636401ddb0 con 0x55a26580a500 2024-04-23T18:44:28.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f6360019030 con 0x55a26580a500 2024-04-23T18:44:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 10 client.17376 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 20 client.17376 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265829bd0 0x55a26582bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:28.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 4 client.17376 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a26577d390 con 0x55a265829bd0 2024-04-23T18:44:28.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.288+0000 7f636bfff640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265829bd0 0x55a26582bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:28.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f636bfff640 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265829bd0 0x55a26582bfc0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8192b37f88a01b server_cookie=afb9ffbc79a4236e in_seq=0 out_seq=0 2024-04-23T18:44:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f634bfff640 10 client.17376 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f634bfff640 1 -- 172.21.15.33:0/3483104620 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a26577d390 con 0x55a265829bd0 2024-04-23T18:44:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f634bfff640 10 client.17376 handle_command_reply: tid=0 2024-04-23T18:44:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f63759b7740 1 client.17376 shutdown 2024-04-23T18:44:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 msgr2=0x55a265819490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.289+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f636000bf70 tx=0x7f63600128e0 comp rx=0 tx=0).stop 2024-04-23T18:44:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 shutdown_connections 2024-04-23T18:44:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265829bd0 0x55a26582bfc0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:28.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a265819c10 0x55a265816cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580ae20 0x55a265816780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:29.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f634bfff640 0 client.17376 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 --2- 172.21.15.33:0/3483104620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265819490 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 >> 172.21.15.33:0/3483104620 conn(0x55a265806b30 msgr2=0x55a26580c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:29.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 shutdown_connections 2024-04-23T18:44:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 1 -- 172.21.15.33:0/3483104620 wait complete. 2024-04-23T18:44:29.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.290+0000 7f63759b7740 20 client.17376 trim_cache size 0 max 16384 2024-04-23T18:44:29.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.304+0000 7f63759b7740 1 Processor -- start 2024-04-23T18:44:29.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.304+0000 7f63759b7740 1 -- start start 2024-04-23T18:44:29.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265811f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a265819c10 0x55a265815760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ad3a0 con 0x55a26580a500 2024-04-23T18:44:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ab190 con 0x55a26580ae20 2024-04-23T18:44:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63759b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1accd0 con 0x55a265819c10 2024-04-23T18:44:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:30.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41342/0 (socket says 172.21.15.33:41342) 2024-04-23T18:44:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 -- 172.21.15.33:0/119473682 learned_addr learned my addr 172.21.15.33:0/119473682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 -- 172.21.15.33:0/119473682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a265819c10 msgr2=0x55a265815760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:30.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a265819c10 0x55a265815760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 -- 172.21.15.33:0/119473682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 msgr2=0x55a265811f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:30.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265811f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:30.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 -- 172.21.15.33:0/119473682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a26580ac50 con 0x55a26580a500 2024-04-23T18:44:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f6370a3f640 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98071675128cc44e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63567fc640 1 -- 172.21.15.33:0/119473682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360031070 con 0x55a26580a500 2024-04-23T18:44:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63567fc640 1 -- 172.21.15.33:0/119473682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360010760 con 0x55a26580a500 2024-04-23T18:44:30.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.305+0000 7f63567fc640 1 -- 172.21.15.33:0/119473682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360022600 con 0x55a26580a500 2024-04-23T18:44:30.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- 172.21.15.33:0/119473682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 msgr2=0x55a265825770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:30.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6360006010 tx=0x7f636000b7e0 comp rx=0 tx=0).stop 2024-04-23T18:44:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- 172.21.15.33:0/119473682 shutdown_connections 2024-04-23T18:44:31.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a265819c10 0x55a265815760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265811f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 --2- 172.21.15.33:0/119473682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a26580a500 0x55a265825770 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- 172.21.15.33:0/119473682 >> 172.21.15.33:0/119473682 conn(0x55a265806b30 msgr2=0x55a265815050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:31.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- 172.21.15.33:0/119473682 shutdown_connections 2024-04-23T18:44:31.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- 172.21.15.33:0/119473682 wait complete. 2024-04-23T18:44:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 -- start start 2024-04-23T18:44:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.306+0000 7f63759b7740 1 Processor -- start 2024-04-23T18:44:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580a500 0x55a265821e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265812fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:31.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1abf30 con 0x55a265819c10 2024-04-23T18:44:31.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ab860 con 0x55a26580ae20 2024-04-23T18:44:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f636c1ac600 con 0x55a26580a500 2024-04-23T18:44:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f63759b7740 10 client.0 _ll_register_callbacks cb 0x7f636000bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41354/0 (socket says 172.21.15.33:41354) 2024-04-23T18:44:31.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 -- 172.21.15.33:0/913504587 learned_addr learned my addr 172.21.15.33:0/913504587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f636bfff640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265812fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:32.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6370a3f640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580a500 0x55a265821e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 -- 172.21.15.33:0/913504587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580a500 msgr2=0x55a265821e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580a500 0x55a265821e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:32.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 -- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 msgr2=0x55a265812fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:32.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265812fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:32.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 -- 172.21.15.33:0/913504587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a265813a30 con 0x55a265819c10 2024-04-23T18:44:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f6371240640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63640096e0 tx=0x7f6364013a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64b487637da5def server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f634bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.307+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364022040 con 0x55a265819c10 2024-04-23T18:44:32.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f6364004980 con 0x55a265819c10 2024-04-23T18:44:32.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6360010760 con 0x55a265819c10 2024-04-23T18:44:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f636403f050 con 0x55a265819c10 2024-04-23T18:44:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 10 client.17382 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a26588fea0 con 0x55a265819c10 2024-04-23T18:44:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6360010760 con 0x55a265819c10 2024-04-23T18:44:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f636400f040 con 0x55a265819c10 2024-04-23T18:44:33.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 10 client.17382 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 20 client.17382 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:33.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265890470 0x55a265892890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 4 client.17382 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a265817050 con 0x55a265890470 2024-04-23T18:44:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.308+0000 7f6370a3f640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265890470 0x55a265892890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.309+0000 7f634bfff640 10 client.17382 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:33.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.309+0000 7f6370a3f640 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265890470 0x55a265892890 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941dfa68a7d03f2e server_cookie=65be0c3f293577d1 in_seq=0 out_seq=0 2024-04-23T18:44:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f634bfff640 1 -- 172.21.15.33:0/913504587 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a265817050 con 0x55a265890470 2024-04-23T18:44:33.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f634bfff640 10 client.17382 handle_command_reply: tid=0 2024-04-23T18:44:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 client.17382 shutdown 2024-04-23T18:44:33.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 msgr2=0x55a2658134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:33.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63640096e0 tx=0x7f6364013a00 comp rx=0 tx=0).stop 2024-04-23T18:44:34.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 shutdown_connections 2024-04-23T18:44:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a265890470 0x55a265892890 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a265819c10 0x55a2658134f0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a26580ae20 0x55a265812fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f634bfff640 0 client.17382 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 --2- 172.21.15.33:0/913504587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a26580a500 0x55a265821e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 >> 172.21.15.33:0/913504587 conn(0x55a265806b30 msgr2=0x55a2658158e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 shutdown_connections 2024-04-23T18:44:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 1 -- 172.21.15.33:0/913504587 wait complete. 2024-04-23T18:44:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.319+0000 7f63759b7740 20 client.17382 trim_cache size 0 max 16384 2024-04-23T18:44:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:44:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:44:34.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:44:34.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:44:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 msgr2=0x7f635c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 0x7f635c0a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f636401f000 tx=0x7f636401d000 comp rx=0 tx=0).stop 2024-04-23T18:44:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 msgr2=0x7f636c198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6358032fd0 tx=0x7f6358004270 comp rx=0 tx=0).stop 2024-04-23T18:44:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 shutdown_connections 2024-04-23T18:44:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f635c0a3dc0 0x7f635c0a6280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f636c19cf70 0x7f636c19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f636c107290 0x7f636c198bf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 --2- 172.21.15.33:0/346393826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f636c1021c0 0x7f636c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 >> 172.21.15.33:0/346393826 conn(0x7f636c0fda20 msgr2=0x7f636c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 shutdown_connections 2024-04-23T18:44:35.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:16.320+0000 7f6372ca3640 1 -- 172.21.15.33:0/346393826 wait complete. 2024-04-23T18:44:35.319 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:25 smithi122 ceph-mon[32546]: pgmap v571: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:44:35.397 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:27 smithi122 ceph-mon[32546]: pgmap v572: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-23T18:44:35.444 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:29 smithi122 ceph-mon[32546]: pgmap v573: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:44:35.485 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:31 smithi122 ceph-mon[32546]: pgmap v574: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:44:35.518 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:33 smithi122 ceph-mon[32546]: pgmap v575: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-23T18:44:35.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:25 smithi106 ceph-mon[32716]: pgmap v571: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:44:35.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:27 smithi106 ceph-mon[32716]: pgmap v572: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-23T18:44:35.618 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:29 smithi106 ceph-mon[32716]: pgmap v573: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:44:35.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:31 smithi106 ceph-mon[32716]: pgmap v574: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:44:35.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:33 smithi106 ceph-mon[32716]: pgmap v575: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-23T18:44:35.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:16 smithi033 ceph-mon[29274]: pgmap v501: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:44:35.852 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:18 smithi033 ceph-mon[29274]: pgmap v503: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:44:35.963 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:20 smithi033 ceph-mon[29274]: pgmap v504: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:44:36.004 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:35 smithi122 ceph-mon[32546]: pgmap v576: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:36.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:35 smithi106 ceph-mon[32716]: pgmap v576: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:36.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 -- 172.21.15.33:0/401391542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 msgr2=0x7fe80c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:36.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 --2- 172.21.15.33:0/401391542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c10ee50 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe804009ca0 tx=0x7fe80402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:44:36.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 -- 172.21.15.33:0/401391542 shutdown_connections 2024-04-23T18:44:36.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 --2- 172.21.15.33:0/401391542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c10ee50 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 --2- 172.21.15.33:0/401391542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102bf0 0x7fe80c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 --2- 172.21.15.33:0/401391542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c102240 0x7fe80c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:36.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 -- 172.21.15.33:0/401391542 >> 172.21.15.33:0/401391542 conn(0x7fe80c0790e0 msgr2=0x7fe80c0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 -- 172.21.15.33:0/401391542 shutdown_connections 2024-04-23T18:44:36.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.174+0000 7fe811e10640 1 -- 172.21.15.33:0/401391542 wait complete. 2024-04-23T18:44:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 Processor -- start 2024-04-23T18:44:36.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 -- start start 2024-04-23T18:44:36.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102240 0x7fe80c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:36.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:36.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c074cd0 0x7fe80c1a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1121e0 con 0x7fe80c107280 2024-04-23T18:44:36.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c112060 con 0x7fe80c102240 2024-04-23T18:44:36.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe811e10640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c112360 con 0x7fe80c074cd0 2024-04-23T18:44:36.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe80affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe80affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35718/0 (socket says 172.21.15.33:35718) 2024-04-23T18:44:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.175+0000 7fe80affd640 1 -- 172.21.15.33:0/1805597172 learned_addr learned my addr 172.21.15.33:0/1805597172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:36.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80bfff640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c074cd0 0x7fe80c1a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80b7fe640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102240 0x7fe80c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 -- 172.21.15.33:0/1805597172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c074cd0 msgr2=0x7fe80c1a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c074cd0 0x7fe80c1a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 -- 172.21.15.33:0/1805597172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102240 msgr2=0x7fe80c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:37.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102240 0x7fe80c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:37.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 -- 172.21.15.33:0/1805597172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c1a6ff0 con 0x7fe80c107280 2024-04-23T18:44:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe80affd640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c970 tx=0x7fe80000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:37.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800007bb0 con 0x7fe80c107280 2024-04-23T18:44:37.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c1a72e0 con 0x7fe80c107280 2024-04-23T18:44:37.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe800007d50 con 0x7fe80c107280 2024-04-23T18:44:37.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe800004e10 con 0x7fe80c107280 2024-04-23T18:44:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.176+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c1a7830 con 0x7fe80c107280 2024-04-23T18:44:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.177+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c0691c0 con 0x7fe80c107280 2024-04-23T18:44:37.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.178+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe800005000 con 0x7fe80c107280 2024-04-23T18:44:37.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.179+0000 7fe808ff9640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 0x7fe7f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.179+0000 7fe80b7fe640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 0x7fe7f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:37.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.179+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fe8000d04b0 con 0x7fe80c107280 2024-04-23T18:44:37.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.180+0000 7fe80b7fe640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 0x7fe7f40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005fd0 tx=0x7fe7fc005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:37.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.184+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe800014030 con 0x7fe80c107280 2024-04-23T18:44:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.408+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe80c10a4a0 con 0x7fe80c107280 2024-04-23T18:44:37.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.409+0000 7fe808ff9640 1 -- 172.21.15.33:0/1805597172 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11101 (secure 0 0 0) 0x7fe80009cb60 con 0x7fe80c107280 2024-04-23T18:44:37.937 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:44:37.987 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2258040370","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2258040370},{"type":"v1","addr":"172.21.15.33:6835","nonce":2258040370}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:42:15.861565+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_17364":{"gid":17364,"name":"d","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6837/2982854948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2982854948},{"type":"v1","addr":"172.21.15.33:6837","nonce":2982854948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24466,24482,24551,24490]},"id":1}]} 2024-04-23T18:44:38.039 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 38 2024-04-23T18:44:38.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 msgr2=0x7fe7f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:38.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 0x7fe7f40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc005fd0 tx=0x7fe7fc005850 comp rx=0 tx=0).stop 2024-04-23T18:44:38.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 msgr2=0x7fe80c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fe80000c970 tx=0x7fe80000ce30 comp rx=0 tx=0).stop 2024-04-23T18:44:38.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 shutdown_connections 2024-04-23T18:44:38.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe7f40a3d50 0x7fe7f40a6210 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:38.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe80c074cd0 0x7fe80c1a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:38.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe80c107280 0x7fe80c074790 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:38.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 --2- 172.21.15.33:0/1805597172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe80c102240 0x7fe80c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 >> 172.21.15.33:0/1805597172 conn(0x7fe80c0790e0 msgr2=0x7fe80c108400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:38.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 shutdown_connections 2024-04-23T18:44:38.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:42:22.411+0000 7fe811e10640 1 -- 172.21.15.33:0/1805597172 wait complete. 2024-04-23T18:44:38.697 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --set-id 6bf9e7c1 --release --await' 2024-04-23T18:44:38.755 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --set-id 6bf9e7c1 --release --await 2024-04-23T18:44:38.864 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:22 smithi033 ceph-mon[29274]: pgmap v505: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:44:38.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:22 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1805597172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:44:38.963 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:24 smithi033 ceph-mon[29274]: pgmap v506: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-04-23T18:44:39.013 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:26 smithi033 ceph-mon[29274]: pgmap v507: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-04-23T18:44:39.063 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:28 smithi033 ceph-mon[29274]: pgmap v508: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.0 KiB/s wr, 10 op/s 2024-04-23T18:44:39.114 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:37 smithi122 ceph-mon[32546]: pgmap v577: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:39.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:37 smithi106 ceph-mon[32716]: pgmap v577: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 -- 172.21.15.33:0/2362677695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 msgr2=0x7fb860101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:39.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 --2- 172.21.15.33:0/2362677695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb860101160 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb850009900 tx=0x7fb85002a410 comp rx=0 tx=0).stop 2024-04-23T18:44:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 -- 172.21.15.33:0/2362677695 shutdown_connections 2024-04-23T18:44:39.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 --2- 172.21.15.33:0/2362677695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb86010e930 0x7fb860110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 --2- 172.21.15.33:0/2362677695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8601016a0 0x7fb86010e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 --2- 172.21.15.33:0/2362677695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb860101160 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 -- 172.21.15.33:0/2362677695 >> 172.21.15.33:0/2362677695 conn(0x7fb8600771d0 msgr2=0x7fb8600775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 -- 172.21.15.33:0/2362677695 shutdown_connections 2024-04-23T18:44:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.419+0000 7fb866168640 1 -- 172.21.15.33:0/2362677695 wait complete. 2024-04-23T18:44:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 Processor -- start 2024-04-23T18:44:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 -- start start 2024-04-23T18:44:39.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8601016a0 0x7fb8601a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86010e930 0x7fb86019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601144a0 con 0x7fb860100d80 2024-04-23T18:44:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb860114320 con 0x7fb8601016a0 2024-04-23T18:44:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.420+0000 7fb866168640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb860114620 con 0x7fb86010e930 2024-04-23T18:44:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55368/0 (socket says 172.21.15.33:55368) 2024-04-23T18:44:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 -- 172.21.15.33:0/3616912636 learned_addr learned my addr 172.21.15.33:0/3616912636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85effd640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8601016a0 0x7fb8601a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85ffff640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86010e930 0x7fb86019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 -- 172.21.15.33:0/3616912636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86010e930 msgr2=0x7fb86019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:40.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86010e930 0x7fb86019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 -- 172.21.15.33:0/3616912636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8601016a0 msgr2=0x7fb8601a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8601016a0 0x7fb8601a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:40.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 -- 172.21.15.33:0/3616912636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86019b030 con 0x7fb860100d80 2024-04-23T18:44:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb850005ab0 tx=0x7fb850003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85cff9640 1 -- 172.21.15.33:0/3616912636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85003f070 con 0x7fb860100d80 2024-04-23T18:44:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85cff9640 1 -- 172.21.15.33:0/3616912636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb850031b60 con 0x7fb860100d80 2024-04-23T18:44:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86019b2c0 con 0x7fb860100d80 2024-04-23T18:44:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb85cff9640 1 -- 172.21.15.33:0/3616912636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850035030 con 0x7fb860100d80 2024-04-23T18:44:41.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.421+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a74f0 con 0x7fb860100d80 2024-04-23T18:44:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb85cff9640 1 -- 172.21.15.33:0/3616912636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb850042070 con 0x7fb860100d80 2024-04-23T18:44:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 Processor -- start 2024-04-23T18:44:41.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 -- start start 2024-04-23T18:44:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba002da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 0x55cba003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:41.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cb9ff92970 con 0x55cba002df40 2024-04-23T18:44:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb9ff2f4a0 con 0x55cba002d620 2024-04-23T18:44:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb868e84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cb9ff8e930 con 0x55cba0038800 2024-04-23T18:44:41.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.424+0000 7fb85cff9640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 0x7fb8480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:41.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85ffff640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 0x7fb8480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55374/0 (socket says 172.21.15.33:55374) 2024-04-23T18:44:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 -- 172.21.15.33:0/365143476 learned_addr learned my addr 172.21.15.33:0/365143476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85effd640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 0x55cba003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85ffff640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba002da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85cff9640 1 -- 172.21.15.33:0/3616912636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fb8500f0f60 con 0x7fb860100d80 2024-04-23T18:44:42.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 -- 172.21.15.33:0/365143476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 msgr2=0x55cba003abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:42.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 0x55cba003abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 -- 172.21.15.33:0/365143476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 msgr2=0x55cba002da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:42.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba002da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 -- 172.21.15.33:0/365143476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb9fca5210 con 0x55cba002df40 2024-04-23T18:44:42.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85effd640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 0x55cba003abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85ffff640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 0x7fb8480a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb86006a570 tx=0x7fb858009e80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:42.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb850005c20 tx=0x7fb85008fa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb193691dc6fd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.425+0000 7fb845ffb640 1 -- 172.21.15.33:0/365143476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850049070 con 0x55cba002df40 2024-04-23T18:44:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb845ffb640 1 -- 172.21.15.33:0/365143476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500c1260 con 0x55cba002df40 2024-04-23T18:44:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb845ffb640 1 -- 172.21.15.33:0/365143476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500b95d0 con 0x55cba002df40 2024-04-23T18:44:42.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 -- 172.21.15.33:0/365143476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 msgr2=0x55cba00382c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb850005c20 tx=0x7fb85008fa50 comp rx=0 tx=0).stop 2024-04-23T18:44:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 -- 172.21.15.33:0/365143476 shutdown_connections 2024-04-23T18:44:42.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba0038800 0x55cba003abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002df40 0x55cba00382c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 --2- 172.21.15.33:0/365143476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba002da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 -- 172.21.15.33:0/365143476 >> 172.21.15.33:0/365143476 conn(0x55cba0029b30 msgr2=0x55cba0029f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:42.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 -- 172.21.15.33:0/365143476 shutdown_connections 2024-04-23T18:44:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.426+0000 7fb868e84740 1 -- 172.21.15.33:0/365143476 wait complete. 2024-04-23T18:44:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 -- start start 2024-04-23T18:44:43.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 Processor -- start 2024-04-23T18:44:43.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba0038150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0035440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cba003b1f0 con 0x55cba0038800 2024-04-23T18:44:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb9ff2d0b0 con 0x55cba002d620 2024-04-23T18:44:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cba003b370 con 0x55cba002df40 2024-04-23T18:44:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb868e84740 10 client.0 _ll_register_callbacks cb 0x55cb9ff1ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:43.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55384/0 (socket says 172.21.15.33:55384) 2024-04-23T18:44:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.427+0000 7fb85ffff640 1 -- 172.21.15.33:0/2211793243 learned_addr learned my addr 172.21.15.33:0/2211793243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba0038150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:43.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85effd640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0035440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:43.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 -- 172.21.15.33:0/2211793243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 msgr2=0x55cba0035440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0035440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:43.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 -- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 msgr2=0x55cba0038150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba0038150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 -- 172.21.15.33:0/2211793243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb9fd71620 con 0x55cba0038800 2024-04-23T18:44:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85effd640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0035440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb85ffff640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb8580094d0 tx=0x7fb85801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6f88ed9550c53d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb83effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:44.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858024070 con 0x55cba0038800 2024-04-23T18:44:44.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fb8580047c0 con 0x55cba0038800 2024-04-23T18:44:44.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8500c1260 con 0x55cba0038800 2024-04-23T18:44:44.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb858031450 con 0x55cba0038800 2024-04-23T18:44:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb868e84740 10 client.17400 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cba0047050 con 0x55cba0038800 2024-04-23T18:44:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.428+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500c1260 con 0x55cba0038800 2024-04-23T18:44:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fb858028400 con 0x55cba0038800 2024-04-23T18:44:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 10 client.17400 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:44.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 20 client.17400 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:44.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00485d0 0x55cba004a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 4 client.17400 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:44.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb9ffa0390 con 0x55cba00485d0 2024-04-23T18:44:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.429+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00485d0 0x55cba004a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00485d0 0x55cba004a9e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33bf7dedf2f3671 server_cookie=3a0306e25166a3c7 in_seq=0 out_seq=0 2024-04-23T18:44:44.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb83effd640 10 client.17400 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:44.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb83effd640 1 -- 172.21.15.33:0/2211793243 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb9ffa0390 con 0x55cba00485d0 2024-04-23T18:44:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb83effd640 10 client.17400 handle_command_reply: tid=0 2024-04-23T18:44:44.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb868e84740 1 client.17400 shutdown 2024-04-23T18:44:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 msgr2=0x55cba00359a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:44.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.430+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb8580094d0 tx=0x7fb85801d000 comp rx=0 tx=0).stop 2024-04-23T18:44:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 shutdown_connections 2024-04-23T18:44:45.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00485d0 0x55cba004a9e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba00359a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0035440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb83effd640 0 client.17400 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 --2- 172.21.15.33:0/2211793243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002d620 0x55cba0038150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 >> 172.21.15.33:0/2211793243 conn(0x55cba0029b30 msgr2=0x55cba002f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 shutdown_connections 2024-04-23T18:44:45.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 1 -- 172.21.15.33:0/2211793243 wait complete. 2024-04-23T18:44:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.431+0000 7fb868e84740 20 client.17400 trim_cache size 0 max 16384 2024-04-23T18:44:45.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 Processor -- start 2024-04-23T18:44:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 -- start start 2024-04-23T18:44:45.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0028b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 0x55cba00502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:45.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b17f0 con 0x55cba002d620 2024-04-23T18:44:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601af5e0 con 0x55cba0038800 2024-04-23T18:44:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb868e84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b1120 con 0x55cba002df40 2024-04-23T18:44:46.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55396/0 (socket says 172.21.15.33:55396) 2024-04-23T18:44:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85f7fe640 1 -- 172.21.15.33:0/917701181 learned_addr learned my addr 172.21.15.33:0/917701181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:46.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85effd640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0028b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85ffff640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 0x55cba00502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:46.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.446+0000 7fb85f7fe640 1 -- 172.21.15.33:0/917701181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 msgr2=0x55cba0028b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0028b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85f7fe640 1 -- 172.21.15.33:0/917701181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 msgr2=0x55cba00502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:46.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 0x55cba00502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85f7fe640 1 -- 172.21.15.33:0/917701181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cba0030fe0 con 0x55cba002d620 2024-04-23T18:44:46.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85ffff640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 0x55cba00502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb8500bcc60 tx=0x7fb85002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781603e47a5cd0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:46.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb8457fa640 1 -- 172.21.15.33:0/917701181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850049040 con 0x55cba002d620 2024-04-23T18:44:46.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb8457fa640 1 -- 172.21.15.33:0/917701181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500c1260 con 0x55cba002d620 2024-04-23T18:44:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb8457fa640 1 -- 172.21.15.33:0/917701181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8500bd1c0 con 0x55cba002d620 2024-04-23T18:44:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 -- 172.21.15.33:0/917701181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 msgr2=0x55cba0043fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb8500bcc60 tx=0x7fb85002a410 comp rx=0 tx=0).stop 2024-04-23T18:44:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 -- 172.21.15.33:0/917701181 shutdown_connections 2024-04-23T18:44:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba0038800 0x55cba00502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002df40 0x55cba0028b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 --2- 172.21.15.33:0/917701181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba002d620 0x55cba0043fe0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.447+0000 7fb868e84740 1 -- 172.21.15.33:0/917701181 >> 172.21.15.33:0/917701181 conn(0x55cba0029b30 msgr2=0x55cba0028420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- 172.21.15.33:0/917701181 shutdown_connections 2024-04-23T18:44:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- 172.21.15.33:0/917701181 wait complete. 2024-04-23T18:44:47.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- start start 2024-04-23T18:44:47.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 Processor -- start 2024-04-23T18:44:47.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002d620 0x55cba0040790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002df40 0x55cba003db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:47.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0380 con 0x55cba0038800 2024-04-23T18:44:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601afcb0 con 0x55cba002df40 2024-04-23T18:44:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0a50 con 0x55cba002d620 2024-04-23T18:44:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb868e84740 10 client.0 _ll_register_callbacks cb 0x7fb85800fef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.448+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55404/0 (socket says 172.21.15.33:55404) 2024-04-23T18:44:48.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 -- 172.21.15.33:0/1605930388 learned_addr learned my addr 172.21.15.33:0/1605930388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002d620 0x55cba0040790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85effd640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002df40 0x55cba003db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:48.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 -- 172.21.15.33:0/1605930388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002d620 msgr2=0x55cba0040790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:48.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002d620 0x55cba0040790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 -- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002df40 msgr2=0x55cba003db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002df40 0x55cba003db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 -- 172.21.15.33:0/1605930388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cba003e720 con 0x55cba0038800 2024-04-23T18:44:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb85ffff640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb85800f870 tx=0x7fb858028c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e986ef618d8fa43 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:48.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb83effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858024040 con 0x55cba0038800 2024-04-23T18:44:48.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8500c1260 con 0x55cba0038800 2024-04-23T18:44:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fb8580049d0 con 0x55cba0038800 2024-04-23T18:44:48.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb858004400 con 0x55cba0038800 2024-04-23T18:44:48.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb868e84740 10 client.17406 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.449+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cba00ae9c0 con 0x55cba0038800 2024-04-23T18:44:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8500c1260 con 0x55cba0038800 2024-04-23T18:44:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fb85801f450 con 0x55cba0038800 2024-04-23T18:44:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 10 client.17406 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 20 client.17406 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00af070 0x55cba00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:49.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 4 client.17406 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb9ff9a200 con 0x55cba00af070 2024-04-23T18:44:49.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00af070 0x55cba00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb85f7fe640 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00af070 0x55cba00b1490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463b01997ee15182 server_cookie=5da7b4a9e3ee9dce in_seq=0 out_seq=0 2024-04-23T18:44:49.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.450+0000 7fb83effd640 10 client.17406 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:49.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.454+0000 7fb83effd640 1 -- 172.21.15.33:0/1605930388 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb9ff9a200 con 0x55cba00af070 2024-04-23T18:44:49.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.454+0000 7fb83effd640 10 client.17406 handle_command_reply: tid=0 2024-04-23T18:44:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.454+0000 7fb868e84740 1 client.17406 shutdown 2024-04-23T18:44:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 msgr2=0x55cba003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:49.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb85800f870 tx=0x7fb858028c90 comp rx=0 tx=0).stop 2024-04-23T18:44:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 shutdown_connections 2024-04-23T18:44:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cba00af070 0x55cba00b1490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cba0038800 0x55cba003e060 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cba002df40 0x55cba003db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 --2- 172.21.15.33:0/1605930388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cba002d620 0x55cba0040790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb83effd640 0 client.17406 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 >> 172.21.15.33:0/1605930388 conn(0x55cba0029b30 msgr2=0x55cba0028cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 shutdown_connections 2024-04-23T18:44:50.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 1 -- 172.21.15.33:0/1605930388 wait complete. 2024-04-23T18:44:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb868e84740 20 client.17406 trim_cache size 0 max 16384 2024-04-23T18:44:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:44:50.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:44:50.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:44:50.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:44:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 msgr2=0x7fb8480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:50.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 0x7fb8480a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb86006a570 tx=0x7fb858009e80 comp rx=0 tx=0).stop 2024-04-23T18:44:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 msgr2=0x7fb8601a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.455+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb850005ab0 tx=0x7fb850003c10 comp rx=0 tx=0).stop 2024-04-23T18:44:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 shutdown_connections 2024-04-23T18:44:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8480a3dc0 0x7fb8480a6280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb86010e930 0x7fb86019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8601016a0 0x7fb8601a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 --2- 172.21.15.33:0/3616912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb860100d80 0x7fb8601a06a0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:50.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 >> 172.21.15.33:0/3616912636 conn(0x7fb8600771d0 msgr2=0x7fb86010fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 shutdown_connections 2024-04-23T18:44:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:29.456+0000 7fb866168640 1 -- 172.21.15.33:0/3616912636 wait complete. 2024-04-23T18:44:50.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:30 smithi033 ceph-mon[29274]: pgmap v509: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 3.9 KiB/s wr, 10 op/s 2024-04-23T18:44:50.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:32 smithi033 ceph-mon[29274]: pgmap v510: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.7 KiB/s wr, 15 op/s 2024-04-23T18:44:50.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:34 smithi033 ceph-mon[29274]: pgmap v511: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:50.766 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:36 smithi033 ceph-mon[29274]: pgmap v512: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:50.783 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:38 smithi033 ceph-mon[29274]: pgmap v513: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:50.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:41 smithi033 ceph-mon[29274]: pgmap v514: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:50.820 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:39 smithi122 ceph-mon[32546]: pgmap v578: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:50.925 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:41 smithi122 ceph-mon[32546]: pgmap v579: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:50.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:43 smithi122 ceph-mon[32546]: pgmap v580: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:50.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:45 smithi122 ceph-mon[32546]: pgmap v581: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:50.991 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:47 smithi122 ceph-mon[32546]: pgmap v582: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:51.118 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:49 smithi122 ceph-mon[32546]: pgmap v583: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:51.142 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:39 smithi106 ceph-mon[32716]: pgmap v578: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:51.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:41 smithi106 ceph-mon[32716]: pgmap v579: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:51.200 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:43 smithi106 ceph-mon[32716]: pgmap v580: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:51.233 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:45 smithi106 ceph-mon[32716]: pgmap v581: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:44:51.258 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:47 smithi106 ceph-mon[32716]: pgmap v582: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:44:51.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:49 smithi106 ceph-mon[32716]: pgmap v583: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:44:51.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 -- 172.21.15.33:0/1831574457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 msgr2=0x7f345410e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:51.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 --2- 172.21.15.33:0/1831574457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f345410e3f0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3444009900 tx=0x7f344402a410 comp rx=0 tx=0).stop 2024-04-23T18:44:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 -- 172.21.15.33:0/1831574457 shutdown_connections 2024-04-23T18:44:51.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 --2- 172.21.15.33:0/1831574457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f345410e930 0x7f3454110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 --2- 172.21.15.33:0/1831574457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f345410e3f0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 --2- 172.21.15.33:0/1831574457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3454100d80 0x7f3454101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:51.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 -- 172.21.15.33:0/1831574457 >> 172.21.15.33:0/1831574457 conn(0x7f34540771d0 msgr2=0x7f34540775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 -- 172.21.15.33:0/1831574457 shutdown_connections 2024-04-23T18:44:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.556+0000 7f345be20640 1 -- 172.21.15.33:0/1831574457 wait complete. 2024-04-23T18:44:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 Processor -- start 2024-04-23T18:44:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 -- start start 2024-04-23T18:44:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454100d80 0x7f34541a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:51.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:51.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 0x7f345419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3454114450 con 0x7f34541016a0 2024-04-23T18:44:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34541142d0 con 0x7f3454100d80 2024-04-23T18:44:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345be20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34541145d0 con 0x7f345410e930 2024-04-23T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35004/0 (socket says 172.21.15.33:35004) 2024-04-23T18:44:52.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 -- 172.21.15.33:0/3089932769 learned_addr learned my addr 172.21.15.33:0/3089932769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f3459bbc640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454100d80 0x7f34541a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345a3bd640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 0x7f345419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 -- 172.21.15.33:0/3089932769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 msgr2=0x7f345419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 0x7f345419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 -- 172.21.15.33:0/3089932769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454100d80 msgr2=0x7f34541a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454100d80 0x7f34541a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f34593bb640 1 -- 172.21.15.33:0/3089932769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345419afd0 con 0x7f34541016a0 2024-04-23T18:44:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.557+0000 7f345a3bd640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 0x7f345419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f34593bb640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3444005940 tx=0x7f34440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:52.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f344affd640 1 -- 172.21.15.33:0/3089932769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344403f030 con 0x7f34541016a0 2024-04-23T18:44:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f344affd640 1 -- 172.21.15.33:0/3089932769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3444003cb0 con 0x7f34541016a0 2024-04-23T18:44:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345419b260 con 0x7f34541016a0 2024-04-23T18:44:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f344affd640 1 -- 172.21.15.33:0/3089932769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444035070 con 0x7f34541016a0 2024-04-23T18:44:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.558+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34541a73f0 con 0x7f34541016a0 2024-04-23T18:44:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.560+0000 7f344affd640 1 -- 172.21.15.33:0/3089932769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3444042030 con 0x7f34541016a0 2024-04-23T18:44:52.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.561+0000 7f345ec0c740 1 Processor -- start 2024-04-23T18:44:52.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.561+0000 7f345ec0c740 1 -- start start 2024-04-23T18:44:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.561+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.561+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac162c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:52.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345ec0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55946ab70970 con 0x55946ac16800 2024-04-23T18:44:52.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345ec0c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55946ab0d4a0 con 0x55946ac0b620 2024-04-23T18:44:53.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345ec0c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55946ab6c930 con 0x55946ac0bf40 2024-04-23T18:44:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f344affd640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 0x7f34400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:53.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:53.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345a3bd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35006/0 (socket says 172.21.15.33:35006) 2024-04-23T18:44:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 -- 172.21.15.33:0/500338972 learned_addr learned my addr 172.21.15.33:0/500338972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:53.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f3459bbc640 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345a3bd640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 0x7f34400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f344affd640 1 -- 172.21.15.33:0/3089932769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f34440f1090 con 0x7f34541016a0 2024-04-23T18:44:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 -- 172.21.15.33:0/500338972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 msgr2=0x55946ac162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 -- 172.21.15.33:0/500338972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 msgr2=0x55946ac0ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac0ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 -- 172.21.15.33:0/500338972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a883210 con 0x55946ac16800 2024-04-23T18:44:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345a3bd640 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:44:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f34593bb640 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f34440bcd40 tx=0x7f3444032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d6366ed7ca065b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.562+0000 7f345a3bd640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 0x7f34400a6280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f344c005c40 tx=0x7f344c009200 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f343ffff640 1 -- 172.21.15.33:0/500338972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3444049070 con 0x55946ac16800 2024-04-23T18:44:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f343ffff640 1 -- 172.21.15.33:0/500338972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f344403b460 con 0x55946ac16800 2024-04-23T18:44:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f343ffff640 1 -- 172.21.15.33:0/500338972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34440eebd0 con 0x55946ac16800 2024-04-23T18:44:53.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 -- 172.21.15.33:0/500338972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 msgr2=0x55946ac18bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f34440bcd40 tx=0x7f3444032f60 comp rx=0 tx=0).stop 2024-04-23T18:44:53.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 -- 172.21.15.33:0/500338972 shutdown_connections 2024-04-23T18:44:54.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac18bf0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 --2- 172.21.15.33:0/500338972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac0ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.563+0000 7f345ec0c740 1 -- 172.21.15.33:0/500338972 >> 172.21.15.33:0/500338972 conn(0x55946ac07b30 msgr2=0x55946ac07f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- 172.21.15.33:0/500338972 shutdown_connections 2024-04-23T18:44:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- 172.21.15.33:0/500338972 wait complete. 2024-04-23T18:44:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- start start 2024-04-23T18:44:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 Processor -- start 2024-04-23T18:44:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55946ac191f0 con 0x55946ac0b620 2024-04-23T18:44:54.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55946ab0b0b0 con 0x55946ac16800 2024-04-23T18:44:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55946ac19370 con 0x55946ac0bf40 2024-04-23T18:44:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.564+0000 7f345ec0c740 10 client.0 _ll_register_callbacks cb 0x55946aaf8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:54.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35022/0 (socket says 172.21.15.33:35022) 2024-04-23T18:44:54.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 -- 172.21.15.33:0/691323348 learned_addr learned my addr 172.21.15.33:0/691323348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f34593bb640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f345a3bd640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:54.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 -- 172.21.15.33:0/691323348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 msgr2=0x55946ac13380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac13380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 -- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 msgr2=0x55946ac138c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac138c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 -- 172.21.15.33:0/691323348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946a94f620 con 0x55946ac0b620 2024-04-23T18:44:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f345a3bd640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:44:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f3459bbc640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f345000ba10 tx=0x7f345000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c3d260df6d0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:55.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f343d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3450003e00 con 0x55946ac0b620 2024-04-23T18:44:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f3450011440 con 0x55946ac0b620 2024-04-23T18:44:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55946ab7e390 con 0x55946ac0b620 2024-04-23T18:44:55.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3450011640 con 0x55946ac0b620 2024-04-23T18:44:55.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f345ec0c740 10 client.17418 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:55.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.565+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55946ac25040 con 0x55946ac0b620 2024-04-23T18:44:55.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55946ab7e390 con 0x55946ac0b620 2024-04-23T18:44:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f345001b920 con 0x55946ac0b620 2024-04-23T18:44:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 10 client.17418 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 20 client.17418 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:55.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac26550 0x55946ac28960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:55.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 4 client.17418 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:44:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55946ab78200 con 0x55946ac26550 2024-04-23T18:44:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.566+0000 7f34593bb640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac26550 0x55946ac28960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:55.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f343d7fa640 10 client.17418 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:55.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f34593bb640 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac26550 0x55946ac28960 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daed14ea9a3fbdfc server_cookie=a8222bc69649f488 in_seq=0 out_seq=0 2024-04-23T18:44:55.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f343d7fa640 1 -- 172.21.15.33:0/691323348 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55946ab78200 con 0x55946ac26550 2024-04-23T18:44:56.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f343d7fa640 10 client.17418 handle_command_reply: tid=0 2024-04-23T18:44:56.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f345ec0c740 1 client.17418 shutdown 2024-04-23T18:44:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 msgr2=0x55946ac16090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.567+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f345000ba10 tx=0x7f345000bed0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 shutdown_connections 2024-04-23T18:44:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac26550 0x55946ac28960 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac138c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac13380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f343d7fa640 0 client.17418 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 --2- 172.21.15.33:0/691323348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0b620 0x55946ac16090 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 >> 172.21.15.33:0/691323348 conn(0x55946ac07b30 msgr2=0x55946ac0d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:56.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 shutdown_connections 2024-04-23T18:44:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 1 -- 172.21.15.33:0/691323348 wait complete. 2024-04-23T18:44:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.568+0000 7f345ec0c740 20 client.17418 trim_cache size 0 max 16384 2024-04-23T18:44:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 Processor -- start 2024-04-23T18:44:56.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 -- start start 2024-04-23T18:44:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac15bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac2e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b17f0 con 0x55946ac16800 2024-04-23T18:44:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34541af5e0 con 0x55946ac0b620 2024-04-23T18:44:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345ec0c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b1120 con 0x55946ac0bf40 2024-04-23T18:44:56.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35030/0 (socket says 172.21.15.33:35030) 2024-04-23T18:44:56.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 -- 172.21.15.33:0/2901000782 learned_addr learned my addr 172.21.15.33:0/2901000782 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f3459bbc640 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac15bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 -- 172.21.15.33:0/2901000782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 msgr2=0x55946ac2e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:56.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f34593bb640 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac2e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:56.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac2e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:56.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 -- 172.21.15.33:0/2901000782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 msgr2=0x55946ac15bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac15bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.583+0000 7f345a3bd640 1 -- 172.21.15.33:0/2901000782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946ac0bd70 con 0x55946ac16800 2024-04-23T18:44:57.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345a3bd640 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f344c011cb0 tx=0x7f344c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecd17ae5a684a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:57.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f343f7fe640 1 -- 172.21.15.33:0/2901000782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c026030 con 0x55946ac16800 2024-04-23T18:44:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f343f7fe640 1 -- 172.21.15.33:0/2901000782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f344c011e60 con 0x55946ac16800 2024-04-23T18:44:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f343f7fe640 1 -- 172.21.15.33:0/2901000782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344c005330 con 0x55946ac16800 2024-04-23T18:44:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 -- 172.21.15.33:0/2901000782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 msgr2=0x55946ac2e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:57.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f344c011cb0 tx=0x7f344c009650 comp rx=0 tx=0).stop 2024-04-23T18:44:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 -- 172.21.15.33:0/2901000782 shutdown_connections 2024-04-23T18:44:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac16800 0x55946ac2e7b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0bf40 0x55946ac2e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2901000782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac0b620 0x55946ac15bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.584+0000 7f345ec0c740 1 -- 172.21.15.33:0/2901000782 >> 172.21.15.33:0/2901000782 conn(0x55946ac07b30 msgr2=0x55946ac06cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- 172.21.15.33:0/2901000782 shutdown_connections 2024-04-23T18:44:57.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- 172.21.15.33:0/2901000782 wait complete. 2024-04-23T18:44:57.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- start start 2024-04-23T18:44:57.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 Processor -- start 2024-04-23T18:44:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0b620 0x55946ac19ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:57.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:57.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac1a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b0380 con 0x55946ac0bf40 2024-04-23T18:44:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34541afcb0 con 0x55946ac16800 2024-04-23T18:44:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34541b0a50 con 0x55946ac0b620 2024-04-23T18:44:57.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f345ec0c740 10 client.0 _ll_register_callbacks cb 0x7f34500044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:44:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:58.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f34593bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35034/0 (socket says 172.21.15.33:35034) 2024-04-23T18:44:58.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.585+0000 7f34593bb640 1 -- 172.21.15.33:0/2631985213 learned_addr learned my addr 172.21.15.33:0/2631985213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:44:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f3459bbc640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0b620 0x55946ac19ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f345a3bd640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac1a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:58.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 -- 172.21.15.33:0/2631985213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0b620 msgr2=0x55946ac19ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0b620 0x55946ac19ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 -- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 msgr2=0x55946ac1a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:44:58.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac1a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 -- 172.21.15.33:0/2631985213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946ac8bae0 con 0x55946ac0bf40 2024-04-23T18:44:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f345a3bd640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac1a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:44:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f34593bb640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f34440b94a0 tx=0x7f344403b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9c515eaed58c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:44:58.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f343d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:44:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344409be60 con 0x55946ac0bf40 2024-04-23T18:44:58.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7f34440b96b0 con 0x55946ac0bf40 2024-04-23T18:44:58.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f344c011e60 con 0x55946ac0bf40 2024-04-23T18:44:58.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3444049040 con 0x55946ac0bf40 2024-04-23T18:44:58.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f345ec0c740 10 client.17424 fetch_fsmap learned FSMap version 38 2024-04-23T18:44:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55946ac8cda0 con 0x55946ac0bf40 2024-04-23T18:44:58.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.586+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f344c011e60 con 0x55946ac0bf40 2024-04-23T18:44:58.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f34440bd330 con 0x55946ac0bf40 2024-04-23T18:44:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 10 client.17424 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:44:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:44:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 20 client.17424 populate_metadata read hostname 'smithi033' 2024-04-23T18:44:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac8d3b0 0x55946ac8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:44:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 4 client.17424 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:44:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55946aaf45f0 con 0x55946ac8d3b0 2024-04-23T18:44:59.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f3459bbc640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac8d3b0 0x55946ac8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:44:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f343d7fa640 10 client.17424 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:59.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.587+0000 7f3459bbc640 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac8d3b0 0x55946ac8f7d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e3e509861431f52 server_cookie=b72f915f4b26ad33 in_seq=0 out_seq=0 2024-04-23T18:44:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f343d7fa640 1 -- 172.21.15.33:0/2631985213 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55946aaf45f0 con 0x55946ac8d3b0 2024-04-23T18:44:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f343d7fa640 10 client.17424 handle_command_reply: tid=0 2024-04-23T18:44:59.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 client.17424 shutdown 2024-04-23T18:44:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 msgr2=0x55946ac1a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:44:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f34440b94a0 tx=0x7f344403b640 comp rx=0 tx=0).stop 2024-04-23T18:44:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 shutdown_connections 2024-04-23T18:44:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55946ac8d3b0 0x55946ac8f7d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55946ac16800 0x55946ac1a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55946ac0bf40 0x55946ac1a3e0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 --2- 172.21.15.33:0/2631985213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55946ac0b620 0x55946ac19ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:44:59.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f343d7fa640 0 client.17424 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:44:59.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 >> 172.21.15.33:0/2631985213 conn(0x55946ac07b30 msgr2=0x55946ac12be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:44:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 shutdown_connections 2024-04-23T18:44:59.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 1 -- 172.21.15.33:0/2631985213 wait complete. 2024-04-23T18:44:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.592+0000 7f345ec0c740 20 client.17424 trim_cache size 0 max 16384 2024-04-23T18:44:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:45:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:45:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:45:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:45:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 msgr2=0x7f34400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 0x7f34400a6280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f344c005c40 tx=0x7f344c009200 comp rx=0 tx=0).stop 2024-04-23T18:45:00.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 msgr2=0x7f34541a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3444005940 tx=0x7f34440042f0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 shutdown_connections 2024-04-23T18:45:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f34400a3dc0 0x7f34400a6280 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f345410e930 0x7f345419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34541016a0 0x7f34541a0b90 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 --2- 172.21.15.33:0/3089932769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3454100d80 0x7f34541a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:00.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 >> 172.21.15.33:0/3089932769 conn(0x7f34540771d0 msgr2=0x7f345410fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 shutdown_connections 2024-04-23T18:45:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:42.593+0000 7f345be20640 1 -- 172.21.15.33:0/3089932769 wait complete. 2024-04-23T18:45:00.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:51 smithi122 ceph-mon[32546]: pgmap v584: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:00.766 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:53 smithi122 ceph-mon[32546]: pgmap v585: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:00.883 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:00.941 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:00.999 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:01.033 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:45:01.074 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:55 smithi122 ceph-mon[32546]: pgmap v586: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.108 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:57 smithi122 ceph-mon[32546]: pgmap v587: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:44:59 smithi122 ceph-mon[32546]: pgmap v588: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:01.166 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:51 smithi106 ceph-mon[32716]: pgmap v584: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.200 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:53 smithi106 ceph-mon[32716]: pgmap v585: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.224 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:01.250 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:01.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:01.375 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:45:01.419 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:55 smithi106 ceph-mon[32716]: pgmap v586: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:57 smithi106 ceph-mon[32716]: pgmap v587: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.458 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:44:59 smithi106 ceph-mon[32716]: pgmap v588: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:01.517 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:43 smithi033 ceph-mon[29274]: pgmap v515: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.541 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:45 smithi033 ceph-mon[29274]: pgmap v516: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:01.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:47 smithi033 ceph-mon[29274]: pgmap v517: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:49 smithi033 ceph-mon[29274]: pgmap v518: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:51 smithi033 ceph-mon[29274]: pgmap v519: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.700 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:51 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:45:01.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:45:01.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:45:01.775 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:45:01.800 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:53 smithi033 ceph-mon[29274]: pgmap v520: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:01.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:55 smithi033 ceph-mon[29274]: pgmap v521: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:01.852 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:01 smithi122 ceph-mon[32546]: pgmap v589: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:01.875 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:01 smithi106 ceph-mon[32716]: pgmap v589: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:01.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 -- 172.21.15.33:0/4138442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103180 msgr2=0x7fedfc103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 --2- 172.21.15.33:0/4138442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103180 0x7fedfc103560 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fedec009900 tx=0x7fedec02a480 comp rx=0 tx=0).stop 2024-04-23T18:45:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 -- 172.21.15.33:0/4138442938 shutdown_connections 2024-04-23T18:45:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 --2- 172.21.15.33:0/4138442938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc10c550 0x7fedfc10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 --2- 172.21.15.33:0/4138442938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc103aa0 0x7fedfc10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 --2- 172.21.15.33:0/4138442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103180 0x7fedfc103560 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 -- 172.21.15.33:0/4138442938 >> 172.21.15.33:0/4138442938 conn(0x7fedfc077fd0 msgr2=0x7fedfc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 -- 172.21.15.33:0/4138442938 shutdown_connections 2024-04-23T18:45:02.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.692+0000 7fee00c3c640 1 -- 172.21.15.33:0/4138442938 wait complete. 2024-04-23T18:45:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.693+0000 7fee00c3c640 1 Processor -- start 2024-04-23T18:45:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.693+0000 7fee00c3c640 1 -- start start 2024-04-23T18:45:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.693+0000 7fee00c3c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc103180 0x7fedfc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc10c550 0x7fedfc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc112200 con 0x7fedfc103aa0 2024-04-23T18:45:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc112080 con 0x7fedfc103180 2024-04-23T18:45:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc112380 con 0x7fedfc10c550 2024-04-23T18:45:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45238/0 (socket says 172.21.15.33:45238) 2024-04-23T18:45:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 -- 172.21.15.33:0/3191130752 learned_addr learned my addr 172.21.15.33:0/3191130752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedfa59c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc103180 0x7fedfc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedfad9d640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc10c550 0x7fedfc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 -- 172.21.15.33:0/3191130752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc10c550 msgr2=0x7fedfc19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc10c550 0x7fedfc19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 -- 172.21.15.33:0/3191130752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc103180 msgr2=0x7fedfc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc103180 0x7fedfc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 -- 172.21.15.33:0/3191130752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc19d640 con 0x7fedfc103aa0 2024-04-23T18:45:02.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fedf000d950 tx=0x7fedf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:02.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedeb7fe640 1 -- 172.21.15.33:0/3191130752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0017070 con 0x7fedfc103aa0 2024-04-23T18:45:02.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedeb7fe640 1 -- 172.21.15.33:0/3191130752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf0009c30 con 0x7fedfc103aa0 2024-04-23T18:45:02.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc19d930 con 0x7fedfc103aa0 2024-04-23T18:45:02.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.694+0000 7fedeb7fe640 1 -- 172.21.15.33:0/3191130752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf000f620 con 0x7fedfc103aa0 2024-04-23T18:45:02.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.695+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1a51c0 con 0x7fedfc103aa0 2024-04-23T18:45:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.697+0000 7fedeb7fe640 1 -- 172.21.15.33:0/3191130752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fedf0010460 con 0x7fedfc103aa0 2024-04-23T18:45:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.697+0000 7fee03a32740 1 Processor -- start 2024-04-23T18:45:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.697+0000 7fee03a32740 1 -- start start 2024-04-23T18:45:02.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedeb7fe640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 0x7fede40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd2cc10 0x55be9bd2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be9bc82970 con 0x55be9bd21a30 2024-04-23T18:45:03.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55be9bc1f4a0 con 0x55be9bd22350 2024-04-23T18:45:03.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fee03a32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be9bc7e930 con 0x55be9bd2cc10 2024-04-23T18:45:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45248/0 (socket says 172.21.15.33:45248) 2024-04-23T18:45:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 -- 172.21.15.33:0/3884152405 learned_addr learned my addr 172.21.15.33:0/3884152405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfa59c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 0x7fede40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedeb7fe640 1 -- 172.21.15.33:0/3191130752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fedf00d03a0 con 0x7fedfc103aa0 2024-04-23T18:45:03.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd2cc10 0x55be9bd2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 -- 172.21.15.33:0/3884152405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd2cc10 msgr2=0x55be9bd2f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd2cc10 0x55be9bd2f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 -- 172.21.15.33:0/3884152405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 msgr2=0x55be9bd2c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd2c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 -- 172.21.15.33:0/3884152405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be9b995210 con 0x55be9bd21a30 2024-04-23T18:45:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd2c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfa59c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 0x7fede40a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedec005ab0 tx=0x7fedec005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.698+0000 7fedfad9d640 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55be9bd1f0d0 tx=0x7fedf4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2d4c0a1f03722b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:03.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fede8ff9640 1 -- 172.21.15.33:0/3884152405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf401a030 con 0x55be9bd21a30 2024-04-23T18:45:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fede8ff9640 1 -- 172.21.15.33:0/3884152405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4014e40 con 0x55be9bd21a30 2024-04-23T18:45:03.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fede8ff9640 1 -- 172.21.15.33:0/3884152405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4012c10 con 0x55be9bd21a30 2024-04-23T18:45:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 -- 172.21.15.33:0/3884152405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 msgr2=0x55be9bd21e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:03.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x55be9bd1f0d0 tx=0x7fedf4009b10 comp rx=0 tx=0).stop 2024-04-23T18:45:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 -- 172.21.15.33:0/3884152405 shutdown_connections 2024-04-23T18:45:03.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd2cc10 0x55be9bd2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd2c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 --2- 172.21.15.33:0/3884152405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd21e10 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:03.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.699+0000 7fee03a32740 1 -- 172.21.15.33:0/3884152405 >> 172.21.15.33:0/3884152405 conn(0x55be9bd1df40 msgr2=0x55be9bd1e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- 172.21.15.33:0/3884152405 shutdown_connections 2024-04-23T18:45:04.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- 172.21.15.33:0/3884152405 wait complete. 2024-04-23T18:45:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- start start 2024-04-23T18:45:04.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 Processor -- start 2024-04-23T18:45:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd2c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd29840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:04.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be9bd2f7e0 con 0x55be9bd2cc10 2024-04-23T18:45:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55be9bc1d0b0 con 0x55be9bd22350 2024-04-23T18:45:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55be9bd2f960 con 0x55be9bd21a30 2024-04-23T18:45:04.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fee03a32740 10 client.0 _ll_register_callbacks cb 0x55be9bc0ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fedfa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd2c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45254/0 (socket says 172.21.15.33:45254) 2024-04-23T18:45:04.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.700+0000 7fedfad9d640 1 -- 172.21.15.33:0/791592763 learned_addr learned my addr 172.21.15.33:0/791592763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:04.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd29840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:04.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 -- 172.21.15.33:0/791592763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 msgr2=0x55be9bd2c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd2c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 -- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 msgr2=0x55be9bd29840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:04.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd29840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:04.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 -- 172.21.15.33:0/791592763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be9ba61620 con 0x55be9bd2cc10 2024-04-23T18:45:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd29840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fedfad9d640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fedf40142f0 tx=0x7fedf4014320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7f1dd127013505 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fede1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:05.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf40129f0 con 0x55be9bd2cc10 2024-04-23T18:45:05.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be9bc90390 con 0x55be9bd2cc10 2024-04-23T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fedf4012b90 con 0x55be9bd2cc10 2024-04-23T18:45:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fedf4012d90 con 0x55be9bd2cc10 2024-04-23T18:45:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fee03a32740 10 client.17436 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55be9bd3b660 con 0x55be9bd2cc10 2024-04-23T18:45:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.701+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be9bc90390 con 0x55be9bd2cc10 2024-04-23T18:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fedf400f040 con 0x55be9bd2cc10 2024-04-23T18:45:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 10 client.17436 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 10 client.17436 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 20 client.17436 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:05.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bd3ca10 0x55be9bd3ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:05.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 4 client.17436 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:05.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be9bc8a200 con 0x55be9bd3ca10 2024-04-23T18:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.702+0000 7fedfa59c640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bd3ca10 0x55be9bd3ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fede1ffb640 10 client.17436 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fedfa59c640 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bd3ca10 0x55be9bd3ee00 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eedbd2f70883278b server_cookie=458c21c14fc8d2b9 in_seq=0 out_seq=0 2024-04-23T18:45:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fede1ffb640 1 -- 172.21.15.33:0/791592763 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55be9bc8a200 con 0x55be9bd3ca10 2024-04-23T18:45:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fede1ffb640 10 client.17436 handle_command_reply: tid=0 2024-04-23T18:45:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 client.17436 shutdown 2024-04-23T18:45:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 msgr2=0x55be9bd29d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fedf40142f0 tx=0x7fedf4014320 comp rx=0 tx=0).stop 2024-04-23T18:45:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 shutdown_connections 2024-04-23T18:45:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bd3ca10 0x55be9bd3ee00 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd29d80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fede1ffb640 0 client.17436 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd29840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.703+0000 7fee03a32740 1 --2- 172.21.15.33:0/791592763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd2c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.704+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 >> 172.21.15.33:0/791592763 conn(0x55be9bd1df40 msgr2=0x55be9bd18cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:06.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.704+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 shutdown_connections 2024-04-23T18:45:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.704+0000 7fee03a32740 1 -- 172.21.15.33:0/791592763 wait complete. 2024-04-23T18:45:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.704+0000 7fee03a32740 20 client.17436 trim_cache size 0 max 16384 2024-04-23T18:45:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 Processor -- start 2024-04-23T18:45:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 -- start start 2024-04-23T18:45:06.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:06.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd18420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ad430 con 0x55be9bd2cc10 2024-04-23T18:45:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ab220 con 0x55be9bd22350 2024-04-23T18:45:06.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fee03a32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1acd60 con 0x55be9bd21a30 2024-04-23T18:45:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45270/0 (socket says 172.21.15.33:45270) 2024-04-23T18:45:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 -- 172.21.15.33:0/4222401178 learned_addr learned my addr 172.21.15.33:0/4222401178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:06.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd18420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 -- 172.21.15.33:0/4222401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 msgr2=0x55be9bd386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 -- 172.21.15.33:0/4222401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 msgr2=0x55be9bd18420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd18420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 -- 172.21.15.33:0/4222401178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be9bd22cf0 con 0x55be9bd2cc10 2024-04-23T18:45:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.719+0000 7fedfad9d640 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009640 tx=0x7fedf4005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc859e5a7eab77f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fede3fff640 1 -- 172.21.15.33:0/4222401178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf401a040 con 0x55be9bd2cc10 2024-04-23T18:45:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fede3fff640 1 -- 172.21.15.33:0/4222401178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4037070 con 0x55be9bd2cc10 2024-04-23T18:45:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fede3fff640 1 -- 172.21.15.33:0/4222401178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4003ca0 con 0x55be9bd2cc10 2024-04-23T18:45:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 -- 172.21.15.33:0/4222401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 msgr2=0x55be9bd44590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009640 tx=0x7fedf4005df0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 -- 172.21.15.33:0/4222401178 shutdown_connections 2024-04-23T18:45:07.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd2cc10 0x55be9bd44590 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd22350 0x55be9bd18420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 --2- 172.21.15.33:0/4222401178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd21a30 0x55be9bd386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 -- 172.21.15.33:0/4222401178 >> 172.21.15.33:0/4222401178 conn(0x55be9bd1df40 msgr2=0x55be9bd23c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:07.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 -- 172.21.15.33:0/4222401178 shutdown_connections 2024-04-23T18:45:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.720+0000 7fee03a32740 1 -- 172.21.15.33:0/4222401178 wait complete. 2024-04-23T18:45:07.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 -- start start 2024-04-23T18:45:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 Processor -- start 2024-04-23T18:45:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd22350 0x55be9bd32390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd2cc10 0x55be9bd328d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1abfc0 con 0x55be9bd21a30 2024-04-23T18:45:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ab8f0 con 0x55be9bd2cc10 2024-04-23T18:45:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ac690 con 0x55be9bd22350 2024-04-23T18:45:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fee03a32740 10 client.0 _ll_register_callbacks cb 0x7fedf40120d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45274/0 (socket says 172.21.15.33:45274) 2024-04-23T18:45:08.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 -- 172.21.15.33:0/2404276150 learned_addr learned my addr 172.21.15.33:0/2404276150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:08.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfad9d640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd2cc10 0x55be9bd328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd22350 0x55be9bd32390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:08.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 -- 172.21.15.33:0/2404276150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd22350 msgr2=0x55be9bd32390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd22350 0x55be9bd32390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 -- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd2cc10 msgr2=0x55be9bd328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd2cc10 0x55be9bd328d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:08.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.721+0000 7fedfa59c640 1 -- 172.21.15.33:0/2404276150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be9bd33100 con 0x55be9bd21a30 2024-04-23T18:45:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fedfa59c640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fedec009230 tx=0x7fedec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919fbaf854db9640 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:08.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec055020 con 0x55be9bd21a30 2024-04-23T18:45:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fedec043040 con 0x55be9bd21a30 2024-04-23T18:45:09.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4037070 con 0x55be9bd21a30 2024-04-23T18:45:09.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fedec060050 con 0x55be9bd21a30 2024-04-23T18:45:09.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 10 client.17442 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55be9bda2e80 con 0x55be9bd21a30 2024-04-23T18:45:09.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4037070 con 0x55be9bd21a30 2024-04-23T18:45:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fedec004770 con 0x55be9bd21a30 2024-04-23T18:45:09.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 10 client.17442 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 10 client.17442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 20 client.17442 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bda35a0 0x55be9bda59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:09.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 4 client.17442 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.722+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be9bc065f0 con 0x55be9bda35a0 2024-04-23T18:45:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.723+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bda35a0 0x55be9bda59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.723+0000 7fede1ffb640 10 client.17442 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.723+0000 7fedf9d9b640 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bda35a0 0x55be9bda59c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfcd87199d8e2d73 server_cookie=d16987d98908f8aa in_seq=0 out_seq=0 2024-04-23T18:45:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fede1ffb640 1 -- 172.21.15.33:0/2404276150 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be9bc065f0 con 0x55be9bda35a0 2024-04-23T18:45:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fede1ffb640 10 client.17442 handle_command_reply: tid=0 2024-04-23T18:45:10.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 client.17442 shutdown 2024-04-23T18:45:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 msgr2=0x55be9bd34fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fedec009230 tx=0x7fedec009260 comp rx=0 tx=0).stop 2024-04-23T18:45:10.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 shutdown_connections 2024-04-23T18:45:10.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55be9bda35a0 0x55be9bda59c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55be9bd2cc10 0x55be9bd328d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55be9bd22350 0x55be9bd32390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 --2- 172.21.15.33:0/2404276150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be9bd21a30 0x55be9bd34fb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:10.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fede1ffb640 0 client.17442 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:10.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.727+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 >> 172.21.15.33:0/2404276150 conn(0x55be9bd1df40 msgr2=0x55be9bd37690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:10.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 shutdown_connections 2024-04-23T18:45:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee03a32740 1 -- 172.21.15.33:0/2404276150 wait complete. 2024-04-23T18:45:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee03a32740 20 client.17442 trim_cache size 0 max 16384 2024-04-23T18:45:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:45:10.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:45:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:45:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:45:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 msgr2=0x7fede40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 0x7fede40a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fedec005ab0 tx=0x7fedec005a20 comp rx=0 tx=0).stop 2024-04-23T18:45:10.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 msgr2=0x7fedfc198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fedf000d950 tx=0x7fedf000de10 comp rx=0 tx=0).stop 2024-04-23T18:45:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 shutdown_connections 2024-04-23T18:45:11.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fede40a3dc0 0x7fede40a6280 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedfc10c550 0x7fedfc19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedfc103aa0 0x7fedfc198be0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 --2- 172.21.15.33:0/3191130752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fedfc103180 0x7fedfc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 >> 172.21.15.33:0/3191130752 conn(0x7fedfc077fd0 msgr2=0x7fedfc0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 shutdown_connections 2024-04-23T18:45:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:42:55.728+0000 7fee00c3c640 1 -- 172.21.15.33:0/3191130752 wait complete. 2024-04-23T18:45:11.437 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:03 smithi122 ceph-mon[32546]: pgmap v590: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:11.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:05 smithi122 ceph-mon[32546]: pgmap v591: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:11.511 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:07 smithi122 ceph-mon[32546]: pgmap v592: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-23T18:45:11.544 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:09 smithi122 ceph-mon[32546]: pgmap v593: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-23T18:45:11.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:10 smithi122 ceph-mon[32546]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:45:11.686 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:03 smithi106 ceph-mon[32716]: pgmap v590: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:11.719 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:05 smithi106 ceph-mon[32716]: pgmap v591: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:11.753 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:07 smithi106 ceph-mon[32716]: pgmap v592: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-23T18:45:11.794 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:09 smithi106 ceph-mon[32716]: pgmap v593: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-23T18:45:11.828 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:10 smithi106 ceph-mon[32716]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:45:11.861 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:11 smithi106 ceph-mon[32716]: pgmap v595: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:11.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:57 smithi033 ceph-mon[29274]: pgmap v522: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:11.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:42:59 smithi033 ceph-mon[29274]: pgmap v523: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:11.994 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:01 smithi033 ceph-mon[29274]: pgmap v524: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:45:12.028 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:03 smithi033 ceph-mon[29274]: pgmap v525: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:12.061 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:05 smithi033 ceph-mon[29274]: pgmap v526: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:12.095 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:07 smithi033 ceph-mon[29274]: pgmap v527: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-23T18:45:12.137 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:11 smithi122 ceph-mon[32546]: pgmap v595: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 -- 172.21.15.33:0/1704264112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 msgr2=0x7fe534110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 --2- 172.21.15.33:0/1704264112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe534110d00 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009ca0 tx=0x7fe52c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 -- 172.21.15.33:0/1704264112 shutdown_connections 2024-04-23T18:45:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 --2- 172.21.15.33:0/1704264112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe534110d00 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 --2- 172.21.15.33:0/1704264112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534101680 0x7fe53410e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 --2- 172.21.15.33:0/1704264112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534100d60 0x7fe534101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 -- 172.21.15.33:0/1704264112 >> 172.21.15.33:0/1704264112 conn(0x7fe534077fd0 msgr2=0x7fe5340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 -- 172.21.15.33:0/1704264112 shutdown_connections 2024-04-23T18:45:12.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.828+0000 7fe53b343640 1 -- 172.21.15.33:0/1704264112 wait complete. 2024-04-23T18:45:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 Processor -- start 2024-04-23T18:45:12.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 -- start start 2024-04-23T18:45:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534100d60 0x7fe5341a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534101680 0x7fe5341a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341143d0 con 0x7fe53410e910 2024-04-23T18:45:12.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe534114250 con 0x7fe534100d60 2024-04-23T18:45:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe53b343640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe534114550 con 0x7fe534101680 2024-04-23T18:45:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60182/0 (socket says 172.21.15.33:60182) 2024-04-23T18:45:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 -- 172.21.15.33:0/400239906 learned_addr learned my addr 172.21.15.33:0/400239906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5390df640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534100d60 0x7fe5341a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5388de640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534101680 0x7fe5341a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 -- 172.21.15.33:0/400239906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534101680 msgr2=0x7fe5341a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534101680 0x7fe5341a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 -- 172.21.15.33:0/400239906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534100d60 msgr2=0x7fe5341a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:13.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534100d60 0x7fe5341a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.829+0000 7fe5398e0640 1 -- 172.21.15.33:0/400239906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53419af80 con 0x7fe53410e910 2024-04-23T18:45:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe5398e0640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0098d0 tx=0x7fe52c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe52a7fc640 1 -- 172.21.15.33:0/400239906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c003b70 con 0x7fe53410e910 2024-04-23T18:45:13.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53419b210 con 0x7fe53410e910 2024-04-23T18:45:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe52a7fc640 1 -- 172.21.15.33:0/400239906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe52c003d10 con 0x7fe53410e910 2024-04-23T18:45:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe52a7fc640 1 -- 172.21.15.33:0/400239906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c02fa50 con 0x7fe53410e910 2024-04-23T18:45:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.830+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5341a7490 con 0x7fe53410e910 2024-04-23T18:45:13.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.832+0000 7fe52a7fc640 1 -- 172.21.15.33:0/400239906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe52c004580 con 0x7fe53410e910 2024-04-23T18:45:13.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 Processor -- start 2024-04-23T18:45:13.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 -- start start 2024-04-23T18:45:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b53a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560694ab8970 con 0x560694b5e800 2024-04-23T18:45:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560694a554a0 con 0x560694b53f40 2024-04-23T18:45:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe53e057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560694ab4930 con 0x560694b53620 2024-04-23T18:45:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe52a7fc640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 0x7fe5200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60198/0 (socket says 172.21.15.33:60198) 2024-04-23T18:45:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5390df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5388de640 1 -- 172.21.15.33:0/1387229957 learned_addr learned my addr 172.21.15.33:0/1387229957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5388de640 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b53a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe5398e0640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 0x7fe5200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:14.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.833+0000 7fe52a7fc640 1 -- 172.21.15.33:0/400239906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fe52c0f1b50 con 0x7fe53410e910 2024-04-23T18:45:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 -- 172.21.15.33:0/1387229957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 msgr2=0x560694b53a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:14.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b53a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 -- 172.21.15.33:0/1387229957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 msgr2=0x560694b5e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 -- 172.21.15.33:0/1387229957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606947cb210 con 0x560694b5e800 2024-04-23T18:45:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5390df640 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:14.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5388de640 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe524014790 tx=0x7fe524009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38139a87bdbf0022 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe5398e0640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 0x7fe5200a6280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ef8b0 tx=0x7fe52c04e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe51f7fe640 1 -- 172.21.15.33:0/1387229957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524014110 con 0x560694b5e800 2024-04-23T18:45:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe51f7fe640 1 -- 172.21.15.33:0/1387229957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5240142b0 con 0x560694b5e800 2024-04-23T18:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.834+0000 7fe51f7fe640 1 -- 172.21.15.33:0/1387229957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5240035f0 con 0x560694b5e800 2024-04-23T18:45:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- 172.21.15.33:0/1387229957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 msgr2=0x560694b60bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:14.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe524014790 tx=0x7fe524009650 comp rx=0 tx=0).stop 2024-04-23T18:45:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- 172.21.15.33:0/1387229957 shutdown_connections 2024-04-23T18:45:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b60bf0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 --2- 172.21.15.33:0/1387229957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b53a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- 172.21.15.33:0/1387229957 >> 172.21.15.33:0/1387229957 conn(0x560694b4fb30 msgr2=0x560694b4ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:15.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- 172.21.15.33:0/1387229957 shutdown_connections 2024-04-23T18:45:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- 172.21.15.33:0/1387229957 wait complete. 2024-04-23T18:45:15.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 -- start start 2024-04-23T18:45:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.835+0000 7fe53e057740 1 Processor -- start 2024-04-23T18:45:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560694b61430 con 0x560694b5e800 2024-04-23T18:45:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560694a530b0 con 0x560694b53f40 2024-04-23T18:45:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560694b615b0 con 0x560694b53620 2024-04-23T18:45:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe53e057740 10 client.0 _ll_register_callbacks cb 0x560694a40d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60204/0 (socket says 172.21.15.33:60204) 2024-04-23T18:45:15.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 -- 172.21.15.33:0/2327775983 learned_addr learned my addr 172.21.15.33:0/2327775983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:15.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5390df640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 -- 172.21.15.33:0/2327775983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 msgr2=0x560694b5e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b5e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 -- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 msgr2=0x560694b5b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:15.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 -- 172.21.15.33:0/2327775983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560694897620 con 0x560694b5e800 2024-04-23T18:45:15.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe5388de640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe52400fa30 tx=0x7fe524005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6866af97398ab9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:16.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.836+0000 7fe51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:16.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524023070 con 0x560694b5e800 2024-04-23T18:45:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fe524004430 con 0x560694b5e800 2024-04-23T18:45:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560694ac6390 con 0x560694b5e800 2024-04-23T18:45:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe524014110 con 0x560694b5e800 2024-04-23T18:45:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 10 client.17454 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560694b6d360 con 0x560694b5e800 2024-04-23T18:45:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560694ac6390 con 0x560694b5e800 2024-04-23T18:45:16.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fe52402d060 con 0x560694b5e800 2024-04-23T18:45:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 10 client.17454 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:16.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 20 client.17454 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694b6e710 0x560694b70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 4 client.17454 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.837+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560694ac0200 con 0x560694b6e710 2024-04-23T18:45:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.838+0000 7fe5390df640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694b6e710 0x560694b70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.838+0000 7fe5390df640 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694b6e710 0x560694b70b00 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebbebfccc1571c8 server_cookie=9806cf26edfb3dc2 in_seq=0 out_seq=0 2024-04-23T18:45:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.838+0000 7fe51cff9640 10 client.17454 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe51cff9640 1 -- 172.21.15.33:0/2327775983 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560694ac0200 con 0x560694b6e710 2024-04-23T18:45:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe51cff9640 10 client.17454 handle_command_reply: tid=0 2024-04-23T18:45:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 client.17454 shutdown 2024-04-23T18:45:16.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 msgr2=0x560694b5b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe52400fa30 tx=0x7fe524005f70 comp rx=0 tx=0).stop 2024-04-23T18:45:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 shutdown_connections 2024-04-23T18:45:17.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694b6e710 0x560694b70b00 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b5b9c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe51cff9640 0 client.17454 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:17.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53f40 0x560694b5b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 --2- 172.21.15.33:0/2327775983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53620 0x560694b5e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 >> 172.21.15.33:0/2327775983 conn(0x560694b4fb30 msgr2=0x560694b55860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:17.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 shutdown_connections 2024-04-23T18:45:17.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 1 -- 172.21.15.33:0/2327775983 wait complete. 2024-04-23T18:45:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.839+0000 7fe53e057740 20 client.17454 trim_cache size 0 max 16384 2024-04-23T18:45:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 Processor -- start 2024-04-23T18:45:17.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 -- start start 2024-04-23T18:45:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694b6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694b4eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341b17f0 con 0x560694b5e800 2024-04-23T18:45:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341af5e0 con 0x560694b53620 2024-04-23T18:45:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe53e057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341b1120 con 0x560694b53f40 2024-04-23T18:45:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60216/0 (socket says 172.21.15.33:60216) 2024-04-23T18:45:18.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.854+0000 7fe5388de640 1 -- 172.21.15.33:0/2836646903 learned_addr learned my addr 172.21.15.33:0/2836646903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694b6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5390df640 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694b4eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:18.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 -- 172.21.15.33:0/2836646903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 msgr2=0x560694b4eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694b4eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 -- 172.21.15.33:0/2836646903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 msgr2=0x560694b6a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694b6a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:18.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 -- 172.21.15.33:0/2836646903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560694b5bd30 con 0x560694b5e800 2024-04-23T18:45:18.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe5388de640 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe524009920 tx=0x7fe524014470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3deda26e823fbd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:18.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe51effd640 1 -- 172.21.15.33:0/2836646903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524037070 con 0x560694b5e800 2024-04-23T18:45:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe51effd640 1 -- 172.21.15.33:0/2836646903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe524003d50 con 0x560694b5e800 2024-04-23T18:45:18.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe51effd640 1 -- 172.21.15.33:0/2836646903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524023040 con 0x560694b5e800 2024-04-23T18:45:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe53e057740 1 -- 172.21.15.33:0/2836646903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 msgr2=0x560694b763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.855+0000 7fe53e057740 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe524009920 tx=0x7fe524014470 comp rx=0 tx=0).stop 2024-04-23T18:45:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- 172.21.15.33:0/2836646903 shutdown_connections 2024-04-23T18:45:19.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b763a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694b4eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- 172.21.15.33:0/2836646903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694b6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- 172.21.15.33:0/2836646903 >> 172.21.15.33:0/2836646903 conn(0x560694b4fb30 msgr2=0x560694b4e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:19.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- 172.21.15.33:0/2836646903 shutdown_connections 2024-04-23T18:45:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- 172.21.15.33:0/2836646903 wait complete. 2024-04-23T18:45:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- start start 2024-04-23T18:45:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 Processor -- start 2024-04-23T18:45:19.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694bccf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694bcd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341b0380 con 0x560694b5e800 2024-04-23T18:45:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341afcb0 con 0x560694b53620 2024-04-23T18:45:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5341b0a50 con 0x560694b53f40 2024-04-23T18:45:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe53e057740 10 client.0 _ll_register_callbacks cb 0x7fe524014690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.856+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60222/0 (socket says 172.21.15.33:60222) 2024-04-23T18:45:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 -- 172.21.15.33:0/1681908386 learned_addr learned my addr 172.21.15.33:0/1681908386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:20.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694bccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5390df640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694bcd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 -- 172.21.15.33:0/1681908386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 msgr2=0x560694bcd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:20.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694bcd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 -- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 msgr2=0x560694bccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694bccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 -- 172.21.15.33:0/1681908386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560694b64160 con 0x560694b5e800 2024-04-23T18:45:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe5388de640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe524009380 tx=0x7fe5240058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a61d9eed2f429f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:20.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe51cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:20.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe524023070 con 0x560694b5e800 2024-04-23T18:45:20.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8498+0+0 (secure 0 0 0) 0x7fe52400f820 con 0x560694b5e800 2024-04-23T18:45:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560694a3c5f0 con 0x560694b5e800 2024-04-23T18:45:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe524037040 con 0x560694b5e800 2024-04-23T18:45:20.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe53e057740 10 client.17460 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:20.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.857+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x560694bd4740 con 0x560694b5e800 2024-04-23T18:45:20.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560694a3c5f0 con 0x560694b5e800 2024-04-23T18:45:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fe52403d040 con 0x560694b5e800 2024-04-23T18:45:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 10 client.17460 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 10 client.17460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 20 client.17460 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694bd4a10 0x560694bd6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 4 client.17460 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560694a89380 con 0x560694bd4a10 2024-04-23T18:45:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe5390df640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694bd4a10 0x560694bd6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe51cff9640 10 client.17460 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:21.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.858+0000 7fe5390df640 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694bd4a10 0x560694bd6e30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9910ba7fce7b5cf server_cookie=6b36eb6b18598426 in_seq=0 out_seq=0 2024-04-23T18:45:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe51cff9640 1 -- 172.21.15.33:0/1681908386 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560694a89380 con 0x560694bd4a10 2024-04-23T18:45:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe51cff9640 10 client.17460 handle_command_reply: tid=0 2024-04-23T18:45:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 client.17460 shutdown 2024-04-23T18:45:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 msgr2=0x560694b63a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe524009380 tx=0x7fe5240058d0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 shutdown_connections 2024-04-23T18:45:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560694bd4a10 0x560694bd6e30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:21.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560694b5e800 0x560694b63a70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560694b53f40 0x560694bcd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe51cff9640 0 client.17460 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 --2- 172.21.15.33:0/1681908386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560694b53620 0x560694bccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 >> 172.21.15.33:0/1681908386 conn(0x560694b4fb30 msgr2=0x560694b4e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:22.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.865+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 shutdown_connections 2024-04-23T18:45:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53e057740 1 -- 172.21.15.33:0/1681908386 wait complete. 2024-04-23T18:45:22.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53e057740 20 client.17460 trim_cache size 0 max 16384 2024-04-23T18:45:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:45:22.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:45:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:45:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:45:22.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 msgr2=0x7fe5200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 0x7fe5200a6280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ef8b0 tx=0x7fe52c04e000 comp rx=0 tx=0).stop 2024-04-23T18:45:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 msgr2=0x7fe53419a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0098d0 tx=0x7fe52c0319f0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 shutdown_connections 2024-04-23T18:45:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe5200a3dc0 0x7fe5200a6280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe53410e910 0x7fe53419a7b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe534101680 0x7fe5341a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 --2- 172.21.15.33:0/400239906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe534100d60 0x7fe5341a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 >> 172.21.15.33:0/400239906 conn(0x7fe534077fd0 msgr2=0x7fe53410fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 shutdown_connections 2024-04-23T18:45:23.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:08.866+0000 7fe53b343640 1 -- 172.21.15.33:0/400239906 wait complete. 2024-04-23T18:45:23.256 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:13 smithi106 ceph-mon[32716]: pgmap v596: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:23.342 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:15 smithi106 ceph-mon[32716]: pgmap v597: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:23.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:16 smithi106 ceph-mon[32716]: pgmap v598: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:45:23.462 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:16 smithi106 ceph-mon[32716]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:45:23.512 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:18 smithi106 ceph-mon[32716]: pgmap v600: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:45:23.562 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:20 smithi106 ceph-mon[32716]: pgmap v601: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:45:23.612 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:13 smithi122 ceph-mon[32546]: pgmap v596: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:23.662 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:15 smithi122 ceph-mon[32546]: pgmap v597: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:23.712 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:16 smithi122 ceph-mon[32546]: pgmap v598: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:45:23.762 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:16 smithi122 ceph-mon[32546]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:45:23.812 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:18 smithi122 ceph-mon[32546]: pgmap v600: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:45:23.862 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:20 smithi122 ceph-mon[32546]: pgmap v601: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:45:23.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:22 smithi122 ceph-mon[32546]: pgmap v602: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:23.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:09 smithi033 ceph-mon[29274]: pgmap v528: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:45:24.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:10 smithi033 ceph-mon[29274]: osdmap e115: 12 total, 12 up, 12 in 2024-04-23T18:45:24.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:11 smithi033 ceph-mon[29274]: pgmap v530: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:45:24.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:13 smithi033 ceph-mon[29274]: pgmap v531: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:45:24.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:15 smithi033 ceph-mon[29274]: pgmap v532: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 31 op/s 2024-04-23T18:45:24.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:16 smithi033 ceph-mon[29274]: pgmap v533: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:45:24.332 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:16 smithi033 ceph-mon[29274]: osdmap e116: 12 total, 12 up, 12 in 2024-04-23T18:45:24.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:18 smithi033 ceph-mon[29274]: pgmap v535: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-23T18:45:24.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:20 smithi033 ceph-mon[29274]: pgmap v536: 97 pgs: 97 active+clean; 15 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-23T18:45:24.483 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:22 smithi106 ceph-mon[32716]: pgmap v602: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 -- 172.21.15.33:0/682160154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 msgr2=0x7f14881025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 --2- 172.21.15.33:0/682160154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881025a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1478009900 tx=0x7f147802a410 comp rx=0 tx=0).stop 2024-04-23T18:45:24.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 -- 172.21.15.33:0/682160154 shutdown_connections 2024-04-23T18:45:24.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 --2- 172.21.15.33:0/682160154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 0x7f148810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 --2- 172.21.15.33:0/682160154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1488102b70 0x7f1488106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 --2- 172.21.15.33:0/682160154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881025a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 -- 172.21.15.33:0/682160154 >> 172.21.15.33:0/682160154 conn(0x7f14880fda20 msgr2=0x7f14880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 -- 172.21.15.33:0/682160154 shutdown_connections 2024-04-23T18:45:25.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.965+0000 7f148df8e640 1 -- 172.21.15.33:0/682160154 wait complete. 2024-04-23T18:45:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 Processor -- start 2024-04-23T18:45:25.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 -- start start 2024-04-23T18:45:25.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 0x7f1488198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148819cfb0 0x7f148819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14881121f0 con 0x7f14881021c0 2024-04-23T18:45:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112070 con 0x7f1488107290 2024-04-23T18:45:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f148df8e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1488112370 con 0x7f148819cfb0 2024-04-23T18:45:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45634/0 (socket says 172.21.15.33:45634) 2024-04-23T18:45:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.966+0000 7f14877fe640 1 -- 172.21.15.33:0/3045457272 learned_addr learned my addr 172.21.15.33:0/3045457272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f1487fff640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148819cfb0 0x7f148819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f1486ffd640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 0x7f1488198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 -- 172.21.15.33:0/3045457272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148819cfb0 msgr2=0x7f148819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148819cfb0 0x7f148819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:25.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 -- 172.21.15.33:0/3045457272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 msgr2=0x7f1488198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 0x7f1488198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 -- 172.21.15.33:0/3045457272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148819daa0 con 0x7f14881021c0 2024-04-23T18:45:26.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f14877fe640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1478005940 tx=0x7f1478004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f1484ff9640 1 -- 172.21.15.33:0/3045457272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147803f030 con 0x7f14881021c0 2024-04-23T18:45:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f1484ff9640 1 -- 172.21.15.33:0/3045457272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478003aa0 con 0x7f14881021c0 2024-04-23T18:45:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148819dd30 con 0x7f14881021c0 2024-04-23T18:45:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f1484ff9640 1 -- 172.21.15.33:0/3045457272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478035070 con 0x7f14881021c0 2024-04-23T18:45:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.967+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14881a5550 con 0x7f14881021c0 2024-04-23T18:45:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1484ff9640 1 -- 172.21.15.33:0/3045457272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1478042030 con 0x7f14881021c0 2024-04-23T18:45:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 Processor -- start 2024-04-23T18:45:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 -- start start 2024-04-23T18:45:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5aff970 con 0x55ffa5b9a620 2024-04-23T18:45:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5a9c4a0 con 0x55ffa5ba5800 2024-04-23T18:45:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.970+0000 7f1490d7c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5afb930 con 0x55ffa5b9af40 2024-04-23T18:45:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1484ff9640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 0x7f14700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:26.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f14877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45636/0 (socket says 172.21.15.33:45636) 2024-04-23T18:45:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 -- 172.21.15.33:0/2879224701 learned_addr learned my addr 172.21.15.33:0/2879224701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 0x7f14700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1486ffd640 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1484ff9640 1 -- 172.21.15.33:0/3045457272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f14780f0e70 con 0x7f14881021c0 2024-04-23T18:45:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 -- 172.21.15.33:0/2879224701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 msgr2=0x55ffa5ba52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 -- 172.21.15.33:0/2879224701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 msgr2=0x55ffa5ba7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 -- 172.21.15.33:0/2879224701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa5812210 con 0x55ffa5b9a620 2024-04-23T18:45:27.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1480012d00 tx=0x7f1480010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae82b792d5bc09b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.971+0000 7f1487fff640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 0x7f14700a6300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1480010c40 tx=0x7f148000c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:27.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f146dffb640 1 -- 172.21.15.33:0/2879224701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480024070 con 0x55ffa5b9a620 2024-04-23T18:45:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f146dffb640 1 -- 172.21.15.33:0/2879224701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1480009d20 con 0x55ffa5b9a620 2024-04-23T18:45:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f146dffb640 1 -- 172.21.15.33:0/2879224701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480005280 con 0x55ffa5b9a620 2024-04-23T18:45:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 -- 172.21.15.33:0/2879224701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 msgr2=0x55ffa5b9aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:27.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1480012d00 tx=0x7f1480010740 comp rx=0 tx=0).stop 2024-04-23T18:45:27.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 -- 172.21.15.33:0/2879224701 shutdown_connections 2024-04-23T18:45:27.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 --2- 172.21.15.33:0/2879224701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5b9aa00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:27.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 -- 172.21.15.33:0/2879224701 >> 172.21.15.33:0/2879224701 conn(0x55ffa5b96b30 msgr2=0x55ffa5b96f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 -- 172.21.15.33:0/2879224701 shutdown_connections 2024-04-23T18:45:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.972+0000 7f1490d7c740 1 -- 172.21.15.33:0/2879224701 wait complete. 2024-04-23T18:45:27.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 -- start start 2024-04-23T18:45:27.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 Processor -- start 2024-04-23T18:45:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:27.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:28.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5ba83d0 con 0x55ffa5b9a620 2024-04-23T18:45:28.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5a9a0b0 con 0x55ffa5ba5800 2024-04-23T18:45:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffa5ba8550 con 0x55ffa5b9af40 2024-04-23T18:45:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f1490d7c740 10 client.0 _ll_register_callbacks cb 0x55ffa5a87d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45642/0 (socket says 172.21.15.33:45642) 2024-04-23T18:45:28.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.973+0000 7f14877fe640 1 -- 172.21.15.33:0/3970998731 learned_addr learned my addr 172.21.15.33:0/3970998731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:28.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1486ffd640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1487fff640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:28.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 -- 172.21.15.33:0/3970998731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 msgr2=0x55ffa5ba24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 -- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 msgr2=0x55ffa5ba2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:28.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 -- 172.21.15.33:0/3970998731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa58de620 con 0x55ffa5b9a620 2024-04-23T18:45:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1487fff640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f14877fe640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f14780eebb0 tx=0x7f147808b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77145201d1d87cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478049040 con 0x55ffa5b9a620 2024-04-23T18:45:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f14780b9470 con 0x55ffa5b9a620 2024-04-23T18:45:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ffa5b0d390 con 0x55ffa5b9a620 2024-04-23T18:45:28.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f14780bce00 con 0x55ffa5b9a620 2024-04-23T18:45:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1490d7c740 10 client.17472 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.974+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ffa5bb4380 con 0x55ffa5b9a620 2024-04-23T18:45:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ffa5b0d390 con 0x55ffa5b9a620 2024-04-23T18:45:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f14780bd060 con 0x55ffa5b9a620 2024-04-23T18:45:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 10 client.17472 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 10 client.17472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:29.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 20 client.17472 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:29.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5bb57e0 0x55ffa5bb7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 4 client.17472 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa5b07200 con 0x55ffa5bb57e0 2024-04-23T18:45:29.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.975+0000 7f1486ffd640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5bb57e0 0x55ffa5bb7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1486ffd640 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5bb57e0 0x55ffa5bb7c60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa977c74d47c11c server_cookie=7404abaced34e642 in_seq=0 out_seq=0 2024-04-23T18:45:29.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1466ffd640 10 client.17472 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1466ffd640 1 -- 172.21.15.33:0/3970998731 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffa5b07200 con 0x55ffa5bb57e0 2024-04-23T18:45:29.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1466ffd640 10 client.17472 handle_command_reply: tid=0 2024-04-23T18:45:29.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1490d7c740 1 client.17472 shutdown 2024-04-23T18:45:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 msgr2=0x55ffa5ba51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.976+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f14780eebb0 tx=0x7f147808b7b0 comp rx=0 tx=0).stop 2024-04-23T18:45:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 shutdown_connections 2024-04-23T18:45:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5bb57e0 0x55ffa5bb7c60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1466ffd640 0 client.17472 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 --2- 172.21.15.33:0/3970998731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba51b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:30.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 >> 172.21.15.33:0/3970998731 conn(0x55ffa5b96b30 msgr2=0x55ffa5b9c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:30.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 shutdown_connections 2024-04-23T18:45:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 1 -- 172.21.15.33:0/3970998731 wait complete. 2024-04-23T18:45:30.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.977+0000 7f1490d7c740 20 client.17472 trim_cache size 0 max 16384 2024-04-23T18:45:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 Processor -- start 2024-04-23T18:45:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 -- start start 2024-04-23T18:45:30.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5b95b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:30.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14881af5d0 con 0x55ffa5b9a620 2024-04-23T18:45:30.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ad3c0 con 0x55ffa5ba5800 2024-04-23T18:45:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f1490d7c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14881aef00 con 0x55ffa5b9af40 2024-04-23T18:45:30.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45650/0 (socket says 172.21.15.33:45650) 2024-04-23T18:45:30.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.992+0000 7f14877fe640 1 -- 172.21.15.33:0/561920313 learned_addr learned my addr 172.21.15.33:0/561920313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f1487fff640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f1486ffd640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5b95b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 -- 172.21.15.33:0/561920313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 msgr2=0x55ffa5b95b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:31.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5b95b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 -- 172.21.15.33:0/561920313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 msgr2=0x55ffa5ba4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 -- 172.21.15.33:0/561920313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa5ba1450 con 0x55ffa5b9a620 2024-04-23T18:45:31.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f1487fff640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:31.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f14877fe640 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f1478003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e5baed0b11b8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f146d7fa640 1 -- 172.21.15.33:0/561920313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478049050 con 0x55ffa5b9a620 2024-04-23T18:45:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f146d7fa640 1 -- 172.21.15.33:0/561920313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14780c11b0 con 0x55ffa5b9a620 2024-04-23T18:45:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.993+0000 7f146d7fa640 1 -- 172.21.15.33:0/561920313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780ee830 con 0x55ffa5b9a620 2024-04-23T18:45:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- 172.21.15.33:0/561920313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 msgr2=0x55ffa5bb1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1478006010 tx=0x7f1478003f10 comp rx=0 tx=0).stop 2024-04-23T18:45:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- 172.21.15.33:0/561920313 shutdown_connections 2024-04-23T18:45:31.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5ba5800 0x55ffa5ba4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5b9af40 0x55ffa5b95b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 --2- 172.21.15.33:0/561920313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5bb1290 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- 172.21.15.33:0/561920313 >> 172.21.15.33:0/561920313 conn(0x55ffa5b96b30 msgr2=0x55ffa5b95420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- 172.21.15.33:0/561920313 shutdown_connections 2024-04-23T18:45:31.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- 172.21.15.33:0/561920313 wait complete. 2024-04-23T18:45:31.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 -- start start 2024-04-23T18:45:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.994+0000 7f1490d7c740 1 Processor -- start 2024-04-23T18:45:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5ba5800 0x55ffa5c1ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ae160 con 0x55ffa5b9a620 2024-04-23T18:45:32.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ada90 con 0x55ffa5b9af40 2024-04-23T18:45:32.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14881ae830 con 0x55ffa5ba5800 2024-04-23T18:45:32.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 10 client.0 _ll_register_callbacks cb 0x7f1478009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:32.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45664/0 (socket says 172.21.15.33:45664) 2024-04-23T18:45:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 -- 172.21.15.33:0/1376726587 learned_addr learned my addr 172.21.15.33:0/1376726587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1486ffd640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1487fff640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5ba5800 0x55ffa5c1ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 -- 172.21.15.33:0/1376726587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5ba5800 msgr2=0x55ffa5c1ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:32.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5ba5800 0x55ffa5c1ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 -- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 msgr2=0x55ffa5ba9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 -- 172.21.15.33:0/1376726587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa5ba7f60 con 0x55ffa5b9a620 2024-04-23T18:45:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1486ffd640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f14877fe640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f14780c5540 tx=0x7f1478003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae005793f8eccc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:33.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1466ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:33.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780db060 con 0x55ffa5b9a620 2024-04-23T18:45:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f14780c57a0 con 0x55ffa5b9a620 2024-04-23T18:45:33.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14780c11b0 con 0x55ffa5b9a620 2024-04-23T18:45:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1478049070 con 0x55ffa5b9a620 2024-04-23T18:45:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 10 client.17478 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.995+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ffa5ba7f60 con 0x55ffa5b9a620 2024-04-23T18:45:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14780c11b0 con 0x55ffa5b9a620 2024-04-23T18:45:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f14780bc370 con 0x55ffa5b9a620 2024-04-23T18:45:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 10 client.17478 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:33.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 10 client.17478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 20 client.17478 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5c1c610 0x55ffa5c1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 4 client.17478 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:34.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffa5a835f0 con 0x55ffa5c1c610 2024-04-23T18:45:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1486ffd640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5c1c610 0x55ffa5c1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1466ffd640 10 client.17478 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:34.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:21.996+0000 7f1486ffd640 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5c1c610 0x55ffa5c1ea00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcb789f3b8adcd2 server_cookie=5d6deddd428080f6 in_seq=0 out_seq=0 2024-04-23T18:45:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1466ffd640 1 -- 172.21.15.33:0/1376726587 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffa5a835f0 con 0x55ffa5c1c610 2024-04-23T18:45:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1466ffd640 10 client.17478 handle_command_reply: tid=0 2024-04-23T18:45:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 client.17478 shutdown 2024-04-23T18:45:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 msgr2=0x55ffa5ba97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f14780c5540 tx=0x7f1478003e80 comp rx=0 tx=0).stop 2024-04-23T18:45:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 shutdown_connections 2024-04-23T18:45:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ffa5c1c610 0x55ffa5c1ea00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffa5ba5800 0x55ffa5c1ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa5b9af40 0x55ffa5ba9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:34.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1466ffd640 0 client.17478 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:34.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 --2- 172.21.15.33:0/1376726587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffa5b9a620 0x55ffa5ba97e0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 >> 172.21.15.33:0/1376726587 conn(0x55ffa5b96b30 msgr2=0x55ffa5b95cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 shutdown_connections 2024-04-23T18:45:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 1 -- 172.21.15.33:0/1376726587 wait complete. 2024-04-23T18:45:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.003+0000 7f1490d7c740 20 client.17478 trim_cache size 0 max 16384 2024-04-23T18:45:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:45:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:45:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:45:35.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:45:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 msgr2=0x7f14700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 0x7f14700a6300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1480010c40 tx=0x7f148000c000 comp rx=0 tx=0).stop 2024-04-23T18:45:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 msgr2=0x7f14881986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:35.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1478005940 tx=0x7f1478004270 comp rx=0 tx=0).stop 2024-04-23T18:45:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 shutdown_connections 2024-04-23T18:45:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f14700a3e40 0x7f14700a6300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f148819cfb0 0x7f148819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1488107290 0x7f1488198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 --2- 172.21.15.33:0/3045457272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881021c0 0x7f14881986f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 >> 172.21.15.33:0/3045457272 conn(0x7f14880fda20 msgr2=0x7f14880fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:35.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 shutdown_connections 2024-04-23T18:45:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:22.004+0000 7f148df8e640 1 -- 172.21.15.33:0/3045457272 wait complete. 2024-04-23T18:45:35.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:22 smithi033 ceph-mon[29274]: pgmap v537: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:35.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:24 smithi033 ceph-mon[29274]: pgmap v538: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:36.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:26 smithi033 ceph-mon[29274]: pgmap v539: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:45:36.036 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:28 smithi033 ceph-mon[29274]: pgmap v540: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.070 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:30 smithi033 ceph-mon[29274]: pgmap v541: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:32 smithi033 ceph-mon[29274]: pgmap v542: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:45:36.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:24 smithi106 ceph-mon[32716]: pgmap v603: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:36.203 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:26 smithi106 ceph-mon[32716]: pgmap v604: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:45:36.259 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:28 smithi106 ceph-mon[32716]: pgmap v605: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.303 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:30 smithi106 ceph-mon[32716]: pgmap v606: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.337 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:32 smithi106 ceph-mon[32716]: pgmap v607: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:45:36.453 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:34 smithi106 ceph-mon[32716]: pgmap v608: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:24 smithi122 ceph-mon[32546]: pgmap v603: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:45:36.520 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:26 smithi122 ceph-mon[32546]: pgmap v604: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:45:36.553 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:28 smithi122 ceph-mon[32546]: pgmap v605: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.587 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:30 smithi122 ceph-mon[32546]: pgmap v606: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:32 smithi122 ceph-mon[32546]: pgmap v607: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:45:36.653 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:34 smithi122 ceph-mon[32546]: pgmap v608: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:36.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 -- 172.21.15.33:0/781259484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 msgr2=0x7fc5a010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 --2- 172.21.15.33:0/781259484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a010bfe0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc588009900 tx=0x7fc58802a480 comp rx=0 tx=0).stop 2024-04-23T18:45:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 -- 172.21.15.33:0/781259484 shutdown_connections 2024-04-23T18:45:36.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 --2- 172.21.15.33:0/781259484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a010c5b0 0x7fc5a010e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 --2- 172.21.15.33:0/781259484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a010bfe0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 --2- 172.21.15.33:0/781259484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a01031e0 0x7fc5a01035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 -- 172.21.15.33:0/781259484 >> 172.21.15.33:0/781259484 conn(0x7fc5a0077f60 msgr2=0x7fc5a0078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 -- 172.21.15.33:0/781259484 shutdown_connections 2024-04-23T18:45:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.104+0000 7fc5a5108640 1 -- 172.21.15.33:0/781259484 wait complete. 2024-04-23T18:45:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 Processor -- start 2024-04-23T18:45:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 -- start start 2024-04-23T18:45:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a010c5b0 0x7fc5a019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01121f0 con 0x7fc5a0103b00 2024-04-23T18:45:37.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0112070 con 0x7fc5a010c5b0 2024-04-23T18:45:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc5a5108640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a0112370 con 0x7fc5a01031e0 2024-04-23T18:45:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc59ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35032/0 (socket says 172.21.15.33:35032) 2024-04-23T18:45:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 -- 172.21.15.33:0/3322738960 learned_addr learned my addr 172.21.15.33:0/3322738960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:37.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.105+0000 7fc59ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:58168/0 (socket says 172.21.15.33:58168) 2024-04-23T18:45:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc59f59e640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a010c5b0 0x7fc5a019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:37.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 -- 172.21.15.33:0/3322738960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 msgr2=0x7fc5a0198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:37.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 -- 172.21.15.33:0/3322738960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a010c5b0 msgr2=0x7fc5a019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a010c5b0 0x7fc5a019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 -- 172.21.15.33:0/3322738960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a019d710 con 0x7fc5a0103b00 2024-04-23T18:45:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc597fff640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc588005940 tx=0x7fc5880040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:38.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc59cd99640 1 -- 172.21.15.33:0/3322738960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58803f030 con 0x7fc5a0103b00 2024-04-23T18:45:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc59cd99640 1 -- 172.21.15.33:0/3322738960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5880044c0 con 0x7fc5a0103b00 2024-04-23T18:45:38.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a019d9a0 con 0x7fc5a0103b00 2024-04-23T18:45:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc59cd99640 1 -- 172.21.15.33:0/3322738960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588035070 con 0x7fc5a0103b00 2024-04-23T18:45:38.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.106+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a01a5290 con 0x7fc5a0103b00 2024-04-23T18:45:38.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.109+0000 7fc59cd99640 1 -- 172.21.15.33:0/3322738960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc588042030 con 0x7fc5a0103b00 2024-04-23T18:45:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.109+0000 7fc5a7e1d740 1 Processor -- start 2024-04-23T18:45:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.109+0000 7fc5a7e1d740 1 -- start start 2024-04-23T18:45:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b8c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:38.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557859ae6970 con 0x557859b8c800 2024-04-23T18:45:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557859a834a0 con 0x557859b81f40 2024-04-23T18:45:38.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557859ae2930 con 0x557859b81620 2024-04-23T18:45:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35036/0 (socket says 172.21.15.33:35036) 2024-04-23T18:45:38.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc597fff640 1 -- 172.21.15.33:0/2075086124 learned_addr learned my addr 172.21.15.33:0/2075086124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc59cd99640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 0x7fc58c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b8c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc59f59e640 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.110+0000 7fc59f59e640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 0x7fc58c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:39.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 -- 172.21.15.33:0/2075086124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 msgr2=0x557859b81a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b81a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 -- 172.21.15.33:0/2075086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 msgr2=0x557859b8c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b8c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc59cd99640 1 -- 172.21.15.33:0/3322738960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fc5880f0bc0 con 0x7fc5a0103b00 2024-04-23T18:45:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 -- 172.21.15.33:0/2075086124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578597f9210 con 0x557859b8c800 2024-04-23T18:45:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc597fff640 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc58808aea0 tx=0x7fc58802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c987b8beb4d0b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc59f59e640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 0x7fc58c0a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc5a006b1f0 tx=0x7fc598009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc5957fa640 1 -- 172.21.15.33:0/2075086124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588049070 con 0x557859b8c800 2024-04-23T18:45:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc5957fa640 1 -- 172.21.15.33:0/2075086124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5880bcba0 con 0x557859b8c800 2024-04-23T18:45:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.111+0000 7fc5957fa640 1 -- 172.21.15.33:0/2075086124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880b88f0 con 0x557859b8c800 2024-04-23T18:45:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2075086124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 msgr2=0x557859b8ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fc58808aea0 tx=0x7fc58802a480 comp rx=0 tx=0).stop 2024-04-23T18:45:39.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2075086124 shutdown_connections 2024-04-23T18:45:39.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b8ebf0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b8c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2075086124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b81a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2075086124 >> 172.21.15.33:0/2075086124 conn(0x557859b7db30 msgr2=0x557859b7df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2075086124 shutdown_connections 2024-04-23T18:45:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2075086124 wait complete. 2024-04-23T18:45:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 -- start start 2024-04-23T18:45:40.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.112+0000 7fc5a7e1d740 1 Processor -- start 2024-04-23T18:45:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b8c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b89380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557859ae6970 con 0x557859b8c800 2024-04-23T18:45:40.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557859a810b0 con 0x557859b81f40 2024-04-23T18:45:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557859b8f1f0 con 0x557859b81620 2024-04-23T18:45:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 10 client.0 _ll_register_callbacks cb 0x557859a6ed30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35044/0 (socket says 172.21.15.33:35044) 2024-04-23T18:45:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 -- 172.21.15.33:0/2138473094 learned_addr learned my addr 172.21.15.33:0/2138473094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc597fff640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b89380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b8c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:41.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 -- 172.21.15.33:0/2138473094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 msgr2=0x557859b8c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b8c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 -- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 msgr2=0x557859b89380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:41.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b89380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 -- 172.21.15.33:0/2138473094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578598c5620 con 0x557859b8c800 2024-04-23T18:45:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:41.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc59f59e640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc59800fec0 tx=0x7fc598004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e0d52342129208 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5867fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:41.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59802e060 con 0x557859b8c800 2024-04-23T18:45:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fc598021c00 con 0x557859b8c800 2024-04-23T18:45:41.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.113+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5880bcba0 con 0x557859b8c800 2024-04-23T18:45:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc598021e00 con 0x557859b8c800 2024-04-23T18:45:41.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 10 client.17490 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557859b9af80 con 0x557859b8c800 2024-04-23T18:45:41.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5880bcba0 con 0x557859b8c800 2024-04-23T18:45:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fc59801d070 con 0x557859b8c800 2024-04-23T18:45:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 10 client.17490 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 10 client.17490 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 20 client.17490 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:42.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b9c490 0x557859b9e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 4 client.17490 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:42.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557859af4390 con 0x557859b9c490 2024-04-23T18:45:42.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.114+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b9c490 0x557859b9e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.115+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b9c490 0x557859b9e880 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85468c00a00b0de5 server_cookie=c20f26fbd9574ebe in_seq=0 out_seq=0 2024-04-23T18:45:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.115+0000 7fc5867fc640 10 client.17490 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5867fc640 1 -- 172.21.15.33:0/2138473094 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557859af4390 con 0x557859b9c490 2024-04-23T18:45:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5867fc640 10 client.17490 handle_command_reply: tid=0 2024-04-23T18:45:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 client.17490 shutdown 2024-04-23T18:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 msgr2=0x557859b898c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc59800fec0 tx=0x7fc598004a40 comp rx=0 tx=0).stop 2024-04-23T18:45:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 shutdown_connections 2024-04-23T18:45:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b9c490 0x557859b9e880 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b8c800 0x557859b898c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b89380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/2138473094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b81620 0x557859b8c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:42.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5867fc640 0 client.17490 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 >> 172.21.15.33:0/2138473094 conn(0x557859b7db30 msgr2=0x557859b838b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 shutdown_connections 2024-04-23T18:45:43.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/2138473094 wait complete. 2024-04-23T18:45:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.116+0000 7fc5a7e1d740 20 client.17490 trim_cache size 0 max 16384 2024-04-23T18:45:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 Processor -- start 2024-04-23T18:45:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 -- start start 2024-04-23T18:45:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b84a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01af670 con 0x557859b81620 2024-04-23T18:45:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ad460 con 0x557859b81f40 2024-04-23T18:45:43.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.131+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01aefa0 con 0x557859b7db30 2024-04-23T18:45:43.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35052/0 (socket says 172.21.15.33:35052) 2024-04-23T18:45:43.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 -- 172.21.15.33:0/4001190993 learned_addr learned my addr 172.21.15.33:0/4001190993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b84a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc59f59e640 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 -- 172.21.15.33:0/4001190993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 msgr2=0x557859b84a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:43.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b84a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 -- 172.21.15.33:0/4001190993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 msgr2=0x557859b836e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b836e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 -- 172.21.15.33:0/4001190993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557859b89ec0 con 0x557859b81620 2024-04-23T18:45:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc597fff640 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc588006010 tx=0x7fc58803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb965d696b6f2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc594ff9640 1 -- 172.21.15.33:0/4001190993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588049040 con 0x557859b81620 2024-04-23T18:45:44.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc594ff9640 1 -- 172.21.15.33:0/4001190993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5880c5760 con 0x557859b81620 2024-04-23T18:45:44.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.132+0000 7fc594ff9640 1 -- 172.21.15.33:0/4001190993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5880ee520 con 0x557859b81620 2024-04-23T18:45:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/4001190993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 msgr2=0x557859b8bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc588006010 tx=0x7fc58803b460 comp rx=0 tx=0).stop 2024-04-23T18:45:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/4001190993 shutdown_connections 2024-04-23T18:45:44.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81f40 0x557859b836e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81620 0x557859b8bbc0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/4001190993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b84a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/4001190993 >> 172.21.15.33:0/4001190993 conn(0x557859b89a20 msgr2=0x557859b82fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/4001190993 shutdown_connections 2024-04-23T18:45:44.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/4001190993 wait complete. 2024-04-23T18:45:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 -- start start 2024-04-23T18:45:44.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.133+0000 7fc5a7e1d740 1 Processor -- start 2024-04-23T18:45:44.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b8ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81620 0x557859b904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ae200 con 0x557859b81f40 2024-04-23T18:45:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01adb30 con 0x557859b81620 2024-04-23T18:45:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a01ae8d0 con 0x557859b7db30 2024-04-23T18:45:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 10 client.0 _ll_register_callbacks cb 0x7fc598009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35056/0 (socket says 172.21.15.33:35056) 2024-04-23T18:45:45.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 -- 172.21.15.33:0/778175218 learned_addr learned my addr 172.21.15.33:0/778175218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b8ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc597fff640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81620 0x557859b904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 -- 172.21.15.33:0/778175218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 msgr2=0x557859b8ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b8ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 -- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81620 msgr2=0x557859b904c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81620 0x557859b904c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 -- 172.21.15.33:0/778175218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557859c01ac0 con 0x557859b81f40 2024-04-23T18:45:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:45.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc59f59e640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc598009270 tx=0x7fc598009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2098164541408c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5867fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:45.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598004e60 con 0x557859b81f40 2024-04-23T18:45:45.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fc5980119b0 con 0x557859b81f40 2024-04-23T18:45:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.134+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5880c5760 con 0x557859b81f40 2024-04-23T18:45:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc598040050 con 0x557859b81f40 2024-04-23T18:45:45.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 10 client.17496 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:45.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557859c02db0 con 0x557859b81f40 2024-04-23T18:45:45.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5880c5760 con 0x557859b81f40 2024-04-23T18:45:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fc59801d040 con 0x557859b81f40 2024-04-23T18:45:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 10 client.17496 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 10 client.17496 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:45.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 20 client.17496 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:45.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b8c800 0x557859c05470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:45.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 4 client.17496 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:46.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557859aee200 con 0x557859b8c800 2024-04-23T18:45:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.135+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b8c800 0x557859c05470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.136+0000 7fc59ed9d640 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b8c800 0x557859c05470 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6f8739b6712821 server_cookie=9109ba606731a1b6 in_seq=0 out_seq=0 2024-04-23T18:45:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.136+0000 7fc5867fc640 10 client.17496 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5867fc640 1 -- 172.21.15.33:0/778175218 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557859aee200 con 0x557859b8c800 2024-04-23T18:45:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5867fc640 10 client.17496 handle_command_reply: tid=0 2024-04-23T18:45:46.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 client.17496 shutdown 2024-04-23T18:45:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 msgr2=0x557859b90a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc598009270 tx=0x7fc598009d50 comp rx=0 tx=0).stop 2024-04-23T18:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 shutdown_connections 2024-04-23T18:45:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557859b8c800 0x557859c05470 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557859b81f40 0x557859b90a00 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557859b81620 0x557859b904c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5867fc640 0 client.17496 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 --2- 172.21.15.33:0/778175218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557859b7db30 0x557859b8ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 >> 172.21.15.33:0/778175218 conn(0x557859b89a20 msgr2=0x557859b96f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:46.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 shutdown_connections 2024-04-23T18:45:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 1 -- 172.21.15.33:0/778175218 wait complete. 2024-04-23T18:45:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a7e1d740 20 client.17496 trim_cache size 0 max 16384 2024-04-23T18:45:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:45:47.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:45:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:45:47.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:45:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 msgr2=0x7fc58c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:47.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 0x7fc58c0a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc5a006b1f0 tx=0x7fc598009500 comp rx=0 tx=0).stop 2024-04-23T18:45:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 msgr2=0x7fc5a0198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:47.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.140+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc588005940 tx=0x7fc5880040c0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 shutdown_connections 2024-04-23T18:45:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc58c0a3dc0 0x7fc58c0a6280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5a010c5b0 0x7fc5a019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5a0103b00 0x7fc5a0198cb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 --2- 172.21.15.33:0/3322738960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a01031e0 0x7fc5a0198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 >> 172.21.15.33:0/3322738960 conn(0x7fc5a0077f60 msgr2=0x7fc5a00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 shutdown_connections 2024-04-23T18:45:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:35.141+0000 7fc5a5108640 1 -- 172.21.15.33:0/3322738960 wait complete. 2024-04-23T18:45:48.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:34 smithi033 ceph-mon[29274]: pgmap v543: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:45:48.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:36 smithi033 ceph-mon[29274]: pgmap v544: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:45:48.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:38 smithi033 ceph-mon[29274]: pgmap v545: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:40 smithi033 ceph-mon[29274]: pgmap v546: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:42 smithi033 ceph-mon[29274]: pgmap v547: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:48.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:44 smithi033 ceph-mon[29274]: pgmap v548: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:46 smithi033 ceph-mon[29274]: pgmap v549: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:48.481 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:36 smithi106 ceph-mon[32716]: pgmap v609: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:45:48.530 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:38 smithi106 ceph-mon[32716]: pgmap v610: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.578 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:40 smithi106 ceph-mon[32716]: pgmap v611: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:42 smithi106 ceph-mon[32716]: pgmap v612: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:48.678 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:44 smithi106 ceph-mon[32716]: pgmap v613: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:46 smithi106 ceph-mon[32716]: pgmap v614: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:48.778 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:36 smithi122 ceph-mon[32546]: pgmap v609: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:45:48.895 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:38 smithi122 ceph-mon[32546]: pgmap v610: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:40 smithi122 ceph-mon[32546]: pgmap v611: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:48.994 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:42 smithi122 ceph-mon[32546]: pgmap v612: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:49.044 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:44 smithi122 ceph-mon[32546]: pgmap v613: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:45:49.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:46 smithi122 ceph-mon[32546]: pgmap v614: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:45:49.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 -- 172.21.15.33:0/3280640618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 msgr2=0x7fab301025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:49.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 --2- 172.21.15.33:0/3280640618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301025a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fab20009900 tx=0x7fab2002a410 comp rx=0 tx=0).stop 2024-04-23T18:45:49.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 -- 172.21.15.33:0/3280640618 shutdown_connections 2024-04-23T18:45:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 --2- 172.21.15.33:0/3280640618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab30107290 0x7fab3010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 --2- 172.21.15.33:0/3280640618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30102b70 0x7fab30106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 --2- 172.21.15.33:0/3280640618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301025a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:49.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 -- 172.21.15.33:0/3280640618 >> 172.21.15.33:0/3280640618 conn(0x7fab300fda20 msgr2=0x7fab300ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:49.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 -- 172.21.15.33:0/3280640618 shutdown_connections 2024-04-23T18:45:49.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.239+0000 7fab36435640 1 -- 172.21.15.33:0/3280640618 wait complete. 2024-04-23T18:45:49.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 Processor -- start 2024-04-23T18:45:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 -- start start 2024-04-23T18:45:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30107290 0x7fab30198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab3019cf40 0x7fab3019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:49.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab301121f0 con 0x7fab301021c0 2024-04-23T18:45:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112070 con 0x7fab3019cf40 2024-04-23T18:45:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.240+0000 7fab36435640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab30112370 con 0x7fab30107290 2024-04-23T18:45:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37828/0 (socket says 172.21.15.33:37828) 2024-04-23T18:45:50.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 -- 172.21.15.33:0/2319964751 learned_addr learned my addr 172.21.15.33:0/2319964751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:50.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30107290 0x7fab30198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab349d2640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab3019cf40 0x7fab3019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 -- 172.21.15.33:0/2319964751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30107290 msgr2=0x7fab30198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30107290 0x7fab30198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 -- 172.21.15.33:0/2319964751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab3019cf40 msgr2=0x7fab3019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:50.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab3019cf40 0x7fab3019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 -- 172.21.15.33:0/2319964751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3019da30 con 0x7fab301021c0 2024-04-23T18:45:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fab20006010 tx=0x7fab200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:50.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2d7fa640 1 -- 172.21.15.33:0/2319964751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2003f030 con 0x7fab301021c0 2024-04-23T18:45:50.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3019dcc0 con 0x7fab301021c0 2024-04-23T18:45:50.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2d7fa640 1 -- 172.21.15.33:0/2319964751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab20003800 con 0x7fab301021c0 2024-04-23T18:45:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab2d7fa640 1 -- 172.21.15.33:0/2319964751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20035070 con 0x7fab301021c0 2024-04-23T18:45:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.241+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab301a54e0 con 0x7fab301021c0 2024-04-23T18:45:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab2d7fa640 1 -- 172.21.15.33:0/2319964751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab20003cb0 con 0x7fab301021c0 2024-04-23T18:45:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 Processor -- start 2024-04-23T18:45:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 -- start start 2024-04-23T18:45:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:50.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f2764e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f27650bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f275a8970 con 0x556f27643620 2024-04-23T18:45:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556f275454a0 con 0x556f27643f40 2024-04-23T18:45:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.244+0000 7fab3922e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f275a4930 con 0x556f2764e800 2024-04-23T18:45:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab2d7fa640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 0x7fab180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:51.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37844/0 (socket says 172.21.15.33:37844) 2024-04-23T18:45:51.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 -- 172.21.15.33:0/1289570556 learned_addr learned my addr 172.21.15.33:0/1289570556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab2ffff640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f2764e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f27650bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 0x7fab180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab2d7fa640 1 -- 172.21.15.33:0/2319964751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fab200f1150 con 0x7fab301021c0 2024-04-23T18:45:51.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 -- 172.21.15.33:0/1289570556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 msgr2=0x556f27650bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:51.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f27650bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 -- 172.21.15.33:0/1289570556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 msgr2=0x556f2764e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f2764e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:51.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 -- 172.21.15.33:0/1289570556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f272bb210 con 0x556f27643620 2024-04-23T18:45:51.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab2ffff640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f2764e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:45:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab349d2640 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fab280139d0 tx=0x7fab28013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b750096af2668f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab167fc640 1 -- 172.21.15.33:0/1289570556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28013050 con 0x556f27643620 2024-04-23T18:45:51.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab167fc640 1 -- 172.21.15.33:0/1289570556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28022070 con 0x556f27643620 2024-04-23T18:45:51.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.245+0000 7fab167fc640 1 -- 172.21.15.33:0/1289570556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28012650 con 0x556f27643620 2024-04-23T18:45:51.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 -- 172.21.15.33:0/1289570556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 msgr2=0x556f27643a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:51.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fab280139d0 tx=0x7fab28013e90 comp rx=0 tx=0).stop 2024-04-23T18:45:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab349d2640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 0x7fab180a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab28021000 tx=0x7fab2801f000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 -- 172.21.15.33:0/1289570556 shutdown_connections 2024-04-23T18:45:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f27650bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f2764e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 --2- 172.21.15.33:0/1289570556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643620 0x556f27643a00 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:52.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 -- 172.21.15.33:0/1289570556 >> 172.21.15.33:0/1289570556 conn(0x556f2763fb30 msgr2=0x556f2763ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 -- 172.21.15.33:0/1289570556 shutdown_connections 2024-04-23T18:45:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.246+0000 7fab3922e740 1 -- 172.21.15.33:0/1289570556 wait complete. 2024-04-23T18:45:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 -- start start 2024-04-23T18:45:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 Processor -- start 2024-04-23T18:45:52.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 0x556f2764e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f2764b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:52.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f27651430 con 0x556f27643f40 2024-04-23T18:45:52.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556f275430b0 con 0x556f27643620 2024-04-23T18:45:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f276515b0 con 0x556f2764e800 2024-04-23T18:45:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab3922e740 10 client.0 _ll_register_callbacks cb 0x556f27530d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37856/0 (socket says 172.21.15.33:37856) 2024-04-23T18:45:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.247+0000 7fab2f7fe640 1 -- 172.21.15.33:0/4087853072 learned_addr learned my addr 172.21.15.33:0/4087853072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab349d2640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f2764b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2ffff640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 0x556f2764e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 -- 172.21.15.33:0/4087853072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 msgr2=0x556f2764b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f2764b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 -- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 msgr2=0x556f2764e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 0x556f2764e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 -- 172.21.15.33:0/4087853072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f27387620 con 0x556f27643f40 2024-04-23T18:45:53.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2ffff640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 0x556f2764e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fab2400afd0 tx=0x7fab24010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1270b3e10486c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:53.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400fc50 con 0x556f27643f40 2024-04-23T18:45:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fab24010e00 con 0x556f27643f40 2024-04-23T18:45:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28022070 con 0x556f27643f40 2024-04-23T18:45:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fab2400fdf0 con 0x556f27643f40 2024-04-23T18:45:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab3922e740 10 client.17508 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556f2765d3b0 con 0x556f27643f40 2024-04-23T18:45:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.248+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28022070 con 0x556f27643f40 2024-04-23T18:45:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fab24019410 con 0x556f27643f40 2024-04-23T18:45:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 10 client.17508 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:54.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 20 client.17508 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:54.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f2765e930 0x556f27660d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 4 client.17508 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:45:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f275b6390 con 0x556f2765e930 2024-04-23T18:45:54.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab2ffff640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f2765e930 0x556f27660d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab2ffff640 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f2765e930 0x556f27660d20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae96242d4c16cdf server_cookie=a58d9a4e50e73b8d in_seq=0 out_seq=0 2024-04-23T18:45:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.249+0000 7fab0f7fe640 10 client.17508 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab0f7fe640 1 -- 172.21.15.33:0/4087853072 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f275b6390 con 0x556f2765e930 2024-04-23T18:45:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab0f7fe640 10 client.17508 handle_command_reply: tid=0 2024-04-23T18:45:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 client.17508 shutdown 2024-04-23T18:45:54.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 msgr2=0x556f2764b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fab2400afd0 tx=0x7fab24010bb0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 shutdown_connections 2024-04-23T18:45:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f2765e930 0x556f27660d20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f2764e800 0x556f2764b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f2764b410 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 --2- 172.21.15.33:0/4087853072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643620 0x556f2764e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab0f7fe640 0 client.17508 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.250+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 >> 172.21.15.33:0/4087853072 conn(0x556f2763fb30 msgr2=0x556f276458b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.251+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 shutdown_connections 2024-04-23T18:45:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.251+0000 7fab3922e740 1 -- 172.21.15.33:0/4087853072 wait complete. 2024-04-23T18:45:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.251+0000 7fab3922e740 20 client.17508 trim_cache size 0 max 16384 2024-04-23T18:45:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 Processor -- start 2024-04-23T18:45:55.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 -- start start 2024-04-23T18:45:55.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f2765a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f27656be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:55.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab301af5b0 con 0x556f2764e800 2024-04-23T18:45:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ad3a0 con 0x556f27643f40 2024-04-23T18:45:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab3922e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab301aeee0 con 0x556f27643620 2024-04-23T18:45:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:55.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37866/0 (socket says 172.21.15.33:37866) 2024-04-23T18:45:55.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 -- 172.21.15.33:0/3145701067 learned_addr learned my addr 172.21.15.33:0/3145701067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab2ffff640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f2765a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f27656be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:56.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 -- 172.21.15.33:0/3145701067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 msgr2=0x556f2765a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f2765a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 -- 172.21.15.33:0/3145701067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 msgr2=0x556f27656be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f27656be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab349d2640 1 -- 172.21.15.33:0/3145701067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f27642e20 con 0x556f2764e800 2024-04-23T18:45:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.266+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f27656be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:45:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab349d2640 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fab28011fd0 tx=0x7fab28007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71cb37e9e296dad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:56.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab15ffb640 1 -- 172.21.15.33:0/3145701067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab28012450 con 0x556f2764e800 2024-04-23T18:45:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab15ffb640 1 -- 172.21.15.33:0/3145701067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28022040 con 0x556f2764e800 2024-04-23T18:45:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab15ffb640 1 -- 172.21.15.33:0/3145701067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab280132b0 con 0x556f2764e800 2024-04-23T18:45:56.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 -- 172.21.15.33:0/3145701067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 msgr2=0x556f2764dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fab28011fd0 tx=0x7fab28007a50 comp rx=0 tx=0).stop 2024-04-23T18:45:56.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 -- 172.21.15.33:0/3145701067 shutdown_connections 2024-04-23T18:45:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f2764e800 0x556f2764dc50 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f27643f40 0x556f27656be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 --2- 172.21.15.33:0/3145701067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f2765a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:57.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 -- 172.21.15.33:0/3145701067 >> 172.21.15.33:0/3145701067 conn(0x556f2763fb30 msgr2=0x556f2763ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 -- 172.21.15.33:0/3145701067 shutdown_connections 2024-04-23T18:45:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.267+0000 7fab3922e740 1 -- 172.21.15.33:0/3145701067 wait complete. 2024-04-23T18:45:57.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 -- start start 2024-04-23T18:45:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 Processor -- start 2024-04-23T18:45:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f276bd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:57.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:57.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 0x556f276c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ae140 con 0x556f27643f40 2024-04-23T18:45:57.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ada70 con 0x556f2764e800 2024-04-23T18:45:57.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab301ae810 con 0x556f27643620 2024-04-23T18:45:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab3922e740 10 client.0 _ll_register_callbacks cb 0x7fab2400af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:45:57.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab2f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37872/0 (socket says 172.21.15.33:37872) 2024-04-23T18:45:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.268+0000 7fab2f7fe640 1 -- 172.21.15.33:0/2084256469 learned_addr learned my addr 172.21.15.33:0/2084256469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:45:58.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f276bd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab349d2640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 0x556f276c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:58.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 -- 172.21.15.33:0/2084256469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 msgr2=0x556f276bd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f276bd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 -- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 msgr2=0x556f276c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:45:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 0x556f276c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:58.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 -- 172.21.15.33:0/2084256469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f276c8410 con 0x556f27643f40 2024-04-23T18:45:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab349d2640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 0x556f276c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:45:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab2f7fe640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fab24019880 tx=0x7fab2400ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5860abb403d01f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:45:58.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:45:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24005040 con 0x556f27643f40 2024-04-23T18:45:58.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7fab24010e00 con 0x556f27643f40 2024-04-23T18:45:58.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab28022040 con 0x556f27643f40 2024-04-23T18:45:58.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fab240051e0 con 0x556f27643f40 2024-04-23T18:45:58.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab3922e740 10 client.17514 fetch_fsmap learned FSMap version 38 2024-04-23T18:45:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556f276c96c0 con 0x556f27643f40 2024-04-23T18:45:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.269+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab28022040 con 0x556f27643f40 2024-04-23T18:45:59.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7fab24029030 con 0x556f27643f40 2024-04-23T18:45:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 10 client.17514 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:45:59.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:45:59.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 20 client.17514 populate_metadata read hostname 'smithi033' 2024-04-23T18:45:59.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f276c9e60 0x556f276cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:45:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 4 client.17514 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:45:59.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f2764bce0 con 0x556f276c9e60 2024-04-23T18:45:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f276c9e60 0x556f276cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:45:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab2ffff640 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f276c9e60 0x556f276cc280 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f8da25a59060226 server_cookie=866cab3d770c4db5 in_seq=0 out_seq=0 2024-04-23T18:45:59.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.270+0000 7fab0f7fe640 10 client.17514 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:59.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.274+0000 7fab0f7fe640 1 -- 172.21.15.33:0/2084256469 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f2764bce0 con 0x556f276c9e60 2024-04-23T18:45:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.274+0000 7fab0f7fe640 10 client.17514 handle_command_reply: tid=0 2024-04-23T18:45:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.274+0000 7fab3922e740 1 client.17514 shutdown 2024-04-23T18:45:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 msgr2=0x556f276bd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:45:59.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fab24019880 tx=0x7fab2400ac60 comp rx=0 tx=0).stop 2024-04-23T18:45:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 shutdown_connections 2024-04-23T18:45:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556f276c9e60 0x556f276cc280 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556f2764e800 0x556f276c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f27643f40 0x556f276bd550 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 --2- 172.21.15.33:0/2084256469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f27643620 0x556f276bd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:45:59.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab0f7fe640 0 client.17514 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:45:59.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 >> 172.21.15.33:0/2084256469 conn(0x556f2763fb30 msgr2=0x556f276592e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:45:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 shutdown_connections 2024-04-23T18:46:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 1 -- 172.21.15.33:0/2084256469 wait complete. 2024-04-23T18:46:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab3922e740 20 client.17514 trim_cache size 0 max 16384 2024-04-23T18:46:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:46:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:46:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:46:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:46:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 msgr2=0x7fab180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 0x7fab180a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab28021000 tx=0x7fab2801f000 comp rx=0 tx=0).stop 2024-04-23T18:46:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 msgr2=0x7fab301986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fab20006010 tx=0x7fab200042f0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 shutdown_connections 2024-04-23T18:46:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fab180a3dc0 0x7fab180a6280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab3019cf40 0x7fab3019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab30107290 0x7fab30198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 --2- 172.21.15.33:0/2319964751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab301021c0 0x7fab301986d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:00.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.275+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 >> 172.21.15.33:0/2319964751 conn(0x7fab300fda20 msgr2=0x7fab300ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.276+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 shutdown_connections 2024-04-23T18:46:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:43:48.276+0000 7fab36435640 1 -- 172.21.15.33:0/2319964751 wait complete. 2024-04-23T18:46:00.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:48 smithi033 ceph-mon[29274]: pgmap v550: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:00.768 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:50 smithi033 ceph-mon[29274]: pgmap v551: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:00.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:51 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:00.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:52 smithi033 ceph-mon[29274]: pgmap v552: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:00.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:00.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:01.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:52 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:46:01.111 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:54 smithi033 ceph-mon[29274]: pgmap v553: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:01.154 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:56 smithi033 ceph-mon[29274]: pgmap v554: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:01.188 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:43:58 smithi033 ceph-mon[29274]: pgmap v555: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:01.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:00 smithi033 ceph-mon[29274]: pgmap v556: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:01.258 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:48 smithi122 ceph-mon[32546]: pgmap v615: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:01.339 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:50 smithi122 ceph-mon[32546]: pgmap v616: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:01.363 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:52 smithi122 ceph-mon[32546]: pgmap v617: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:01.396 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:01.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:01.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:01.497 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:53 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:46:01.605 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:54 smithi122 ceph-mon[32546]: pgmap v618: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:01.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:56 smithi122 ceph-mon[32546]: pgmap v619: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:01.671 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:45:58 smithi122 ceph-mon[32546]: pgmap v620: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:01.744 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:48 smithi106 ceph-mon[32716]: pgmap v615: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:01.788 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:50 smithi106 ceph-mon[32716]: pgmap v616: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:46:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:52 smithi106 ceph-mon[32716]: pgmap v617: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:01.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:01.938 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:01.988 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:02.038 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:53 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:46:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:54 smithi106 ceph-mon[32716]: pgmap v618: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:02.138 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:56 smithi106 ceph-mon[32716]: pgmap v619: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:02.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:45:58 smithi106 ceph-mon[32716]: pgmap v620: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:02.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:00 smithi122 ceph-mon[32546]: pgmap v621: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:02.289 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:00 smithi106 ceph-mon[32716]: pgmap v621: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.374+0000 7f1bd84e8640 1 -- 172.21.15.33:0/1028097118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103180 msgr2=0x7f1bd0103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.374+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/1028097118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103180 0x7f1bd0103560 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc02a480 comp rx=0 tx=0).stop 2024-04-23T18:46:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 -- 172.21.15.33:0/1028097118 shutdown_connections 2024-04-23T18:46:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/1028097118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd010c550 0x7f1bd010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/1028097118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0103aa0 0x7f1bd010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/1028097118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103180 0x7f1bd0103560 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 -- 172.21.15.33:0/1028097118 >> 172.21.15.33:0/1028097118 conn(0x7f1bd0077fd0 msgr2=0x7f1bd00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 -- 172.21.15.33:0/1028097118 shutdown_connections 2024-04-23T18:46:02.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 -- 172.21.15.33:0/1028097118 wait complete. 2024-04-23T18:46:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 Processor -- start 2024-04-23T18:46:02.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.375+0000 7f1bd84e8640 1 -- start start 2024-04-23T18:46:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd0103180 0x7f1bd0100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 0x7f1bd0101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01121b0 con 0x7f1bd0103aa0 2024-04-23T18:46:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd0112030 con 0x7f1bd010c550 2024-04-23T18:46:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd0112330 con 0x7f1bd0103180 2024-04-23T18:46:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60084/0 (socket says 172.21.15.33:60084) 2024-04-23T18:46:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 -- 172.21.15.33:0/906879923 learned_addr learned my addr 172.21.15.33:0/906879923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd6284640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd0103180 0x7f1bd0100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 0x7f1bd0101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 -- 172.21.15.33:0/906879923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd0103180 msgr2=0x7f1bd0100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd0103180 0x7f1bd0100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 -- 172.21.15.33:0/906879923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 msgr2=0x7f1bd0101850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 0x7f1bd0101850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 -- 172.21.15.33:0/906879923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd01a6bd0 con 0x7f1bd0103aa0 2024-04-23T18:46:03.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 0x7f1bd0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d950 tx=0x7f1bc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bc77fe640 1 -- 172.21.15.33:0/906879923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0017030 con 0x7f1bd0103aa0 2024-04-23T18:46:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bc77fe640 1 -- 172.21.15.33:0/906879923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bc0004020 con 0x7f1bd0103aa0 2024-04-23T18:46:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd01a6ec0 con 0x7f1bd0103aa0 2024-04-23T18:46:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.376+0000 7f1bc77fe640 1 -- 172.21.15.33:0/906879923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc00047a0 con 0x7f1bd0103aa0 2024-04-23T18:46:04.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.377+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd01a7380 con 0x7f1bd0103aa0 2024-04-23T18:46:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.379+0000 7f1bc77fe640 1 -- 172.21.15.33:0/906879923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1bc0009a30 con 0x7f1bd0103aa0 2024-04-23T18:46:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.379+0000 7f1bdb202740 1 Processor -- start 2024-04-23T18:46:04.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 -- start start 2024-04-23T18:46:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afb620 0x55d884afba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b08bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d884a60970 con 0x55d884b06800 2024-04-23T18:46:04.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d8849fd4a0 con 0x55d884afb620 2024-04-23T18:46:04.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bdb202740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d884a5c930 con 0x55d884afbf40 2024-04-23T18:46:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bc77fe640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 0x7f1bbc0a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bd5a83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b08bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 0x7f1bbc0a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.380+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:35752/0 (socket says 172.21.15.33:35752) 2024-04-23T18:46:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 -- 172.21.15.33:0/3700669601 learned_addr learned my addr 172.21.15.33:0/3700669601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:05.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bc77fe640 1 -- 172.21.15.33:0/906879923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f1bc00d02a0 con 0x7f1bd0103aa0 2024-04-23T18:46:05.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afb620 0x55d884afba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:05.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 -- 172.21.15.33:0/3700669601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afb620 msgr2=0x55d884afba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:05.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afb620 0x55d884afba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 -- 172.21.15.33:0/3700669601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 msgr2=0x55d884b08bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b08bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 -- 172.21.15.33:0/3700669601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884773210 con 0x55d884afbf40 2024-04-23T18:46:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 0x7f1bbc0a62b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8005c40 tx=0x7f1bc8009d50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc032bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed89cf222ee8b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bc4bfe640 1 -- 172.21.15.33:0/3700669601 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc040070 con 0x55d884afbf40 2024-04-23T18:46:05.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bc4bfe640 1 -- 172.21.15.33:0/3700669601 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc004030 con 0x55d884afbf40 2024-04-23T18:46:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.381+0000 7f1bc4bfe640 1 -- 172.21.15.33:0/3700669601 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc035040 con 0x55d884afbf40 2024-04-23T18:46:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 -- 172.21.15.33:0/3700669601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 msgr2=0x55d884b062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc032bd0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 -- 172.21.15.33:0/3700669601 shutdown_connections 2024-04-23T18:46:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b08bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b062c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3700669601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afb620 0x55d884afba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 -- 172.21.15.33:0/3700669601 >> 172.21.15.33:0/3700669601 conn(0x55d884af7b30 msgr2=0x55d884af7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 -- 172.21.15.33:0/3700669601 shutdown_connections 2024-04-23T18:46:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.382+0000 7f1bdb202740 1 -- 172.21.15.33:0/3700669601 wait complete. 2024-04-23T18:46:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 -- start start 2024-04-23T18:46:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 Processor -- start 2024-04-23T18:46:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afb620 0x55d884b06180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b03470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d884b093d0 con 0x55d884b06800 2024-04-23T18:46:06.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d8849fb0b0 con 0x55d884afbf40 2024-04-23T18:46:06.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d884b09550 con 0x55d884afb620 2024-04-23T18:46:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bdb202740 10 client.0 _ll_register_callbacks cb 0x55d8849e8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60106/0 (socket says 172.21.15.33:60106) 2024-04-23T18:46:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 -- 172.21.15.33:0/1861632586 learned_addr learned my addr 172.21.15.33:0/1861632586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:06.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b03470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6284640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afb620 0x55d884b06180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 -- 172.21.15.33:0/1861632586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afb620 msgr2=0x55d884b06180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afb620 0x55d884b06180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:06.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 -- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 msgr2=0x55d884b03470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b03470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd6a85640 1 -- 172.21.15.33:0/1861632586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88483f620 con 0x55d884b06800 2024-04-23T18:46:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.383+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b03470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8010d20 tx=0x7f1bc8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65160a6fffb8044f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:07.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc801ee40 con 0x55d884b06800 2024-04-23T18:46:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f1bc802a410 con 0x55d884b06800 2024-04-23T18:46:07.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc004030 con 0x55d884b06800 2024-04-23T18:46:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1bc802a650 con 0x55d884b06800 2024-04-23T18:46:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 10 client.17526 fetch_fsmap learned FSMap version 38 2024-04-23T18:46:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d884b15320 con 0x55d884b06800 2024-04-23T18:46:07.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc004030 con 0x55d884b06800 2024-04-23T18:46:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f1bc801f070 con 0x55d884b06800 2024-04-23T18:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 10 client.17526 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:46:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 20 client.17526 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:07.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b167e0 0x55d884b18bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 4 client.17526 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:46:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.384+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d884a6e390 con 0x55d884b167e0 2024-04-23T18:46:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.385+0000 7f1bd6284640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b167e0 0x55d884b18bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:07.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.385+0000 7f1bd6284640 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b167e0 0x55d884b18bf0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1094b442e36715 server_cookie=c4a95f4804babd07 in_seq=0 out_seq=0 2024-04-23T18:46:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.385+0000 7f1bb9ffb640 10 client.17526 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:07.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/1861632586 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d884a6e390 con 0x55d884b167e0 2024-04-23T18:46:07.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bb9ffb640 10 client.17526 handle_command_reply: tid=0 2024-04-23T18:46:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 client.17526 shutdown 2024-04-23T18:46:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 msgr2=0x55d884b039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8010d20 tx=0x7f1bc8005bd0 comp rx=0 tx=0).stop 2024-04-23T18:46:08.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 shutdown_connections 2024-04-23T18:46:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b167e0 0x55d884b18bf0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884b06800 0x55d884b039f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:08.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b03470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bb9ffb640 0 client.17526 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 --2- 172.21.15.33:0/1861632586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afb620 0x55d884b06180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 >> 172.21.15.33:0/1861632586 conn(0x55d884af7b30 msgr2=0x55d884afd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 shutdown_connections 2024-04-23T18:46:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 1 -- 172.21.15.33:0/1861632586 wait complete. 2024-04-23T18:46:08.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.386+0000 7f1bdb202740 20 client.17526 trim_cache size 0 max 16384 2024-04-23T18:46:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 Processor -- start 2024-04-23T18:46:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 -- start start 2024-04-23T18:46:08.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b1e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 0x55d884b1ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01b1840 con 0x55d884afb620 2024-04-23T18:46:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01af630 con 0x55d884afbf40 2024-04-23T18:46:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bdb202740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01b1170 con 0x55d884b06800 2024-04-23T18:46:09.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60110/0 (socket says 172.21.15.33:60110) 2024-04-23T18:46:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 -- 172.21.15.33:0/3048720194 learned_addr learned my addr 172.21.15.33:0/3048720194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:09.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 -- 172.21.15.33:0/3048720194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 msgr2=0x55d884b1ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 0x55d884b1ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:09.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b1e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 0x55d884b1ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 -- 172.21.15.33:0/3048720194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 msgr2=0x55d884b1e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b1e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 -- 172.21.15.33:0/3048720194 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d884b08bf0 con 0x55d884afb620 2024-04-23T18:46:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 0x55d884b1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bd6284640 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ad35a07486086f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:09.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.402+0000 7f1bbbfff640 1 -- 172.21.15.33:0/3048720194 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc054070 con 0x55d884afb620 2024-04-23T18:46:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bbbfff640 1 -- 172.21.15.33:0/3048720194 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc0044b0 con 0x55d884afb620 2024-04-23T18:46:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bbbfff640 1 -- 172.21.15.33:0/3048720194 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc040040 con 0x55d884afb620 2024-04-23T18:46:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- 172.21.15.33:0/3048720194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 msgr2=0x55d884b05cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:09.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009900 tx=0x7f1bcc005a20 comp rx=0 tx=0).stop 2024-04-23T18:46:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- 172.21.15.33:0/3048720194 shutdown_connections 2024-04-23T18:46:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884b06800 0x55d884b1ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884afbf40 0x55d884b1e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 --2- 172.21.15.33:0/3048720194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b05cb0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- 172.21.15.33:0/3048720194 >> 172.21.15.33:0/3048720194 conn(0x55d884af7b30 msgr2=0x55d884b03430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:10.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- 172.21.15.33:0/3048720194 shutdown_connections 2024-04-23T18:46:10.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- 172.21.15.33:0/3048720194 wait complete. 2024-04-23T18:46:10.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.403+0000 7f1bdb202740 1 -- start start 2024-04-23T18:46:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 Processor -- start 2024-04-23T18:46:10.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b0bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:10.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 0x55d884b0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:10.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01b03d0 con 0x55d884afb620 2024-04-23T18:46:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01afd00 con 0x55d884b06800 2024-04-23T18:46:10.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01b0aa0 con 0x55d884afbf40 2024-04-23T18:46:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bdb202740 10 client.0 _ll_register_callbacks cb 0x7f1bc8009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:11.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60124/0 (socket says 172.21.15.33:60124) 2024-04-23T18:46:11.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 -- 172.21.15.33:0/2959220569 learned_addr learned my addr 172.21.15.33:0/2959220569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b0bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 0x55d884b0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 -- 172.21.15.33:0/2959220569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 msgr2=0x55d884b0bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b0bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 -- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 msgr2=0x55d884b0c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 0x55d884b0c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:11.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 -- 172.21.15.33:0/2959220569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d884b0cc20 con 0x55d884afb620 2024-04-23T18:46:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6a85640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 0x55d884b0c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bd6284640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0043a0 tx=0x7f1bcc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867351e960423dce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:11.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.404+0000 7f1bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:11.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc054070 con 0x55d884afb620 2024-04-23T18:46:11.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8542+0+0 (secure 0 0 0) 0x7f1bcc005150 con 0x55d884afb620 2024-04-23T18:46:11.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bcc0044b0 con 0x55d884afb620 2024-04-23T18:46:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc060050 con 0x55d884afb620 2024-04-23T18:46:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 10 client.17532 fetch_fsmap learned FSMap version 38 2024-04-23T18:46:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d884b7cd10 con 0x55d884afb620 2024-04-23T18:46:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bcc0044b0 con 0x55d884afb620 2024-04-23T18:46:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 38) ==== 7749+0+0 (secure 0 0 0) 0x7f1bcc04f040 con 0x55d884afb620 2024-04-23T18:46:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 10 client.17532 fetch_fsmap finished waiting for FSMap version 38 2024-04-23T18:46:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 10 client.17532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 20 client.17532 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b7cf30 0x55d884b7f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 4 client.17532 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:46:12.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d884a68200 con 0x55d884b7cf30 2024-04-23T18:46:12.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.405+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b7cf30 0x55d884b7f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.406+0000 7f1bd5a83640 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b7cf30 0x55d884b7f320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a8c6f7229b335f server_cookie=543dbf7472d3522f in_seq=0 out_seq=0 2024-04-23T18:46:12.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:01.406+0000 7f1bb9ffb640 10 client.17532 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bb9ffb640 1 -- 172.21.15.33:0/2959220569 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d884a68200 con 0x55d884b7cf30 2024-04-23T18:46:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bb9ffb640 10 client.17532 handle_command_reply: tid=0 2024-04-23T18:46:12.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 client.17532 shutdown 2024-04-23T18:46:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 msgr2=0x55d884b0eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0043a0 tx=0x7f1bcc02a480 comp rx=0 tx=0).stop 2024-04-23T18:46:13.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 shutdown_connections 2024-04-23T18:46:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d884b7cf30 0x55d884b7f320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d884b06800 0x55d884b0c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d884afbf40 0x55d884b0bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 --2- 172.21.15.33:0/2959220569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d884afb620 0x55d884b0eb70 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:13.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bb9ffb640 0 client.17532 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:13.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 >> 172.21.15.33:0/2959220569 conn(0x55d884af7b30 msgr2=0x55d884af6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 shutdown_connections 2024-04-23T18:46:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 1 -- 172.21.15.33:0/2959220569 wait complete. 2024-04-23T18:46:13.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bdb202740 20 client.17532 trim_cache size 0 max 16384 2024-04-23T18:46:13.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:46:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:46:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:46:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:46:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 msgr2=0x7f1bbc0a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 0x7f1bbc0a62b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8005c40 tx=0x7f1bc8009d50 comp rx=0 tx=0).stop 2024-04-23T18:46:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 msgr2=0x7f1bd0101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.186+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f1bc000d950 tx=0x7f1bc000de10 comp rx=0 tx=0).stop 2024-04-23T18:46:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 shutdown_connections 2024-04-23T18:46:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1bbc0a3df0 0x7f1bbc0a62b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:14.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd010c550 0x7f1bd0101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1bd0103aa0 0x7f1bd0101310 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 --2- 172.21.15.33:0/906879923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1bd0103180 0x7f1bd0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 >> 172.21.15.33:0/906879923 conn(0x7f1bd0077fd0 msgr2=0x7f1bd00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:14.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 shutdown_connections 2024-04-23T18:46:14.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:03.187+0000 7f1bd84e8640 1 -- 172.21.15.33:0/906879923 wait complete. 2024-04-23T18:46:14.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:02 smithi033 ceph-mon[29274]: pgmap v557: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:14.530 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:46:14.562 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: in thread 7fd380980700 thread_name:md_log_replay 2024-04-23T18:46:14.604 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:46:14.646 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd391d41d20] 2024-04-23T18:46:14.707 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2: /usr/bin/ceph-mds(+0x549372) [0x55eb0e63a372] 2024-04-23T18:46:14.771 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb0e63db23] 2024-04-23T18:46:14.813 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 4: (MDLog::_replay_thread()+0x617) [0x55eb0e698647] 2024-04-23T18:46:14.867 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb0e6a1d31] 2024-04-23T18:46:14.904 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd391d371ca] 2024-04-23T18:46:14.938 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 7: clone() 2024-04-23T18:46:14.984 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:44:04.060+0000 7fd380980700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:46:15.013 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: in thread 7fd380980700 thread_name:md_log_replay 2024-04-23T18:46:15.088 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:15.121 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:46:15.171 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd391d41d20] 2024-04-23T18:46:15.230 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2: /usr/bin/ceph-mds(+0x549372) [0x55eb0e63a372] 2024-04-23T18:46:15.263 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb0e63db23] 2024-04-23T18:46:15.300 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 4: (MDLog::_replay_thread()+0x617) [0x55eb0e698647] 2024-04-23T18:46:15.330 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb0e6a1d31] 2024-04-23T18:46:15.363 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd391d371ca] 2024-04-23T18:46:15.396 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 7: clone() 2024-04-23T18:46:15.430 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:46:15.463 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:15.500 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 0> 2024-04-23T18:44:04.060+0000 7fd380980700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:46:15.613 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: in thread 7fd380980700 thread_name:md_log_replay 2024-04-23T18:46:15.647 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:15.680 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:46:15.713 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd391d41d20] 2024-04-23T18:46:15.772 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2: /usr/bin/ceph-mds(+0x549372) [0x55eb0e63a372] 2024-04-23T18:46:15.805 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb0e63db23] 2024-04-23T18:46:15.838 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 4: (MDLog::_replay_thread()+0x617) [0x55eb0e698647] 2024-04-23T18:46:15.872 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb0e6a1d31] 2024-04-23T18:46:15.919 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd391d371ca] 2024-04-23T18:46:15.947 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 7: clone() 2024-04-23T18:46:15.995 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:46:16.043 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:16.080 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 0> 2024-04-23T18:44:04.060+0000 7fd380980700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:46:16.114 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: in thread 7fd380980700 thread_name:md_log_replay 2024-04-23T18:46:16.190 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:16.222 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:46:16.255 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd391d41d20] 2024-04-23T18:46:16.354 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2: /usr/bin/ceph-mds(+0x549372) [0x55eb0e63a372] 2024-04-23T18:46:16.399 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55eb0e63db23] 2024-04-23T18:46:16.422 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 4: (MDLog::_replay_thread()+0x617) [0x55eb0e698647] 2024-04-23T18:46:16.514 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55eb0e6a1d31] 2024-04-23T18:46:16.548 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fd391d371ca] 2024-04-23T18:46:16.589 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 7: clone() 2024-04-23T18:46:16.632 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:46:16.656 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[92548]: 2024-04-23T18:46:16.689 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 podman[93334]: 2024-04-23 18:44:04.119249055 +0000 UTC m=+0.023496660 container died c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:46:16.723 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 podman[93334]: 2024-04-23 18:44:04.714482116 +0000 UTC m=+0.618729709 container remove c7532f5bafd4ffc9083a6d34ad7349ee837634fbf108eb959ecde5f9ff82a920 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T18:46:16.756 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:46:16.798 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:46:16.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-mon[29274]: pgmap v558: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:16.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:04 smithi033 ceph-mon[29274]: osdmap e117: 12 total, 12 up, 12 in 2024-04-23T18:46:16.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:06 smithi033 ceph-mon[29274]: pgmap v560: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:46:16.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:08 smithi033 ceph-mon[29274]: pgmap v561: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:46:16.990 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:10 smithi033 ceph-mon[29274]: pgmap v562: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:46:17.063 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:10 smithi033 ceph-mon[29274]: osdmap e118: 12 total, 12 up, 12 in 2024-04-23T18:46:17.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:12 smithi033 ceph-mon[29274]: pgmap v564: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-23T18:46:17.148 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:14 smithi033 ceph-mon[29274]: pgmap v565: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 30 op/s 2024-04-23T18:46:17.249 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:02 smithi122 ceph-mon[32546]: pgmap v622: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:17.291 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:04 smithi122 ceph-mon[32546]: pgmap v623: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:17.337 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:04 smithi122 ceph-mon[32546]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:46:17.373 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:06 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:46:17.407 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:07 smithi122 ceph-mon[32546]: pgmap v625: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:17.440 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:09 smithi122 ceph-mon[32546]: pgmap v626: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:17.473 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:11 smithi122 ceph-mon[32546]: pgmap v627: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:17.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:11 smithi122 ceph-mon[32546]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:46:17.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:13 smithi122 ceph-mon[32546]: pgmap v629: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 78 KiB/s wr, 40 op/s 2024-04-23T18:46:17.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:15 smithi122 ceph-mon[32546]: pgmap v630: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T18:46:17.623 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:16 smithi122 ceph-mon[32546]: daemon mds.d restarted 2024-04-23T18:46:17.657 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:16 smithi122 ceph-mon[32546]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:46:17.690 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/1458865300,v1:172.21.15.33:6837/1458865300] up:boot 2024-04-23T18:46:17.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:46:17.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:46:17.891 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:02 smithi106 ceph-mon[32716]: pgmap v622: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:17.924 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:04 smithi106 ceph-mon[32716]: pgmap v623: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:17.965 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:04 smithi106 ceph-mon[32716]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:46:18.032 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:06 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:46:18.066 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:07 smithi106 ceph-mon[32716]: pgmap v625: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:18.107 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:09 smithi106 ceph-mon[32716]: pgmap v626: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:18.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:11 smithi106 ceph-mon[32716]: pgmap v627: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:46:18.199 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:11 smithi106 ceph-mon[32716]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:46:18.232 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:13 smithi106 ceph-mon[32716]: pgmap v629: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 78 KiB/s wr, 40 op/s 2024-04-23T18:46:18.266 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:15 smithi106 ceph-mon[32716]: pgmap v630: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T18:46:18.299 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:16 smithi106 ceph-mon[32716]: daemon mds.d restarted 2024-04-23T18:46:18.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:16 smithi106 ceph-mon[32716]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:46:18.366 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/1458865300,v1:172.21.15.33:6837/1458865300] up:boot 2024-04-23T18:46:18.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:46:18.433 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:46:18.466 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-04-23T18:46:18.518 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:14 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:46:18.583 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:14 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:46:18.616 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 podman[93432]: 2024-04-23 18:44:15.215071827 +0000 UTC m=+0.098511217 container create eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-23T18:46:18.656 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 podman[93432]: 2024-04-23 18:44:15.136272082 +0000 UTC m=+0.019711433 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:46:18.691 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 podman[93432]: 2024-04-23 18:44:15.451036807 +0000 UTC m=+0.334476149 container init eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-23T18:46:18.724 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 podman[93432]: 2024-04-23 18:44:15.453950196 +0000 UTC m=+0.337389528 container start eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-23T18:46:18.758 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mds[93452]: -- 172.21.15.33:0/90452778 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56309e525a40 con 0x56309e4f8800 2024-04-23T18:46:18.791 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: starting mds.d at 2024-04-23T18:46:18.832 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 bash[93432]: eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 2024-04-23T18:46:18.866 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:44:15 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:46:18.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mon[29274]: daemon mds.d restarted 2024-04-23T18:46:18.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mon[29274]: osdmap e119: 12 total, 12 up, 12 in 2024-04-23T18:46:18.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/1425237468,v1:172.21.15.33:6837/1425237468] up:boot 2024-04-23T18:46:19.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:46:19.058 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:46:19.095 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:17 smithi122 ceph-mon[32546]: pgmap v631: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:46:19.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:17 smithi106 ceph-mon[32716]: pgmap v631: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:46:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 -- 172.21.15.33:0/3006936122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981031e0 msgr2=0x7fca981035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 --2- 172.21.15.33:0/3006936122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981031e0 0x7fca981035c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca8802a510 comp rx=0 tx=0).stop 2024-04-23T18:46:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 -- 172.21.15.33:0/3006936122 shutdown_connections 2024-04-23T18:46:19.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 --2- 172.21.15.33:0/3006936122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca9810c520 0x7fca9810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 --2- 172.21.15.33:0/3006936122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca98103b00 0x7fca9810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 --2- 172.21.15.33:0/3006936122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca981031e0 0x7fca981035c0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 -- 172.21.15.33:0/3006936122 >> 172.21.15.33:0/3006936122 conn(0x7fca98077fd0 msgr2=0x7fca980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 -- 172.21.15.33:0/3006936122 shutdown_connections 2024-04-23T18:46:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.279+0000 7fca9e344640 1 -- 172.21.15.33:0/3006936122 wait complete. 2024-04-23T18:46:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 Processor -- start 2024-04-23T18:46:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 -- start start 2024-04-23T18:46:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca981031e0 0x7fca981986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 0x7fca98198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca981121b0 con 0x7fca9810c520 2024-04-23T18:46:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca98112030 con 0x7fca981031e0 2024-04-23T18:46:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.280+0000 7fca9e344640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca98112330 con 0x7fca98103b00 2024-04-23T18:46:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:20.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 0x7fca98198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:20.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59074/0 (socket says 172.21.15.33:59074) 2024-04-23T18:46:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9c8e1640 1 -- 172.21.15.33:0/1401848917 learned_addr learned my addr 172.21.15.33:0/1401848917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca97fff640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca981031e0 0x7fca981986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:20.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 -- 172.21.15.33:0/1401848917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca981031e0 msgr2=0x7fca981986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:20.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca981031e0 0x7fca981986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:20.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 -- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 msgr2=0x7fca9819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:20.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 -- 172.21.15.33:0/1401848917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819d6a0 con 0x7fca98103b00 2024-04-23T18:46:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:46:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca977fe640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 0x7fca98198bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d6d0 tx=0x7fca8c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:20.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca957fa640 1 -- 172.21.15.33:0/1401848917 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c017070 con 0x7fca98103b00 2024-04-23T18:46:20.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819d990 con 0x7fca98103b00 2024-04-23T18:46:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.281+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a5220 con 0x7fca98103b00 2024-04-23T18:46:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.282+0000 7fca957fa640 1 -- 172.21.15.33:0/1401848917 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8c009820 con 0x7fca98103b00 2024-04-23T18:46:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.282+0000 7fca957fa640 1 -- 172.21.15.33:0/1401848917 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c020060 con 0x7fca98103b00 2024-04-23T18:46:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fca957fa640 1 -- 172.21.15.33:0/1401848917 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca8c009cd0 con 0x7fca98103b00 2024-04-23T18:46:21.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 Processor -- start 2024-04-23T18:46:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 -- start start 2024-04-23T18:46:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2442c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:21.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b246bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:21.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56298b19e970 con 0x56298b239620 2024-04-23T18:46:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56298b13b4a0 con 0x56298b239f40 2024-04-23T18:46:21.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fcaa1067740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56298b19a930 con 0x56298b244800 2024-04-23T18:46:21.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59078/0 (socket says 172.21.15.33:59078) 2024-04-23T18:46:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fca9c8e1640 1 -- 172.21.15.33:0/1906397385 learned_addr learned my addr 172.21.15.33:0/1906397385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.284+0000 7fca957fa640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 0x7fca800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca977fe640 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b246bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:21.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca97fff640 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 0x7fca800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca957fa640 1 -- 172.21.15.33:0/1401848917 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7fca8c0d0270 con 0x7fca98103b00 2024-04-23T18:46:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 -- 172.21.15.33:0/1906397385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 msgr2=0x56298b246bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:21.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b246bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 -- 172.21.15.33:0/1906397385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 msgr2=0x56298b2442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:21.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 -- 172.21.15.33:0/1906397385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298aeb1210 con 0x56298b239620 2024-04-23T18:46:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fca90009fd0 tx=0x7fca9000f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8881b8a9d7333b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca7e7fc640 1 -- 172.21.15.33:0/1906397385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90022020 con 0x56298b239620 2024-04-23T18:46:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca7e7fc640 1 -- 172.21.15.33:0/1906397385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90013110 con 0x56298b239620 2024-04-23T18:46:22.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.285+0000 7fca7e7fc640 1 -- 172.21.15.33:0/1906397385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90010d50 con 0x56298b239620 2024-04-23T18:46:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- 172.21.15.33:0/1906397385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 msgr2=0x56298b239a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:22.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fca90009fd0 tx=0x7fca9000f750 comp rx=0 tx=0).stop 2024-04-23T18:46:22.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 0x7fca800a6280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca90023000 tx=0x7fca90021000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:22.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- 172.21.15.33:0/1906397385 shutdown_connections 2024-04-23T18:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b246bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:22.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 --2- 172.21.15.33:0/1906397385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b239a00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:22.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- 172.21.15.33:0/1906397385 >> 172.21.15.33:0/1906397385 conn(0x56298b235b30 msgr2=0x56298b235f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:22.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- 172.21.15.33:0/1906397385 shutdown_connections 2024-04-23T18:46:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- 172.21.15.33:0/1906397385 wait complete. 2024-04-23T18:46:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.286+0000 7fcaa1067740 1 -- start start 2024-04-23T18:46:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 Processor -- start 2024-04-23T18:46:22.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:22.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b241900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56298b247430 con 0x56298b239620 2024-04-23T18:46:22.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56298b1390b0 con 0x56298b239f40 2024-04-23T18:46:22.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56298b2475b0 con 0x56298b244800 2024-04-23T18:46:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fcaa1067740 10 client.0 _ll_register_callbacks cb 0x56298b126d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59084/0 (socket says 172.21.15.33:59084) 2024-04-23T18:46:23.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 -- 172.21.15.33:0/2630076937 learned_addr learned my addr 172.21.15.33:0/2630076937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b241900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca977fe640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:23.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 -- 172.21.15.33:0/2630076937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 msgr2=0x56298b241900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:23.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b241900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:23.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 -- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 msgr2=0x56298b2413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:23.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:23.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca97fff640 1 -- 172.21.15.33:0/2630076937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298af7d620 con 0x56298b239620 2024-04-23T18:46:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.287+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b241900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:23.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca97fff640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca880316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91afc61c0f31f913 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:23.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:23.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88035040 con 0x56298b239620 2024-04-23T18:46:23.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7fca88004040 con 0x56298b239620 2024-04-23T18:46:23.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca90013110 con 0x56298b239620 2024-04-23T18:46:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fca88004240 con 0x56298b239620 2024-04-23T18:46:23.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 10 client.17550 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:23.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56298b253320 con 0x56298b239620 2024-04-23T18:46:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90013110 con 0x56298b239620 2024-04-23T18:46:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fca88004850 con 0x56298b239620 2024-04-23T18:46:24.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 10 client.17550 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:24.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:24.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 20 client.17550 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:24.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b254790 0x56298b256bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:24.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 4 client.17550 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:46:24.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.288+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56298b1ac390 con 0x56298b254790 2024-04-23T18:46:24.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.289+0000 7fca977fe640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b254790 0x56298b256bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:24.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.289+0000 7fca977fe640 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b254790 0x56298b256bc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1667783bec6fcdfa server_cookie=32579b13e1aec6b8 in_seq=0 out_seq=0 2024-04-23T18:46:24.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.289+0000 7fca777fe640 10 client.17550 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fca777fe640 1 -- 172.21.15.33:0/2630076937 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56298b1ac390 con 0x56298b254790 2024-04-23T18:46:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fca777fe640 10 client.17550 handle_command_reply: tid=0 2024-04-23T18:46:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 client.17550 shutdown 2024-04-23T18:46:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 msgr2=0x56298b2440d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fca88009900 tx=0x7fca880316d0 comp rx=0 tx=0).stop 2024-04-23T18:46:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 shutdown_connections 2024-04-23T18:46:24.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b254790 0x56298b256bc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:24.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b241900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:24.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fca777fe640 0 client.17550 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239f40 0x56298b2413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:24.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 --2- 172.21.15.33:0/2630076937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239620 0x56298b2440d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 >> 172.21.15.33:0/2630076937 conn(0x56298b235b30 msgr2=0x56298b23b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 shutdown_connections 2024-04-23T18:46:25.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 1 -- 172.21.15.33:0/2630076937 wait complete. 2024-04-23T18:46:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.290+0000 7fcaa1067740 20 client.17550 trim_cache size 0 max 16384 2024-04-23T18:46:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 Processor -- start 2024-04-23T18:46:25.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 -- start start 2024-04-23T18:46:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b2502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:25.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b25ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:25.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca981af580 con 0x56298b239f40 2024-04-23T18:46:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca981ad370 con 0x56298b239620 2024-04-23T18:46:25.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fcaa1067740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca981aeeb0 con 0x56298b244800 2024-04-23T18:46:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fca977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fca977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59098/0 (socket says 172.21.15.33:59098) 2024-04-23T18:46:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.305+0000 7fca977fe640 1 -- 172.21.15.33:0/3391472514 learned_addr learned my addr 172.21.15.33:0/3391472514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b25ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca97fff640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b2502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 -- 172.21.15.33:0/3391472514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 msgr2=0x56298b25ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b25ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 -- 172.21.15.33:0/3391472514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 msgr2=0x56298b2502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b2502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:25.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 -- 172.21.15.33:0/3391472514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298b238e20 con 0x56298b239f40 2024-04-23T18:46:26.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca97fff640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b2502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:26.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca977fe640 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d6a0 tx=0x7fca8c06dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e37882a06bc30f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca7dffb640 1 -- 172.21.15.33:0/3391472514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0a4e90 con 0x56298b239f40 2024-04-23T18:46:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca7dffb640 1 -- 172.21.15.33:0/3391472514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8c0d0070 con 0x56298b239f40 2024-04-23T18:46:26.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.306+0000 7fca7dffb640 1 -- 172.21.15.33:0/3391472514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca8c005220 con 0x56298b239f40 2024-04-23T18:46:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- 172.21.15.33:0/3391472514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 msgr2=0x56298b25c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d6a0 tx=0x7fca8c06dd10 comp rx=0 tx=0).stop 2024-04-23T18:46:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- 172.21.15.33:0/3391472514 shutdown_connections 2024-04-23T18:46:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b244800 0x56298b25ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b239f40 0x56298b25c4d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 --2- 172.21.15.33:0/3391472514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b2502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:26.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- 172.21.15.33:0/3391472514 >> 172.21.15.33:0/3391472514 conn(0x56298b235b30 msgr2=0x56298b240c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- 172.21.15.33:0/3391472514 shutdown_connections 2024-04-23T18:46:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- 172.21.15.33:0/3391472514 wait complete. 2024-04-23T18:46:26.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 -- start start 2024-04-23T18:46:26.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.307+0000 7fcaa1067740 1 Processor -- start 2024-04-23T18:46:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b24ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b239f40 0x56298b24b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:26.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca981ae110 con 0x56298b244800 2024-04-23T18:46:26.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fca981ada40 con 0x56298b239620 2024-04-23T18:46:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca981ae7e0 con 0x56298b239f40 2024-04-23T18:46:27.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 10 client.0 _ll_register_callbacks cb 0x7fca88032cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59104/0 (socket says 172.21.15.33:59104) 2024-04-23T18:46:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 -- 172.21.15.33:0/836989000 learned_addr learned my addr 172.21.15.33:0/836989000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca97fff640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b24ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:27.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca977fe640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b239f40 0x56298b24b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 -- 172.21.15.33:0/836989000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b239f40 msgr2=0x56298b24b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b239f40 0x56298b24b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 -- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 msgr2=0x56298b24ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b24ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:27.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 -- 172.21.15.33:0/836989000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298b2b4380 con 0x56298b244800 2024-04-23T18:46:27.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca9c8e1640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fca900094a0 tx=0x7fca90009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea1235e8fae9a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90037060 con 0x56298b244800 2024-04-23T18:46:27.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7fca900103d0 con 0x56298b244800 2024-04-23T18:46:27.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.308+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca8c0d0070 con 0x56298b244800 2024-04-23T18:46:27.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fca90022050 con 0x56298b244800 2024-04-23T18:46:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 10 client.17556 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56298b2bb730 con 0x56298b244800 2024-04-23T18:46:27.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8c0d0070 con 0x56298b244800 2024-04-23T18:46:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fca9000f040 con 0x56298b244800 2024-04-23T18:46:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 10 client.17556 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 10 client.17556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 20 client.17556 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b2bbd10 0x56298b2be130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:28.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 4 client.17556 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:46:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56298b1a6200 con 0x56298b2bbd10 2024-04-23T18:46:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.309+0000 7fca97fff640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b2bbd10 0x56298b2be130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.310+0000 7fca97fff640 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b2bbd10 0x56298b2be130 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c71f7b3aab42bde server_cookie=8e2fdbbe974a5df5 in_seq=0 out_seq=0 2024-04-23T18:46:28.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.310+0000 7fca777fe640 10 client.17556 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.316+0000 7fca777fe640 1 -- 172.21.15.33:0/836989000 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56298b1a6200 con 0x56298b2bbd10 2024-04-23T18:46:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.316+0000 7fca777fe640 10 client.17556 handle_command_reply: tid=0 2024-04-23T18:46:28.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.316+0000 7fcaa1067740 1 client.17556 shutdown 2024-04-23T18:46:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 msgr2=0x56298b2b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fca900094a0 tx=0x7fca90009900 comp rx=0 tx=0).stop 2024-04-23T18:46:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 shutdown_connections 2024-04-23T18:46:28.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56298b2bbd10 0x56298b2be130 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56298b244800 0x56298b2b3d90 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fca777fe640 0 client.17556 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56298b239f40 0x56298b24b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 --2- 172.21.15.33:0/836989000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56298b239620 0x56298b24ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 >> 172.21.15.33:0/836989000 conn(0x56298b235b30 msgr2=0x56298b234cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:28.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 shutdown_connections 2024-04-23T18:46:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 1 -- 172.21.15.33:0/836989000 wait complete. 2024-04-23T18:46:29.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fcaa1067740 20 client.17556 trim_cache size 0 max 16384 2024-04-23T18:46:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:46:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:46:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:46:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:46:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 msgr2=0x7fca800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:29.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 0x7fca800a6280 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca90023000 tx=0x7fca90021000 comp rx=0 tx=0).stop 2024-04-23T18:46:29.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 msgr2=0x7fca98198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.317+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 0x7fca98198bf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00d6d0 tx=0x7fca8c00db90 comp rx=0 tx=0).stop 2024-04-23T18:46:29.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 shutdown_connections 2024-04-23T18:46:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fca800a3dc0 0x7fca800a6280 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca9810c520 0x7fca9819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:29.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca98103b00 0x7fca98198bf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:29.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 --2- 172.21.15.33:0/1401848917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fca981031e0 0x7fca981986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 >> 172.21.15.33:0/1401848917 conn(0x7fca98077fd0 msgr2=0x7fca980fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:29.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 shutdown_connections 2024-04-23T18:46:29.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:16.318+0000 7fca9e344640 1 -- 172.21.15.33:0/1401848917 wait complete. 2024-04-23T18:46:29.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:16 smithi033 ceph-mon[29274]: pgmap v566: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:46:29.765 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:18 smithi033 ceph-mon[29274]: pgmap v568: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:46:29.802 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:20 smithi033 ceph-mon[29274]: pgmap v569: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:46:29.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:22 smithi033 ceph-mon[29274]: pgmap v570: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:46:29.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:25 smithi033 ceph-mon[29274]: pgmap v571: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:46:29.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:27 smithi033 ceph-mon[29274]: pgmap v572: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-04-23T18:46:29.935 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:19 smithi122 ceph-mon[32546]: pgmap v633: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:46:29.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:21 smithi122 ceph-mon[32546]: pgmap v634: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-23T18:46:29.999 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:23 smithi122 ceph-mon[32546]: pgmap v635: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:46:30.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:25 smithi122 ceph-mon[32546]: pgmap v636: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:46:30.115 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:27 smithi122 ceph-mon[32546]: pgmap v637: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-23T18:46:30.149 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:29 smithi122 ceph-mon[32546]: pgmap v638: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:46:30.224 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:19 smithi106 ceph-mon[32716]: pgmap v633: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:46:30.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:21 smithi106 ceph-mon[32716]: pgmap v634: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-23T18:46:30.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:23 smithi106 ceph-mon[32716]: pgmap v635: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:46:30.375 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:25 smithi106 ceph-mon[32716]: pgmap v636: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:46:30.409 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:27 smithi106 ceph-mon[32716]: pgmap v637: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-23T18:46:30.441 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:29 smithi106 ceph-mon[32716]: pgmap v638: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:46:30.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.417+0000 7f8afa220640 1 -- 172.21.15.33:0/167052988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41061b0 msgr2=0x7f8af410dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.417+0000 7f8afa220640 1 --2- 172.21.15.33:0/167052988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41061b0 0x7f8af410dd80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009ca0 tx=0x7f8aec02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 -- 172.21.15.33:0/167052988 shutdown_connections 2024-04-23T18:46:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 --2- 172.21.15.33:0/167052988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41061b0 0x7f8af410dd80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:30.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 --2- 172.21.15.33:0/167052988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af4101a90 0x7f8af4105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 --2- 172.21.15.33:0/167052988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 0x7f8af41014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 -- 172.21.15.33:0/167052988 >> 172.21.15.33:0/167052988 conn(0x7f8af40771d0 msgr2=0x7f8af40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:30.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 -- 172.21.15.33:0/167052988 shutdown_connections 2024-04-23T18:46:30.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 -- 172.21.15.33:0/167052988 wait complete. 2024-04-23T18:46:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 Processor -- start 2024-04-23T18:46:31.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.418+0000 7f8afa220640 1 -- start start 2024-04-23T18:46:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 0x7f8af4102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:31.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af41061b0 0x7f8af41030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:31.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4111170 con 0x7f8af41035f0 2024-04-23T18:46:31.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8af4110ff0 con 0x7f8af41061b0 2024-04-23T18:46:31.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8afa220640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41112f0 con 0x7f8af41010e0 2024-04-23T18:46:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44890/0 (socket says 172.21.15.33:44890) 2024-04-23T18:46:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 -- 172.21.15.33:0/617940765 learned_addr learned my addr 172.21.15.33:0/617940765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af41061b0 0x7f8af41030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:31.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af37fe640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 0x7f8af4102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 -- 172.21.15.33:0/617940765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 msgr2=0x7f8af4102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 0x7f8af4102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 -- 172.21.15.33:0/617940765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af41061b0 msgr2=0x7f8af41030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af41061b0 0x7f8af41030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 -- 172.21.15.33:0/617940765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af41a6e90 con 0x7f8af41035f0 2024-04-23T18:46:31.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.419+0000 7f8af3fff640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009430 tx=0x7f8aec0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.420+0000 7f8af0ff9640 1 -- 172.21.15.33:0/617940765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec003b70 con 0x7f8af41035f0 2024-04-23T18:46:31.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.420+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af41a7120 con 0x7f8af41035f0 2024-04-23T18:46:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.420+0000 7f8af0ff9640 1 -- 172.21.15.33:0/617940765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec003d10 con 0x7f8af41035f0 2024-04-23T18:46:31.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.420+0000 7f8af0ff9640 1 -- 172.21.15.33:0/617940765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec02fa30 con 0x7f8af41035f0 2024-04-23T18:46:31.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.420+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af41a7670 con 0x7f8af41035f0 2024-04-23T18:46:31.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.422+0000 7f8af0ff9640 1 -- 172.21.15.33:0/617940765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8aec02fbd0 con 0x7f8af41035f0 2024-04-23T18:46:31.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 Processor -- start 2024-04-23T18:46:31.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 -- start start 2024-04-23T18:46:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af0ff9640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 0x7f8adc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae4463d970 con 0x55ae446e3800 2024-04-23T18:46:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ae445da4a0 con 0x55ae446d8620 2024-04-23T18:46:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8afd00e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae44639930 con 0x55ae446d8f40 2024-04-23T18:46:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 0x7f8adc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44898/0 (socket says 172.21.15.33:44898) 2024-04-23T18:46:32.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af3fff640 1 -- 172.21.15.33:0/2752039898 learned_addr learned my addr 172.21.15.33:0/2752039898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af0ff9640 1 -- 172.21.15.33:0/617940765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f8aec0f1be0 con 0x7f8af41035f0 2024-04-23T18:46:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.423+0000 7f8af37fe640 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:32.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 -- 172.21.15.33:0/2752039898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 msgr2=0x55ae446e32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:32.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:32.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 -- 172.21.15.33:0/2752039898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 msgr2=0x55ae446d8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:32.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446d8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:32.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 -- 172.21.15.33:0/2752039898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae44350210 con 0x55ae446e3800 2024-04-23T18:46:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 0x7f8adc0a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005e10 tx=0x7f8ae800aa80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a1100 tx=0x7f8aec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a1e022147ff096 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8ad9ffb640 1 -- 172.21.15.33:0/2752039898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0bd5a0 con 0x55ae446e3800 2024-04-23T18:46:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8ad9ffb640 1 -- 172.21.15.33:0/2752039898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.424+0000 7f8ad9ffb640 1 -- 172.21.15.33:0/2752039898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0cb6a0 con 0x55ae446e3800 2024-04-23T18:46:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- 172.21.15.33:0/2752039898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 msgr2=0x55ae446e5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a1100 tx=0x7f8aec0095c0 comp rx=0 tx=0).stop 2024-04-23T18:46:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- 172.21.15.33:0/2752039898 shutdown_connections 2024-04-23T18:46:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e5bf0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2752039898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:33.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- 172.21.15.33:0/2752039898 >> 172.21.15.33:0/2752039898 conn(0x55ae446d4b30 msgr2=0x55ae446d4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- 172.21.15.33:0/2752039898 shutdown_connections 2024-04-23T18:46:33.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- 172.21.15.33:0/2752039898 wait complete. 2024-04-23T18:46:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 -- start start 2024-04-23T18:46:33.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.425+0000 7f8afd00e740 1 Processor -- start 2024-04-23T18:46:33.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae4463d970 con 0x55ae446e3800 2024-04-23T18:46:33.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ae445d80b0 con 0x55ae446d8620 2024-04-23T18:46:33.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae446e61f0 con 0x55ae446d8f40 2024-04-23T18:46:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8afd00e740 10 client.0 _ll_register_callbacks cb 0x55ae445c5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44900/0 (socket says 172.21.15.33:44900) 2024-04-23T18:46:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 -- 172.21.15.33:0/3195018767 learned_addr learned my addr 172.21.15.33:0/3195018767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af37fe640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 -- 172.21.15.33:0/3195018767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 msgr2=0x55ae446e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:34.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 -- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 msgr2=0x55ae446e31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:34.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 -- 172.21.15.33:0/3195018767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4441c620 con 0x55ae446e3800 2024-04-23T18:46:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af37fe640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8af3fff640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a70d0 tx=0x7f8aec0bd5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c051096713091a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0be340 con 0x55ae446e3800 2024-04-23T18:46:34.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.426+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f8aec0ef560 con 0x55ae446e3800 2024-04-23T18:46:34.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8aec0c1ff0 con 0x55ae446e3800 2024-04-23T18:46:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 10 client.17568 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae446f1f50 con 0x55ae446e3800 2024-04-23T18:46:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f8aec0bdea0 con 0x55ae446e3800 2024-04-23T18:46:34.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 10 client.17568 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 10 client.17568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:34.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 20 client.17568 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae446f3360 0x55ae446f5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:34.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 4 client.17568 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:46:34.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4464b390 con 0x55ae446f3360 2024-04-23T18:46:34.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.427+0000 7f8af37fe640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae446f3360 0x55ae446f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:34.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.428+0000 7f8ad2ffd640 10 client.17568 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:34.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.428+0000 7f8af37fe640 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae446f3360 0x55ae446f5750 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8ea946ebdea045 server_cookie=b1558cac89b9b274 in_seq=0 out_seq=0 2024-04-23T18:46:35.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.428+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/3195018767 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae4464b390 con 0x55ae446f3360 2024-04-23T18:46:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.428+0000 7f8ad2ffd640 10 client.17568 handle_command_reply: tid=0 2024-04-23T18:46:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 client.17568 shutdown 2024-04-23T18:46:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 msgr2=0x55ae446e0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0a70d0 tx=0x7f8aec0bd5a0 comp rx=0 tx=0).stop 2024-04-23T18:46:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 shutdown_connections 2024-04-23T18:46:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae446f3360 0x55ae446f5750 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e0a30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:35.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8ad2ffd640 0 client.17568 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 --2- 172.21.15.33:0/3195018767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 >> 172.21.15.33:0/3195018767 conn(0x55ae446d4b30 msgr2=0x55ae446da8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:35.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 shutdown_connections 2024-04-23T18:46:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 1 -- 172.21.15.33:0/3195018767 wait complete. 2024-04-23T18:46:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.429+0000 7f8afd00e740 20 client.17568 trim_cache size 0 max 16384 2024-04-23T18:46:35.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 Processor -- start 2024-04-23T18:46:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 -- start start 2024-04-23T18:46:35.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8620 0x55ae446eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:35.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8f40 0x55ae446fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:35.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41b1660 con 0x55ae446e3800 2024-04-23T18:46:35.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41af450 con 0x55ae446d8f40 2024-04-23T18:46:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8afd00e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41b0f90 con 0x55ae446d8620 2024-04-23T18:46:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44910/0 (socket says 172.21.15.33:44910) 2024-04-23T18:46:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.444+0000 7f8af3fff640 1 -- 172.21.15.33:0/2111226864 learned_addr learned my addr 172.21.15.33:0/2111226864 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:36.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8f40 0x55ae446fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af37fe640 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8620 0x55ae446eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:36.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 -- 172.21.15.33:0/2111226864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8620 msgr2=0x55ae446eeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8620 0x55ae446eeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 -- 172.21.15.33:0/2111226864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8f40 msgr2=0x55ae446fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:36.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8f40 0x55ae446fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 -- 172.21.15.33:0/2111226864 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae446d0500 con 0x55ae446e3800 2024-04-23T18:46:36.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8af3fff640 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ba0a0 tx=0x7f8aec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905e9490a15f42e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8ad97fa640 1 -- 172.21.15.33:0/2111226864 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0be340 con 0x55ae446e3800 2024-04-23T18:46:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8ad97fa640 1 -- 172.21.15.33:0/2111226864 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.445+0000 7f8ad97fa640 1 -- 172.21.15.33:0/2111226864 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0bda10 con 0x55ae446e3800 2024-04-23T18:46:36.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- 172.21.15.33:0/2111226864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 msgr2=0x55ae446fb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ba0a0 tx=0x7f8aec0095c0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- 172.21.15.33:0/2111226864 shutdown_connections 2024-04-23T18:46:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446fb560 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8f40 0x55ae446fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- 172.21.15.33:0/2111226864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8620 0x55ae446eeea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- 172.21.15.33:0/2111226864 >> 172.21.15.33:0/2111226864 conn(0x55ae446d4b30 msgr2=0x55ae446d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- 172.21.15.33:0/2111226864 shutdown_connections 2024-04-23T18:46:36.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- 172.21.15.33:0/2111226864 wait complete. 2024-04-23T18:46:36.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- start start 2024-04-23T18:46:37.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 Processor -- start 2024-04-23T18:46:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:37.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.446+0000 7f8afd00e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41b01f0 con 0x55ae446e3800 2024-04-23T18:46:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41afb20 con 0x55ae446d8620 2024-04-23T18:46:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8af41b08c0 con 0x55ae446d8f40 2024-04-23T18:46:37.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 10 client.0 _ll_register_callbacks cb 0x7f8aec0c68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:37.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44920/0 (socket says 172.21.15.33:44920) 2024-04-23T18:46:37.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:37.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 -- 172.21.15.33:0/999545770 learned_addr learned my addr 172.21.15.33:0/999545770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af2ffd640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 -- 172.21.15.33:0/999545770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 msgr2=0x55ae446e9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:37.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 -- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 msgr2=0x55ae446e8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 -- 172.21.15.33:0/999545770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae447588a0 con 0x55ae446e3800 2024-04-23T18:46:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8af3fff640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ba5c0 tx=0x7f8aec02f500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3767f204351ca4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:37.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0bda10 con 0x55ae446e3800 2024-04-23T18:46:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f8aec0c6650 con 0x55ae446e3800 2024-04-23T18:46:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:38.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8aec0be340 con 0x55ae446e3800 2024-04-23T18:46:38.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 10 client.17574 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.447+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae447588a0 con 0x55ae446e3800 2024-04-23T18:46:38.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec036070 con 0x55ae446e3800 2024-04-23T18:46:38.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f8aec0c6b00 con 0x55ae446e3800 2024-04-23T18:46:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 10 client.17574 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:38.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 10 client.17574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:38.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 20 client.17574 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae44759e40 0x55ae4475c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 4 client.17574 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:46:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae44645200 con 0x55ae44759e40 2024-04-23T18:46:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8af37fe640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae44759e40 0x55ae4475c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:38.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8af37fe640 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae44759e40 0x55ae4475c230 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=905b7cdc0b590b36 server_cookie=5434f8e5f2af0175 in_seq=0 out_seq=0 2024-04-23T18:46:38.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.448+0000 7f8ad2ffd640 10 client.17574 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8ad2ffd640 1 -- 172.21.15.33:0/999545770 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae44645200 con 0x55ae44759e40 2024-04-23T18:46:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8ad2ffd640 10 client.17574 handle_command_reply: tid=0 2024-04-23T18:46:38.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 client.17574 shutdown 2024-04-23T18:46:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 msgr2=0x55ae446e9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:38.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8aec0ba5c0 tx=0x7f8aec02f500 comp rx=0 tx=0).stop 2024-04-23T18:46:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 shutdown_connections 2024-04-23T18:46:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae44759e40 0x55ae4475c230 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae446e3800 0x55ae446e9870 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae446d8f40 0x55ae446e9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8ad2ffd640 0 client.17574 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 --2- 172.21.15.33:0/999545770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae446d8620 0x55ae446e8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.453+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 >> 172.21.15.33:0/999545770 conn(0x55ae446d4b30 msgr2=0x55ae446d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 shutdown_connections 2024-04-23T18:46:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afd00e740 1 -- 172.21.15.33:0/999545770 wait complete. 2024-04-23T18:46:39.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afd00e740 20 client.17574 trim_cache size 0 max 16384 2024-04-23T18:46:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:46:39.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:46:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:46:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:46:39.383 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:31 smithi122 ceph-mon[32546]: pgmap v639: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:46:39.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:33 smithi122 ceph-mon[32546]: pgmap v640: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-04-23T18:46:39.464 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:35 smithi122 ceph-mon[32546]: pgmap v641: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:39.498 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:37 smithi122 ceph-mon[32546]: pgmap v642: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:39.532 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:31 smithi106 ceph-mon[32716]: pgmap v639: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:46:39.605 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:33 smithi106 ceph-mon[32716]: pgmap v640: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-04-23T18:46:39.647 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:35 smithi106 ceph-mon[32716]: pgmap v641: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:39.723 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:37 smithi106 ceph-mon[32716]: pgmap v642: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:39.757 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:39 smithi106 ceph-mon[32716]: pgmap v643: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:39.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 msgr2=0x7f8adc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 0x7f8adc0a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005e10 tx=0x7f8ae800aa80 comp rx=0 tx=0).stop 2024-04-23T18:46:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 msgr2=0x7f8af41a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:39.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009430 tx=0x7f8aec0319f0 comp rx=0 tx=0).stop 2024-04-23T18:46:39.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 shutdown_connections 2024-04-23T18:46:39.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8adc0a3dc0 0x7f8adc0a6280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8af41035f0 0x7f8af41a67f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8af41061b0 0x7f8af41030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 --2- 172.21.15.33:0/617940765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8af41010e0 0x7f8af4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 >> 172.21.15.33:0/617940765 conn(0x7f8af40771d0 msgr2=0x7f8af406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 shutdown_connections 2024-04-23T18:46:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:29.454+0000 7f8afa220640 1 -- 172.21.15.33:0/617940765 wait complete. 2024-04-23T18:46:40.259 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:29 smithi033 ceph-mon[29274]: pgmap v573: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:46:40.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:31 smithi033 ceph-mon[29274]: pgmap v574: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-04-23T18:46:40.341 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:33 smithi033 ceph-mon[29274]: pgmap v575: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-23T18:46:40.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:35 smithi033 ceph-mon[29274]: pgmap v576: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:40.493 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:37 smithi033 ceph-mon[29274]: pgmap v577: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:40.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:39 smithi033 ceph-mon[29274]: pgmap v578: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:40.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:41 smithi033 ceph-mon[29274]: pgmap v579: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:40.594 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:39 smithi122 ceph-mon[32546]: pgmap v643: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.554+0000 7f6f017ad640 1 -- 172.21.15.33:0/1952401431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc106170 msgr2=0x7f6efc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.554+0000 7f6f017ad640 1 --2- 172.21.15.33:0/1952401431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc106170 0x7f6efc10dd40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009ca0 tx=0x7f6ef402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 -- 172.21.15.33:0/1952401431 shutdown_connections 2024-04-23T18:46:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 --2- 172.21.15.33:0/1952401431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc106170 0x7f6efc10dd40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 --2- 172.21.15.33:0/1952401431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc101aa0 0x7f6efc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 --2- 172.21.15.33:0/1952401431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc1010f0 0x7f6efc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 -- 172.21.15.33:0/1952401431 >> 172.21.15.33:0/1952401431 conn(0x7f6efc077fd0 msgr2=0x7f6efc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 -- 172.21.15.33:0/1952401431 shutdown_connections 2024-04-23T18:46:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 -- 172.21.15.33:0/1952401431 wait complete. 2024-04-23T18:46:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.555+0000 7f6f017ad640 1 Processor -- start 2024-04-23T18:46:41.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 -- start start 2024-04-23T18:46:41.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:41.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 0x7f6efc198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc19cf50 0x7f6efc19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1121f0 con 0x7f6efc1010f0 2024-04-23T18:46:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112070 con 0x7f6efc106170 2024-04-23T18:46:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6f017ad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112370 con 0x7f6efc19cf50 2024-04-23T18:46:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:41.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38294/0 (socket says 172.21.15.33:38294) 2024-04-23T18:46:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 -- 172.21.15.33:0/3798352508 learned_addr learned my addr 172.21.15.33:0/3798352508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc19cf50 0x7f6efc19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 0x7f6efc198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 -- 172.21.15.33:0/3798352508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc19cf50 msgr2=0x7f6efc19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:41.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc19cf50 0x7f6efc19d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:41.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 -- 172.21.15.33:0/3798352508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 msgr2=0x7f6efc198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 0x7f6efc198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 -- 172.21.15.33:0/3798352508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc19da40 con 0x7f6efc1010f0 2024-04-23T18:46:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 0x7f6efc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.556+0000 7f6efaffd640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c920 tx=0x7f6eec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.557+0000 7f6ee3fff640 1 -- 172.21.15.33:0/3798352508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec007c70 con 0x7f6efc1010f0 2024-04-23T18:46:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.557+0000 7f6ee3fff640 1 -- 172.21.15.33:0/3798352508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec007e10 con 0x7f6efc1010f0 2024-04-23T18:46:42.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.557+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc19dd30 con 0x7f6efc1010f0 2024-04-23T18:46:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.557+0000 7f6ee3fff640 1 -- 172.21.15.33:0/3798352508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec018820 con 0x7f6efc1010f0 2024-04-23T18:46:42.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.557+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1a5520 con 0x7f6efc1010f0 2024-04-23T18:46:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.559+0000 7f6ee3fff640 1 -- 172.21.15.33:0/3798352508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6eec00f450 con 0x7f6efc1010f0 2024-04-23T18:46:42.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.559+0000 7f6f045a6740 1 Processor -- start 2024-04-23T18:46:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 -- start start 2024-04-23T18:46:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:42.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56535400a970 con 0x5653540a5620 2024-04-23T18:46:42.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x565353fa74a0 con 0x5653540b0800 2024-04-23T18:46:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6f045a6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565354006930 con 0x5653540a5f40 2024-04-23T18:46:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6ee3fff640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 0x7f6ee40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:42.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38300/0 (socket says 172.21.15.33:38300) 2024-04-23T18:46:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 -- 172.21.15.33:0/1287775635 learned_addr learned my addr 172.21.15.33:0/1287775635 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:42.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 0x7f6ee40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6ee3fff640 1 -- 172.21.15.33:0/3798352508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f6eec0d0970 con 0x7f6efc1010f0 2024-04-23T18:46:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efaffd640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:43.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 -- 172.21.15.33:0/1287775635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 msgr2=0x5653540b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:43.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 -- 172.21.15.33:0/1287775635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 msgr2=0x5653540b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efb7fe640 1 -- 172.21.15.33:0/1287775635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565353d1d210 con 0x5653540a5620 2024-04-23T18:46:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.560+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:43.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4035000 tx=0x7f6ef4031e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61ca85e2e737cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6ee17fa640 1 -- 172.21.15.33:0/1287775635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4004d40 con 0x5653540a5620 2024-04-23T18:46:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6ee17fa640 1 -- 172.21.15.33:0/1287775635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4036070 con 0x5653540a5620 2024-04-23T18:46:43.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 0x7f6ee40a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ef404e000 tx=0x7f6ef404c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6ee17fa640 1 -- 172.21.15.33:0/1287775635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4004850 con 0x5653540a5620 2024-04-23T18:46:43.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 -- 172.21.15.33:0/1287775635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 msgr2=0x5653540a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:43.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4035000 tx=0x7f6ef4031e40 comp rx=0 tx=0).stop 2024-04-23T18:46:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 -- 172.21.15.33:0/1287775635 shutdown_connections 2024-04-23T18:46:43.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:43.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 --2- 172.21.15.33:0/1287775635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540a5a00 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.561+0000 7f6f045a6740 1 -- 172.21.15.33:0/1287775635 >> 172.21.15.33:0/1287775635 conn(0x5653540a1b30 msgr2=0x5653540a1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:43.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- 172.21.15.33:0/1287775635 shutdown_connections 2024-04-23T18:46:43.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- 172.21.15.33:0/1287775635 wait complete. 2024-04-23T18:46:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- start start 2024-04-23T18:46:43.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 Processor -- start 2024-04-23T18:46:44.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653540b32b0 con 0x5653540a5620 2024-04-23T18:46:44.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x565353fa50b0 con 0x5653540b0800 2024-04-23T18:46:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653540b3430 con 0x5653540a5f40 2024-04-23T18:46:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6f045a6740 10 client.0 _ll_register_callbacks cb 0x565353f92d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38304/0 (socket says 172.21.15.33:38304) 2024-04-23T18:46:44.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.562+0000 7f6efaffd640 1 -- 172.21.15.33:0/2288050582 learned_addr learned my addr 172.21.15.33:0/2288050582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:44.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:44.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 -- 172.21.15.33:0/2288050582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 msgr2=0x5653540ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 -- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 msgr2=0x5653540adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:44.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 -- 172.21.15.33:0/2288050582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565353de9620 con 0x5653540a5620 2024-04-23T18:46:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:45.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6efaffd640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6eec098d20 tx=0x7f6eec09c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fc323a809d79e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6edeff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0a5530 con 0x5653540a5620 2024-04-23T18:46:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f6eec098f30 con 0x5653540a5620 2024-04-23T18:46:45.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef4036070 con 0x5653540a5620 2024-04-23T18:46:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6eec0ce2f0 con 0x5653540a5620 2024-04-23T18:46:45.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6f045a6740 10 client.17586 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:45.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5653540bf350 con 0x5653540a5620 2024-04-23T18:46:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.563+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4036070 con 0x5653540a5620 2024-04-23T18:46:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f6eec0a50a0 con 0x5653540a5620 2024-04-23T18:46:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 10 client.17586 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 20 client.17586 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653540c07c0 0x5653540c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:45.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 4 client.17586 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:46:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565354018390 con 0x5653540c07c0 2024-04-23T18:46:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653540c07c0 0x5653540c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653540c07c0 0x5653540c2bb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a355e6762bc5b4d7 server_cookie=a59b6cc36797f06d in_seq=0 out_seq=0 2024-04-23T18:46:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.564+0000 7f6edeff5640 10 client.17586 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6edeff5640 1 -- 172.21.15.33:0/2288050582 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565354018390 con 0x5653540c07c0 2024-04-23T18:46:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6edeff5640 10 client.17586 handle_command_reply: tid=0 2024-04-23T18:46:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 client.17586 shutdown 2024-04-23T18:46:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 msgr2=0x5653540b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6eec098d20 tx=0x7f6eec09c2e0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 shutdown_connections 2024-04-23T18:46:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5653540c07c0 0x5653540c2bb0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x5653540adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x5653540ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6edeff5640 0 client.17586 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 --2- 172.21.15.33:0/2288050582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540b0210 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 >> 172.21.15.33:0/2288050582 conn(0x5653540a1b30 msgr2=0x5653540a7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:47.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 shutdown_connections 2024-04-23T18:46:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 1 -- 172.21.15.33:0/2288050582 wait complete. 2024-04-23T18:46:47.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.565+0000 7f6f045a6740 20 client.17586 trim_cache size 0 max 16384 2024-04-23T18:46:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.580+0000 7f6f045a6740 1 Processor -- start 2024-04-23T18:46:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.580+0000 7f6f045a6740 1 -- start start 2024-04-23T18:46:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540a5f40 0x5653540b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540b0800 0x5653540afd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1af4c0 con 0x5653540a5620 2024-04-23T18:46:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ad2b0 con 0x5653540a5f40 2024-04-23T18:46:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6f045a6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1aedf0 con 0x5653540b0800 2024-04-23T18:46:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38306/0 (socket says 172.21.15.33:38306) 2024-04-23T18:46:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 -- 172.21.15.33:0/4078038705 learned_addr learned my addr 172.21.15.33:0/4078038705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540a5f40 0x5653540b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540b0800 0x5653540afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 -- 172.21.15.33:0/4078038705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540b0800 msgr2=0x5653540afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540b0800 0x5653540afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 -- 172.21.15.33:0/4078038705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540a5f40 msgr2=0x5653540b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540a5f40 0x5653540b8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 -- 172.21.15.33:0/4078038705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653540a65b0 con 0x5653540a5620 2024-04-23T18:46:48.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6efaffd640 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6eec09c850 tx=0x7f6eec09c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2cfdef412aa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.581+0000 7f6ee0ff9640 1 -- 172.21.15.33:0/4078038705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec098b60 con 0x5653540a5620 2024-04-23T18:46:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6ee0ff9640 1 -- 172.21.15.33:0/4078038705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec0ce490 con 0x5653540a5620 2024-04-23T18:46:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6ee0ff9640 1 -- 172.21.15.33:0/4078038705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec098870 con 0x5653540a5620 2024-04-23T18:46:48.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- 172.21.15.33:0/4078038705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 msgr2=0x5653540bc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6eec09c850 tx=0x7f6eec09c7c0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- 172.21.15.33:0/4078038705 shutdown_connections 2024-04-23T18:46:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540b0800 0x5653540afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540a5f40 0x5653540b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 --2- 172.21.15.33:0/4078038705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x5653540bc170 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- 172.21.15.33:0/4078038705 >> 172.21.15.33:0/4078038705 conn(0x5653540a1b30 msgr2=0x5653540a0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- 172.21.15.33:0/4078038705 shutdown_connections 2024-04-23T18:46:48.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- 172.21.15.33:0/4078038705 wait complete. 2024-04-23T18:46:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.582+0000 7f6f045a6740 1 -- start start 2024-04-23T18:46:48.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 Processor -- start 2024-04-23T18:46:49.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x56535411f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x56535411f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:49.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ae050 con 0x5653540a5620 2024-04-23T18:46:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ad980 con 0x5653540b0800 2024-04-23T18:46:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ae720 con 0x5653540a5f40 2024-04-23T18:46:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6f045a6740 10 client.0 _ll_register_callbacks cb 0x7f6eec09c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:49.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38312/0 (socket says 172.21.15.33:38312) 2024-04-23T18:46:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 -- 172.21.15.33:0/652153241 learned_addr learned my addr 172.21.15.33:0/652153241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:49.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x56535411f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x56535411f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:49.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 -- 172.21.15.33:0/652153241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 msgr2=0x56535411f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x56535411f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 -- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 msgr2=0x56535411f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:49.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x56535411f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 -- 172.21.15.33:0/652153241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565354129f00 con 0x5653540a5620 2024-04-23T18:46:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efb7fe640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x56535411f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:46:50.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6efaffd640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6eec099290 tx=0x7f6eec0991e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1e417be644d525 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:50.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6edeff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.583+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec09bd10 con 0x5653540a5620 2024-04-23T18:46:50.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f6eec09beb0 con 0x5653540a5620 2024-04-23T18:46:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eec0ce490 con 0x5653540a5620 2024-04-23T18:46:50.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6eec0672b0 con 0x5653540a5620 2024-04-23T18:46:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 10 client.17592 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56535412b1f0 con 0x5653540a5620 2024-04-23T18:46:50.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec0ce490 con 0x5653540a5620 2024-04-23T18:46:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f6eec0a1f00 con 0x5653540a5620 2024-04-23T18:46:50.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 10 client.17592 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 10 client.17592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:50.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 20 client.17592 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535412b5d0 0x56535412d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 4 client.17592 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:46:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565354012200 con 0x56535412b5d0 2024-04-23T18:46:50.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.584+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535412b5d0 0x56535412d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.585+0000 7f6edeff5640 10 client.17592 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.585+0000 7f6efa7fc640 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535412b5d0 0x56535412d9c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa58f25041b7b70c server_cookie=6d381e1949a13680 in_seq=0 out_seq=0 2024-04-23T18:46:51.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6edeff5640 1 -- 172.21.15.33:0/652153241 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565354012200 con 0x56535412b5d0 2024-04-23T18:46:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6edeff5640 10 client.17592 handle_command_reply: tid=0 2024-04-23T18:46:51.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 client.17592 shutdown 2024-04-23T18:46:51.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 msgr2=0x56535411ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6eec099290 tx=0x7f6eec0991e0 comp rx=0 tx=0).stop 2024-04-23T18:46:51.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 shutdown_connections 2024-04-23T18:46:51.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535412b5d0 0x56535412d9c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5653540b0800 0x56535411f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653540a5f40 0x56535411f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:51.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 --2- 172.21.15.33:0/652153241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653540a5620 0x56535411ecb0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6edeff5640 0 client.17592 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:51.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 >> 172.21.15.33:0/652153241 conn(0x5653540a1b30 msgr2=0x5653540bb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:51.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 shutdown_connections 2024-04-23T18:46:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 1 -- 172.21.15.33:0/652153241 wait complete. 2024-04-23T18:46:51.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.589+0000 7f6f045a6740 20 client.17592 trim_cache size 0 max 16384 2024-04-23T18:46:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:46:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:46:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:46:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:46:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 msgr2=0x7f6ee40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 0x7f6ee40a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ef404e000 tx=0x7f6ef404c000 comp rx=0 tx=0).stop 2024-04-23T18:46:52.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 msgr2=0x7f6efc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c920 tx=0x7f6eec00cde0 comp rx=0 tx=0).stop 2024-04-23T18:46:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 shutdown_connections 2024-04-23T18:46:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6ee40a3dc0 0x7f6ee40a6280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6efc19cf50 0x7f6efc19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6efc106170 0x7f6efc198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:52.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 --2- 172.21.15.33:0/3798352508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6efc1010f0 0x7f6efc198690 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 >> 172.21.15.33:0/3798352508 conn(0x7f6efc077fd0 msgr2=0x7f6efc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 shutdown_connections 2024-04-23T18:46:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:42.590+0000 7f6f017ad640 1 -- 172.21.15.33:0/3798352508 wait complete. 2024-04-23T18:46:52.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:43 smithi033 ceph-mon[29274]: pgmap v580: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:52.802 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:45 smithi033 ceph-mon[29274]: pgmap v581: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:52.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:47 smithi033 ceph-mon[29274]: pgmap v582: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:52.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:49 smithi033 ceph-mon[29274]: pgmap v583: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:52.902 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:51 smithi033 ceph-mon[29274]: pgmap v584: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:52.978 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:53 smithi033 ceph-mon[29274]: pgmap v585: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:46:53.028 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:46:53.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:46:53.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:46:53.186 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:46:53.248 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:55 smithi033 ceph-mon[29274]: pgmap v586: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:46:53.289 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:41 smithi122 ceph-mon[32546]: pgmap v644: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.319 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:43 smithi122 ceph-mon[32546]: pgmap v645: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.352 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:45 smithi122 ceph-mon[32546]: pgmap v646: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:53.386 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:47 smithi122 ceph-mon[32546]: pgmap v647: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:49 smithi122 ceph-mon[32546]: pgmap v648: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.478 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:51 smithi122 ceph-mon[32546]: pgmap v649: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.511 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:41 smithi106 ceph-mon[32716]: pgmap v644: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.544 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:43 smithi106 ceph-mon[32716]: pgmap v645: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.586 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:45 smithi106 ceph-mon[32716]: pgmap v646: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:46:53.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:47 smithi106 ceph-mon[32716]: pgmap v647: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.653 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:49 smithi106 ceph-mon[32716]: pgmap v648: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.720 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:51 smithi106 ceph-mon[32716]: pgmap v649: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:46:53.754 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:53 smithi122 ceph-mon[32546]: pgmap v650: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.895 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:53 smithi106 ceph-mon[32716]: pgmap v650: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:46:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3484135185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b24102b80 msgr2=0x7f0b24106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3484135185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b24102b80 0x7f0b24106bc0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0b10009900 tx=0x7f0b1002a410 comp rx=0 tx=0).stop 2024-04-23T18:46:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3484135185 shutdown_connections 2024-04-23T18:46:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3484135185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241072a0 0x7f0b2410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:54.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3484135185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b24102b80 0x7f0b24106bc0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3484135185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b241021d0 0x7f0b241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:54.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3484135185 >> 172.21.15.33:0/3484135185 conn(0x7f0b240793e0 msgr2=0x7f0b240797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:54.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3484135185 shutdown_connections 2024-04-23T18:46:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3484135185 wait complete. 2024-04-23T18:46:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 Processor -- start 2024-04-23T18:46:54.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.690+0000 7f0b2aff7640 1 -- start start 2024-04-23T18:46:54.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241021d0 0x7f0b24198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:54.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 0x7f0b2419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241121e0 con 0x7f0b241072a0 2024-04-23T18:46:54.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24112060 con 0x7f0b2419cf40 2024-04-23T18:46:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b2aff7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b24112360 con 0x7f0b241021d0 2024-04-23T18:46:54.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:54.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43512/0 (socket says 172.21.15.33:43512) 2024-04-23T18:46:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 -- 172.21.15.33:0/3171669018 learned_addr learned my addr 172.21.15.33:0/3171669018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b28d93640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241021d0 0x7f0b24198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b29594640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 0x7f0b2419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 -- 172.21.15.33:0/3171669018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241021d0 msgr2=0x7f0b24198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241021d0 0x7f0b24198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:54.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 -- 172.21.15.33:0/3171669018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 msgr2=0x7f0b2419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 0x7f0b2419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 -- 172.21.15.33:0/3171669018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2419da30 con 0x7f0b241072a0 2024-04-23T18:46:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b29594640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 0x7f0b2419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:55.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.691+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0b10032fd0 tx=0x7f0b10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.692+0000 7f0b21ffb640 1 -- 172.21.15.33:0/3171669018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1003f030 con 0x7f0b241072a0 2024-04-23T18:46:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.692+0000 7f0b21ffb640 1 -- 172.21.15.33:0/3171669018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b10004490 con 0x7f0b241072a0 2024-04-23T18:46:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.692+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2419dcc0 con 0x7f0b241072a0 2024-04-23T18:46:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.692+0000 7f0b21ffb640 1 -- 172.21.15.33:0/3171669018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10035070 con 0x7f0b241072a0 2024-04-23T18:46:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.692+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b241a5480 con 0x7f0b241072a0 2024-04-23T18:46:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.694+0000 7f0b21ffb640 1 -- 172.21.15.33:0/3171669018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0b10042030 con 0x7f0b241072a0 2024-04-23T18:46:55.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 Processor -- start 2024-04-23T18:46:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 -- start start 2024-04-23T18:46:55.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37685a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec376902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ec375ea970 con 0x55ec37690800 2024-04-23T18:46:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ec375874a0 con 0x55ec37685f40 2024-04-23T18:46:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec375e6930 con 0x55ec37685620 2024-04-23T18:46:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b21ffb640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 0x7f0b140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:55.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43522/0 (socket says 172.21.15.33:43522) 2024-04-23T18:46:55.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b23fff640 1 -- 172.21.15.33:0/3814723484 learned_addr learned my addr 172.21.15.33:0/3814723484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:55.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b28d93640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec376902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.695+0000 7f0b29594640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37685a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:56.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b29594640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 0x7f0b140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b21ffb640 1 -- 172.21.15.33:0/3171669018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f0b100f0cc0 con 0x7f0b241072a0 2024-04-23T18:46:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 -- 172.21.15.33:0/3814723484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 msgr2=0x55ec37685a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37685a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 -- 172.21.15.33:0/3814723484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 msgr2=0x55ec376902c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec376902c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 -- 172.21.15.33:0/3814723484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec372fd210 con 0x55ec37690800 2024-04-23T18:46:56.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b28d93640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec376902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:46:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b23fff640 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0b100b9620 tx=0x7f0b100a7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3c6e6581f84397 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:56.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b29594640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 0x7f0b140a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009fd0 tx=0x7f0b1c009280 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b0effd640 1 -- 172.21.15.33:0/3814723484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10049070 con 0x55ec37690800 2024-04-23T18:46:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b0effd640 1 -- 172.21.15.33:0/3814723484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100bcc50 con 0x55ec37690800 2024-04-23T18:46:56.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.696+0000 7f0b0effd640 1 -- 172.21.15.33:0/3814723484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100ee680 con 0x55ec37690800 2024-04-23T18:46:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- 172.21.15.33:0/3814723484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 msgr2=0x55ec37692bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:56.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f0b100b9620 tx=0x7f0b100a7780 comp rx=0 tx=0).stop 2024-04-23T18:46:56.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- 172.21.15.33:0/3814723484 shutdown_connections 2024-04-23T18:46:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37692bf0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:57.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec376902c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/3814723484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37685a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- 172.21.15.33:0/3814723484 >> 172.21.15.33:0/3814723484 conn(0x55ec37681b30 msgr2=0x55ec37681f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- 172.21.15.33:0/3814723484 shutdown_connections 2024-04-23T18:46:57.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- 172.21.15.33:0/3814723484 wait complete. 2024-04-23T18:46:57.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 -- start start 2024-04-23T18:46:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.697+0000 7f0b2dde5740 1 Processor -- start 2024-04-23T18:46:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685620 0x55ec376900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:57.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 0x55ec3768d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ec376931f0 con 0x55ec37685f40 2024-04-23T18:46:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ec375850b0 con 0x55ec37685620 2024-04-23T18:46:57.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec37693370 con 0x55ec37690800 2024-04-23T18:46:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b2dde5740 10 client.0 _ll_register_callbacks cb 0x55ec37572d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:46:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43526/0 (socket says 172.21.15.33:43526) 2024-04-23T18:46:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 -- 172.21.15.33:0/1154765345 learned_addr learned my addr 172.21.15.33:0/1154765345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:46:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b28d93640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685620 0x55ec376900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b29594640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 0x55ec3768d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 -- 172.21.15.33:0/1154765345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 msgr2=0x55ec3768d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:57.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 0x55ec3768d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:57.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 -- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685620 msgr2=0x55ec376900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:46:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685620 0x55ec376900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 -- 172.21.15.33:0/1154765345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec373c9620 con 0x55ec37685f40 2024-04-23T18:46:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b29594640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 0x55ec3768d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:46:58.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.698+0000 7f0b23fff640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0b100bc950 tx=0x7f0b10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a977269ef53741fb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:46:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:46:58.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10049040 con 0x55ec37685f40 2024-04-23T18:46:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f0b1003bc20 con 0x55ec37685f40 2024-04-23T18:46:58.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b100bcc50 con 0x55ec37685f40 2024-04-23T18:46:58.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b100ee890 con 0x55ec37685f40 2024-04-23T18:46:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 10 client.17604 fetch_fsmap learned FSMap version 39 2024-04-23T18:46:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec376931f0 con 0x55ec37685f40 2024-04-23T18:46:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100bcc50 con 0x55ec37685f40 2024-04-23T18:46:58.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f0b100dd050 con 0x55ec37685f40 2024-04-23T18:46:58.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 10 client.17604 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:46:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 10 client.17604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:46:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 20 client.17604 populate_metadata read hostname 'smithi033' 2024-04-23T18:46:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec376a0200 0x55ec376a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 4 client.17604 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:46:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.699+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec375f8390 con 0x55ec376a0200 2024-04-23T18:46:58.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.700+0000 7f0b28d93640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec376a0200 0x55ec376a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:46:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.700+0000 7f0b03fff640 10 client.17604 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:58.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.700+0000 7f0b28d93640 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec376a0200 0x55ec376a25f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6299f7a54a3cb214 server_cookie=1f78ae85da3166ab in_seq=0 out_seq=0 2024-04-23T18:46:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b03fff640 1 -- 172.21.15.33:0/1154765345 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec375f8390 con 0x55ec376a0200 2024-04-23T18:46:58.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b03fff640 10 client.17604 handle_command_reply: tid=0 2024-04-23T18:46:58.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 client.17604 shutdown 2024-04-23T18:46:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 msgr2=0x55ec3768d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:46:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f0b100bc950 tx=0x7f0b10000bc0 comp rx=0 tx=0).stop 2024-04-23T18:46:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 shutdown_connections 2024-04-23T18:46:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec376a0200 0x55ec376a25f0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37690800 0x55ec3768d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:59.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768d3c0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:59.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b03fff640 0 client.17604 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:46:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/1154765345 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685620 0x55ec376900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:46:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 >> 172.21.15.33:0/1154765345 conn(0x55ec37681b30 msgr2=0x55ec37687860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:46:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 shutdown_connections 2024-04-23T18:46:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 1 -- 172.21.15.33:0/1154765345 wait complete. 2024-04-23T18:46:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.701+0000 7f0b2dde5740 20 client.17604 trim_cache size 0 max 16384 2024-04-23T18:46:59.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.716+0000 7f0b2dde5740 1 Processor -- start 2024-04-23T18:46:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 -- start start 2024-04-23T18:46:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37688af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:59.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 0x55ec37680b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:46:59.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241af5e0 con 0x55ec37685f40 2024-04-23T18:46:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241ad3d0 con 0x55ec37690800 2024-04-23T18:47:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241aef10 con 0x55ec37685620 2024-04-23T18:47:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43540/0 (socket says 172.21.15.33:43540) 2024-04-23T18:47:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 -- 172.21.15.33:0/74628869 learned_addr learned my addr 172.21.15.33:0/74628869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b28d93640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37688af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 -- 172.21.15.33:0/74628869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 msgr2=0x55ec37688af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37688af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 -- 172.21.15.33:0/74628869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 msgr2=0x55ec37680b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b29594640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 0x55ec37680b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 0x55ec37680b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 -- 172.21.15.33:0/74628869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec376a2960 con 0x55ec37685f40 2024-04-23T18:47:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b29594640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 0x55ec37680b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:47:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b23fff640 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005c40 tx=0x7f0b10003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cea2107ea956618 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.717+0000 7f0b0e7fc640 1 -- 172.21.15.33:0/74628869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10049070 con 0x55ec37685f40 2024-04-23T18:47:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b0e7fc640 1 -- 172.21.15.33:0/74628869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100c1000 con 0x55ec37685f40 2024-04-23T18:47:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b0e7fc640 1 -- 172.21.15.33:0/74628869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100c5480 con 0x55ec37685f40 2024-04-23T18:47:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- 172.21.15.33:0/74628869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 msgr2=0x55ec3768fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f0b10005c40 tx=0x7f0b10003f90 comp rx=0 tx=0).stop 2024-04-23T18:47:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- 172.21.15.33:0/74628869 shutdown_connections 2024-04-23T18:47:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37690800 0x55ec37680b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:01.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37685f40 0x55ec3768fc00 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:01.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/74628869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37688af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:01.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- 172.21.15.33:0/74628869 >> 172.21.15.33:0/74628869 conn(0x55ec37681b30 msgr2=0x55ec37680420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- 172.21.15.33:0/74628869 shutdown_connections 2024-04-23T18:47:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- 172.21.15.33:0/74628869 wait complete. 2024-04-23T18:47:01.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.718+0000 7f0b2dde5740 1 -- start start 2024-04-23T18:47:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 Processor -- start 2024-04-23T18:47:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37698590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec37693af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:01.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241ae170 con 0x55ec37690800 2024-04-23T18:47:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241adaa0 con 0x55ec37685f40 2024-04-23T18:47:01.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b241ae840 con 0x55ec37685620 2024-04-23T18:47:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b2dde5740 10 client.0 _ll_register_callbacks cb 0x7f0b2406b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43548/0 (socket says 172.21.15.33:43548) 2024-04-23T18:47:01.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 -- 172.21.15.33:0/4100799032 learned_addr learned my addr 172.21.15.33:0/4100799032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b28d93640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37698590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b23fff640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec37693af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 -- 172.21.15.33:0/4100799032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 msgr2=0x55ec37698590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:01.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37698590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:01.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 -- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 msgr2=0x55ec37693af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:01.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.719+0000 7f0b29594640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec37693af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b29594640 1 -- 172.21.15.33:0/4100799032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec37694570 con 0x55ec37690800 2024-04-23T18:47:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b23fff640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec37693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:01.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b29594640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c005fd0 tx=0x7f0b1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbd563907de1d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:02.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c022d70 con 0x55ec37690800 2024-04-23T18:47:02.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f0b1c004b20 con 0x55ec37690800 2024-04-23T18:47:02.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b100c1000 con 0x55ec37690800 2024-04-23T18:47:02.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c004d20 con 0x55ec37690800 2024-04-23T18:47:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 10 client.17610 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec37706330 con 0x55ec37690800 2024-04-23T18:47:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b100c1000 con 0x55ec37690800 2024-04-23T18:47:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f0b1c029030 con 0x55ec37690800 2024-04-23T18:47:02.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 10 client.17610 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:02.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 20 client.17610 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec37706b00 0x55ec37708f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 4 client.17610 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.720+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec375f2200 con 0x55ec37706b00 2024-04-23T18:47:02.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.721+0000 7f0b28d93640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec37706b00 0x55ec37708f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:02.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.721+0000 7f0b28d93640 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec37706b00 0x55ec37708f20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3dfd392081bbf3 server_cookie=c7b79f04b175dd4 in_seq=0 out_seq=0 2024-04-23T18:47:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.721+0000 7f0b03fff640 10 client.17610 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:02.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.725+0000 7f0b03fff640 1 -- 172.21.15.33:0/4100799032 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec375f2200 con 0x55ec37706b00 2024-04-23T18:47:02.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.725+0000 7f0b03fff640 10 client.17610 handle_command_reply: tid=0 2024-04-23T18:47:02.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.725+0000 7f0b2dde5740 1 client.17610 shutdown 2024-04-23T18:47:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.725+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 msgr2=0x55ec37694030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.725+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c005fd0 tx=0x7f0b1c004a40 comp rx=0 tx=0).stop 2024-04-23T18:47:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 shutdown_connections 2024-04-23T18:47:02.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ec37706b00 0x55ec37708f20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:02.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ec37690800 0x55ec37694030 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:02.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ec37685f40 0x55ec37693af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b03fff640 0 client.17610 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:03.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 --2- 172.21.15.33:0/4100799032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec37685620 0x55ec37698590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:03.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 >> 172.21.15.33:0/4100799032 conn(0x55ec37681b30 msgr2=0x55ec3768cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 shutdown_connections 2024-04-23T18:47:03.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 1 -- 172.21.15.33:0/4100799032 wait complete. 2024-04-23T18:47:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2dde5740 20 client.17610 trim_cache size 0 max 16384 2024-04-23T18:47:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:03.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 msgr2=0x7f0b140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 0x7f0b140a6280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c009fd0 tx=0x7f0b1c009280 comp rx=0 tx=0).stop 2024-04-23T18:47:03.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 msgr2=0x7f0b24198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0b10032fd0 tx=0x7f0b10004270 comp rx=0 tx=0).stop 2024-04-23T18:47:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.726+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 shutdown_connections 2024-04-23T18:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0b140a3dc0 0x7f0b140a6280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b2419cf40 0x7f0b2419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:03.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b241072a0 0x7f0b24198bc0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 --2- 172.21.15.33:0/3171669018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b241021d0 0x7f0b24198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:03.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 >> 172.21.15.33:0/3171669018 conn(0x7f0b240793e0 msgr2=0x7f0b240ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:03.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 shutdown_connections 2024-04-23T18:47:03.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:44:55.727+0000 7f0b2aff7640 1 -- 172.21.15.33:0/3171669018 wait complete. 2024-04-23T18:47:03.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:57 smithi033 ceph-mon[29274]: pgmap v587: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:03.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:44:59 smithi033 ceph-mon[29274]: pgmap v588: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:03.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:01 smithi033 ceph-mon[29274]: pgmap v589: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:04.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:03 smithi033 ceph-mon[29274]: pgmap v590: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:04.119 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:05 smithi033 ceph-mon[29274]: pgmap v591: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:04.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:07 smithi033 ceph-mon[29274]: pgmap v592: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-23T18:47:04.137 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:54 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:47:04.144 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:55 smithi122 ceph-mon[32546]: pgmap v651: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:04.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.245 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.253 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: pgmap v652: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:04.401 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.409 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:04.417 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.425 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.502 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:47:04.589 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:47:04.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.607 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.616 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.624 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:47:04.742 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:47:04.759 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.768 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.836 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:04.860 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:47:04.926 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:47:05.030 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:56 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.073 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:57 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:47:05.102 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:57 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:47:05.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:57 smithi122 ceph-mon[32546]: pgmap v653: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:05.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:57 smithi122 ceph-mon[32546]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:47:05.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:46:59 smithi122 ceph-mon[32546]: pgmap v654: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:05.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:01 smithi122 ceph-mon[32546]: pgmap v655: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:05.226 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:03 smithi122 ceph-mon[32546]: pgmap v656: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:05.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:54 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:47:05.354 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:55 smithi106 ceph-mon[32716]: pgmap v651: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:05.438 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.455 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.593 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: pgmap v652: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:05.644 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.653 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:05.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:05.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:05.828 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:05.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:05.907 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:47:06.008 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:47:06.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.085 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.093 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.102 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.111 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:47:06.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:47:06.187 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.195 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.329 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.341 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:47:06.405 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:47:06.413 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:47:06.439 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:56 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:06.531 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:57 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:47:06.582 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:57 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:47:06.591 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:57 smithi106 ceph-mon[32716]: pgmap v653: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:06.599 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:57 smithi106 ceph-mon[32716]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:47:06.608 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:46:59 smithi106 ceph-mon[32716]: pgmap v654: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:06.616 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:01 smithi106 ceph-mon[32716]: pgmap v655: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:47:06.664 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:03 smithi106 ceph-mon[32716]: pgmap v656: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:06.673 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:05 smithi122 ceph-mon[32546]: pgmap v657: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:06.681 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:05 smithi106 ceph-mon[32716]: pgmap v657: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 -- 172.21.15.33:0/1340338064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc107290 msgr2=0x7f11bc10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:06.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 --2- 172.21.15.33:0/1340338064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc107290 0x7f11bc10ee60 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:47:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 -- 172.21.15.33:0/1340338064 shutdown_connections 2024-04-23T18:47:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 --2- 172.21.15.33:0/1340338064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc107290 0x7f11bc10ee60 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 --2- 172.21.15.33:0/1340338064 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc102b70 0x7f11bc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 --2- 172.21.15.33:0/1340338064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 -- 172.21.15.33:0/1340338064 >> 172.21.15.33:0/1340338064 conn(0x7f11bc0fda20 msgr2=0x7f11bc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 -- 172.21.15.33:0/1340338064 shutdown_connections 2024-04-23T18:47:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.826+0000 7f11c1336640 1 -- 172.21.15.33:0/1340338064 wait complete. 2024-04-23T18:47:07.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 Processor -- start 2024-04-23T18:47:07.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 -- start start 2024-04-23T18:47:07.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc107290 0x7f11bc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc112200 con 0x7f11bc19cf10 2024-04-23T18:47:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc112080 con 0x7f11bc107290 2024-04-23T18:47:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.827+0000 7f11c1336640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc112380 con 0x7f11bc1021c0 2024-04-23T18:47:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34976/0 (socket says 172.21.15.33:34976) 2024-04-23T18:47:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11bb7fe640 1 -- 172.21.15.33:0/2454431747 learned_addr learned my addr 172.21.15.33:0/2454431747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:07.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:07.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc107290 0x7f11bc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 -- 172.21.15.33:0/2454431747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc107290 msgr2=0x7f11bc198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc107290 0x7f11bc198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 -- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 msgr2=0x7f11bc19d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 -- 172.21.15.33:0/2454431747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19da00 con 0x7f11bc1021c0 2024-04-23T18:47:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:47:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1986a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11a3fff640 1 -- 172.21.15.33:0/2454431747 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00ccc0 con 0x7f11bc1021c0 2024-04-23T18:47:08.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19dcf0 con 0x7f11bc1021c0 2024-04-23T18:47:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.828+0000 7f11a3fff640 1 -- 172.21.15.33:0/2454431747 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ac005020 con 0x7f11bc1021c0 2024-04-23T18:47:08.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.829+0000 7f11a3fff640 1 -- 172.21.15.33:0/2454431747 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac004a00 con 0x7f11bc1021c0 2024-04-23T18:47:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.829+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a54e0 con 0x7f11bc1021c0 2024-04-23T18:47:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 Processor -- start 2024-04-23T18:47:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 -- start start 2024-04-23T18:47:08.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11a3fff640 1 -- 172.21.15.33:0/2454431747 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11ac004ba0 con 0x7f11bc1021c0 2024-04-23T18:47:08.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:08.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6f40 0x562782cf12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782cf1800 0x562782cf3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562782c4b970 con 0x562782ce6620 2024-04-23T18:47:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562782be84a0 con 0x562782ce6f40 2024-04-23T18:47:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11c4055740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562782c47930 con 0x562782cf1800 2024-04-23T18:47:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:09.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34986/0 (socket says 172.21.15.33:34986) 2024-04-23T18:47:09.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.831+0000 7f11bb7fe640 1 -- 172.21.15.33:0/1485012208 learned_addr learned my addr 172.21.15.33:0/1485012208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:09.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782cf1800 0x562782cf3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11baffd640 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6f40 0x562782cf12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 -- 172.21.15.33:0/1485012208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782cf1800 msgr2=0x562782cf3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782cf1800 0x562782cf3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 -- 172.21.15.33:0/1485012208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6f40 msgr2=0x562782cf12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:09.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6f40 0x562782cf12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 -- 172.21.15.33:0/1485012208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278295e210 con 0x562782ce6620 2024-04-23T18:47:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11a3fff640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 0x7f11a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005f80 tx=0x7f11b402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291d3e8e59776c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:09.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 0x7f11a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:09.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11a3fff640 1 -- 172.21.15.33:0/2454431747 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f11ac0d07b0 con 0x7f11bc1021c0 2024-04-23T18:47:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11a17fa640 1 -- 172.21.15.33:0/1485012208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4031450 con 0x562782ce6620 2024-04-23T18:47:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11a17fa640 1 -- 172.21.15.33:0/1485012208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4036040 con 0x562782ce6620 2024-04-23T18:47:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.832+0000 7f11a17fa640 1 -- 172.21.15.33:0/1485012208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b40044e0 con 0x562782ce6620 2024-04-23T18:47:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- 172.21.15.33:0/1485012208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 msgr2=0x562782ce6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11baffd640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 0x7f11a40a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cdfc0 tx=0x7f11ac009f60 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:09.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005f80 tx=0x7f11b402f310 comp rx=0 tx=0).stop 2024-04-23T18:47:09.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- 172.21.15.33:0/1485012208 shutdown_connections 2024-04-23T18:47:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782cf1800 0x562782cf3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6f40 0x562782cf12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 --2- 172.21.15.33:0/1485012208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782ce6a00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- 172.21.15.33:0/1485012208 >> 172.21.15.33:0/1485012208 conn(0x562782ce2b30 msgr2=0x562782ce2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- 172.21.15.33:0/1485012208 shutdown_connections 2024-04-23T18:47:10.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- 172.21.15.33:0/1485012208 wait complete. 2024-04-23T18:47:10.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 -- start start 2024-04-23T18:47:10.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.833+0000 7f11c4055740 1 Processor -- start 2024-04-23T18:47:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cee3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:10.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782cee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:10.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562782cf4250 con 0x562782ce6620 2024-04-23T18:47:10.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562782be60b0 con 0x562782cf1800 2024-04-23T18:47:10.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562782cf43d0 con 0x562782ce6f40 2024-04-23T18:47:10.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 10 client.0 _ll_register_callbacks cb 0x562782bd3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:10.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:10.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34988/0 (socket says 172.21.15.33:34988) 2024-04-23T18:47:10.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 -- 172.21.15.33:0/2815186113 learned_addr learned my addr 172.21.15.33:0/2815186113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cee3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:10.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782cee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 -- 172.21.15.33:0/2815186113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 msgr2=0x562782cee3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:10.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cee3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:10.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 -- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 msgr2=0x562782cee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:10.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782cee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 -- 172.21.15.33:0/2815186113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562782a2a620 con 0x562782ce6620 2024-04-23T18:47:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:10.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f11b402a900 tx=0x7f11b40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a793f09524f890f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:10.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f119eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4058060 con 0x562782ce6620 2024-04-23T18:47:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f11b40041e0 con 0x562782ce6620 2024-04-23T18:47:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.834+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b4036040 con 0x562782ce6620 2024-04-23T18:47:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f11b40043e0 con 0x562782ce6620 2024-04-23T18:47:11.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 10 client.17622 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:11.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562782cf4250 con 0x562782ce6620 2024-04-23T18:47:11.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4036040 con 0x562782ce6620 2024-04-23T18:47:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f11b4044070 con 0x562782ce6620 2024-04-23T18:47:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 10 client.17622 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 20 client.17622 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d01260 0x562782d03650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:11.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 4 client.17622 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:47:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562782c59390 con 0x562782d01260 2024-04-23T18:47:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.835+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d01260 0x562782d03650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.836+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d01260 0x562782d03650 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b622a28c5f39901 server_cookie=eef1f7ace3b2e63e in_seq=0 out_seq=0 2024-04-23T18:47:11.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.836+0000 7f119eff5640 10 client.17622 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f119eff5640 1 -- 172.21.15.33:0/2815186113 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562782c59390 con 0x562782d01260 2024-04-23T18:47:11.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f119eff5640 10 client.17622 handle_command_reply: tid=0 2024-04-23T18:47:11.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 client.17622 shutdown 2024-04-23T18:47:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 msgr2=0x562782cf10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:11.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f11b402a900 tx=0x7f11b40095c0 comp rx=0 tx=0).stop 2024-04-23T18:47:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 shutdown_connections 2024-04-23T18:47:11.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d01260 0x562782d03650 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:11.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782cee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:12.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cee3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f119eff5640 0 client.17622 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:12.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 --2- 172.21.15.33:0/2815186113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cf10c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:12.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 >> 172.21.15.33:0/2815186113 conn(0x562782ce2b30 msgr2=0x562782ce88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:12.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 shutdown_connections 2024-04-23T18:47:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 1 -- 172.21.15.33:0/2815186113 wait complete. 2024-04-23T18:47:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.837+0000 7f11c4055740 20 client.17622 trim_cache size 0 max 16384 2024-04-23T18:47:12.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 Processor -- start 2024-04-23T18:47:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 -- start start 2024-04-23T18:47:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:12.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782d08ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1af550 con 0x562782ce6620 2024-04-23T18:47:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ad340 con 0x562782cf1800 2024-04-23T18:47:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11c4055740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1aee80 con 0x562782ce6f40 2024-04-23T18:47:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34998/0 (socket says 172.21.15.33:34998) 2024-04-23T18:47:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3491402839 learned_addr learned my addr 172.21.15.33:0/3491402839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.852+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:12.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3491402839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 msgr2=0x562782cf0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3491402839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 msgr2=0x562782d08ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782d08ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3491402839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562782ce2960 con 0x562782ce6620 2024-04-23T18:47:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f11b4033020 tx=0x7f11b402f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fe45703109acaa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11a0ff9640 1 -- 172.21.15.33:0/3491402839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4058070 con 0x562782ce6620 2024-04-23T18:47:13.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11a0ff9640 1 -- 172.21.15.33:0/3491402839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4040020 con 0x562782ce6620 2024-04-23T18:47:13.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11a0ff9640 1 -- 172.21.15.33:0/3491402839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004230 con 0x562782ce6620 2024-04-23T18:47:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 -- 172.21.15.33:0/3491402839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 msgr2=0x562782cfd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:13.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f11b4033020 tx=0x7f11b402f310 comp rx=0 tx=0).stop 2024-04-23T18:47:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 -- 172.21.15.33:0/3491402839 shutdown_connections 2024-04-23T18:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782cf1800 0x562782d08ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:13.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 --2- 172.21.15.33:0/3491402839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782ce6620 0x562782cfd110 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.853+0000 7f11c4055740 1 -- 172.21.15.33:0/3491402839 >> 172.21.15.33:0/3491402839 conn(0x562782ce2b30 msgr2=0x562782ce8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- 172.21.15.33:0/3491402839 shutdown_connections 2024-04-23T18:47:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- 172.21.15.33:0/3491402839 wait complete. 2024-04-23T18:47:13.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- start start 2024-04-23T18:47:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 Processor -- start 2024-04-23T18:47:13.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6620 0x562782cf4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:13.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:13.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ae0e0 con 0x562782cf1800 2024-04-23T18:47:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ada10 con 0x562782ce6620 2024-04-23T18:47:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ae7b0 con 0x562782ce6f40 2024-04-23T18:47:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11c4055740 10 client.0 _ll_register_callbacks cb 0x7f11b40098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:13.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:13.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35014/0 (socket says 172.21.15.33:35014) 2024-04-23T18:47:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3797380969 learned_addr learned my addr 172.21.15.33:0/3797380969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:14.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6620 0x562782cf4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.854+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3797380969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 msgr2=0x562782cf5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6620 msgr2=0x562782cf4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:14.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6620 0x562782cf4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 -- 172.21.15.33:0/3797380969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562782d66740 con 0x562782cf1800 2024-04-23T18:47:14.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11bb7fe640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f11b4056040 tx=0x7f11b40319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280d5d3a6a94ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:14.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4058070 con 0x562782cf1800 2024-04-23T18:47:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8498+0+0 (secure 0 0 0) 0x7f11b40040c0 con 0x562782cf1800 2024-04-23T18:47:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b4040020 con 0x562782cf1800 2024-04-23T18:47:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f11b402f660 con 0x562782cf1800 2024-04-23T18:47:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11c4055740 10 client.17628 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:14.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562782d67950 con 0x562782cf1800 2024-04-23T18:47:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b4040020 con 0x562782cf1800 2024-04-23T18:47:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.855+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f11b4044070 con 0x562782cf1800 2024-04-23T18:47:15.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 10 client.17628 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 20 client.17628 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:15.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d67f50 0x562782d6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 4 client.17628 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562782c53200 con 0x562782d67f50 2024-04-23T18:47:15.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d67f50 0x562782d6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f11ba7fc640 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d67f50 0x562782d6a340 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cefbffb3c8b46ad6 server_cookie=92acbeab4b834212 in_seq=0 out_seq=0 2024-04-23T18:47:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.856+0000 7f119eff5640 10 client.17628 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:15.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f119eff5640 1 -- 172.21.15.33:0/3797380969 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562782c53200 con 0x562782d67f50 2024-04-23T18:47:15.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f119eff5640 10 client.17628 handle_command_reply: tid=0 2024-04-23T18:47:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 client.17628 shutdown 2024-04-23T18:47:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 msgr2=0x562782cf59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f11b4056040 tx=0x7f11b40319c0 comp rx=0 tx=0).stop 2024-04-23T18:47:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 shutdown_connections 2024-04-23T18:47:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562782d67f50 0x562782d6a340 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562782cf1800 0x562782cf59c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562782ce6f40 0x562782cf5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f119eff5640 0 client.17628 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 --2- 172.21.15.33:0/3797380969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562782ce6620 0x562782cf4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 >> 172.21.15.33:0/3797380969 conn(0x562782ce2b30 msgr2=0x562782cedc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:15.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 shutdown_connections 2024-04-23T18:47:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.863+0000 7f11c4055740 1 -- 172.21.15.33:0/3797380969 wait complete. 2024-04-23T18:47:15.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c4055740 20 client.17628 trim_cache size 0 max 16384 2024-04-23T18:47:15.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:16.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:16.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 msgr2=0x7f11a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 0x7f11a40a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cdfc0 tx=0x7f11ac009f60 comp rx=0 tx=0).stop 2024-04-23T18:47:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 msgr2=0x7f11bc1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:16.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1986a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00e9f0 tx=0x7f11ac00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:47:16.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 shutdown_connections 2024-04-23T18:47:16.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f11a40a3dc0 0x7f11a40a6280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:16.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc19cf10 0x7f11bc19d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11bc107290 0x7f11bc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 --2- 172.21.15.33:0/2454431747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc1021c0 0x7f11bc1986a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:16.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 >> 172.21.15.33:0/2454431747 conn(0x7f11bc0fda20 msgr2=0x7f11bc0fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:16.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 shutdown_connections 2024-04-23T18:47:16.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:08.864+0000 7f11c1336640 1 -- 172.21.15.33:0/2454431747 wait complete. 2024-04-23T18:47:16.629 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:09 smithi033 ceph-mon[29274]: pgmap v593: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 46 KiB/s wr, 26 op/s 2024-04-23T18:47:16.639 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:10 smithi033 ceph-mon[29274]: osdmap e120: 12 total, 12 up, 12 in 2024-04-23T18:47:16.729 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:11 smithi033 ceph-mon[29274]: pgmap v595: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:47:16.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:13 smithi033 ceph-mon[29274]: pgmap v596: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:47:16.772 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:15 smithi033 ceph-mon[29274]: pgmap v597: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 56 KiB/s wr, 31 op/s 2024-04-23T18:47:16.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:16 smithi033 ceph-mon[29274]: pgmap v598: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:47:16.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:16 smithi033 ceph-mon[29274]: osdmap e121: 12 total, 12 up, 12 in 2024-04-23T18:47:16.964 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:18 smithi033 ceph-mon[29274]: pgmap v600: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-04-23T18:47:16.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:20 smithi033 ceph-mon[29274]: pgmap v601: 97 pgs: 97 active+clean; 15 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-04-23T18:47:16.979 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:07 smithi122 ceph-mon[32546]: pgmap v658: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:16.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:09 smithi122 ceph-mon[32546]: pgmap v659: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:47:16.995 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:09 smithi122 ceph-mon[32546]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:47:17.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:11 smithi122 ceph-mon[32546]: pgmap v661: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:47:17.126 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:13 smithi122 ceph-mon[32546]: pgmap v662: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:47:17.240 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:15 smithi122 ceph-mon[32546]: pgmap v663: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:47:17.263 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:07 smithi106 ceph-mon[32716]: pgmap v658: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:47:17.277 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:09 smithi106 ceph-mon[32716]: pgmap v659: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:47:17.357 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:09 smithi106 ceph-mon[32716]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:47:17.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:11 smithi106 ceph-mon[32716]: pgmap v661: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:47:17.546 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:13 smithi106 ceph-mon[32716]: pgmap v662: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:47:17.554 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:15 smithi106 ceph-mon[32716]: pgmap v663: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:47:17.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:16 smithi122 ceph-mon[32546]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:47:17.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:16 smithi106 ceph-mon[32716]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:47:17.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- 172.21.15.33:0/3422095015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241072a0 msgr2=0x7fa42410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:17.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 --2- 172.21.15.33:0/3422095015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241072a0 0x7fa42410ee70 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fa41c009ca0 tx=0x7fa41c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:47:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- 172.21.15.33:0/3422095015 shutdown_connections 2024-04-23T18:47:17.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 --2- 172.21.15.33:0/3422095015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241072a0 0x7fa42410ee70 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:17.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 --2- 172.21.15.33:0/3422095015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa424102b80 0x7fa424106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 --2- 172.21.15.33:0/3422095015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241021d0 0x7fa4241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- 172.21.15.33:0/3422095015 >> 172.21.15.33:0/3422095015 conn(0x7fa4240793e0 msgr2=0x7fa4240797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- 172.21.15.33:0/3422095015 shutdown_connections 2024-04-23T18:47:17.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- 172.21.15.33:0/3422095015 wait complete. 2024-04-23T18:47:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 Processor -- start 2024-04-23T18:47:17.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.966+0000 7fa42b87c640 1 -- start start 2024-04-23T18:47:18.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241072a0 0x7fa424198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa42419cef0 0x7fa42419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241121f0 con 0x7fa4241021d0 2024-04-23T18:47:18.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa424112070 con 0x7fa4241072a0 2024-04-23T18:47:18.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa424112370 con 0x7fa42419cef0 2024-04-23T18:47:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:18.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33254/0 (socket says 172.21.15.33:33254) 2024-04-23T18:47:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 -- 172.21.15.33:0/1745082910 learned_addr learned my addr 172.21.15.33:0/1745082910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa428e17640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241072a0 0x7fa424198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429e19640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa42419cef0 0x7fa42419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 -- 172.21.15.33:0/1745082910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa42419cef0 msgr2=0x7fa42419d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa42419cef0 0x7fa42419d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 -- 172.21.15.33:0/1745082910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241072a0 msgr2=0x7fa424198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241072a0 0x7fa424198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:18.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 -- 172.21.15.33:0/1745082910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42419d9e0 con 0x7fa4241021d0 2024-04-23T18:47:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa429618640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9f0 tx=0x7fa42000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa41a7fc640 1 -- 172.21.15.33:0/1745082910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42000cd80 con 0x7fa4241021d0 2024-04-23T18:47:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.967+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42419dcd0 con 0x7fa4241021d0 2024-04-23T18:47:18.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.968+0000 7fa41a7fc640 1 -- 172.21.15.33:0/1745082910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4200108e0 con 0x7fa4241021d0 2024-04-23T18:47:18.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.968+0000 7fa41a7fc640 1 -- 172.21.15.33:0/1745082910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420018980 con 0x7fa4241021d0 2024-04-23T18:47:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.968+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4241a5510 con 0x7fa4241021d0 2024-04-23T18:47:18.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa41a7fc640 1 -- 172.21.15.33:0/1745082910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa420010430 con 0x7fa4241021d0 2024-04-23T18:47:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 Processor -- start 2024-04-23T18:47:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 -- start start 2024-04-23T18:47:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a60690b970 con 0x55a6069a6f40 2024-04-23T18:47:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a6068a84a0 con 0x55a6069a6620 2024-04-23T18:47:19.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa42e592740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a606907930 con 0x55a6069b1800 2024-04-23T18:47:19.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33264/0 (socket says 172.21.15.33:33264) 2024-04-23T18:47:19.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa429618640 1 -- 172.21.15.33:0/2128932374 learned_addr learned my addr 172.21.15.33:0/2128932374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.971+0000 7fa41a7fc640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 0x7fa4100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429e19640 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:19.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa428e17640 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429e19640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 0x7fa4100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa41a7fc640 1 -- 172.21.15.33:0/1745082910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fa4200d0ba0 con 0x7fa4241021d0 2024-04-23T18:47:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 -- 172.21.15.33:0/2128932374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 msgr2=0x55a6069b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:19.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 -- 172.21.15.33:0/2128932374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 msgr2=0x55a6069a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 -- 172.21.15.33:0/2128932374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a60661e210 con 0x55a6069a6f40 2024-04-23T18:47:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429618640 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa42009c830 tx=0x7fa42000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120ef222de3a3b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa429e19640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 0x7fa4100a6280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004af0 tx=0x7fa41c0095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa40f7fe640 1 -- 172.21.15.33:0/2128932374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42009d230 con 0x55a6069a6f40 2024-04-23T18:47:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa40f7fe640 1 -- 172.21.15.33:0/2128932374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42009cb30 con 0x55a6069a6f40 2024-04-23T18:47:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.972+0000 7fa40f7fe640 1 -- 172.21.15.33:0/2128932374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4200ce560 con 0x55a6069a6f40 2024-04-23T18:47:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- 172.21.15.33:0/2128932374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 msgr2=0x55a6069b12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa42009c830 tx=0x7fa42000cf20 comp rx=0 tx=0).stop 2024-04-23T18:47:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- 172.21.15.33:0/2128932374 shutdown_connections 2024-04-23T18:47:19.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069b12c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:19.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 --2- 172.21.15.33:0/2128932374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- 172.21.15.33:0/2128932374 >> 172.21.15.33:0/2128932374 conn(0x55a6069a2b30 msgr2=0x55a6069a2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- 172.21.15.33:0/2128932374 shutdown_connections 2024-04-23T18:47:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- 172.21.15.33:0/2128932374 wait complete. 2024-04-23T18:47:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.973+0000 7fa42e592740 1 -- start start 2024-04-23T18:47:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 Processor -- start 2024-04-23T18:47:20.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:20.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:20.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6069b43d0 con 0x55a6069a6620 2024-04-23T18:47:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a6068a60b0 con 0x55a6069b1800 2024-04-23T18:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a6069b4550 con 0x55a6069a6f40 2024-04-23T18:47:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa42e592740 10 client.0 _ll_register_callbacks cb 0x55a606893d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33270/0 (socket says 172.21.15.33:33270) 2024-04-23T18:47:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa429618640 1 -- 172.21.15.33:0/3981595845 learned_addr learned my addr 172.21.15.33:0/3981595845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:20.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa428e17640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.974+0000 7fa429e19640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 -- 172.21.15.33:0/3981595845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 msgr2=0x55a6069ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 -- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 msgr2=0x55a6069ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:20.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 -- 172.21.15.33:0/3981595845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6066ea620 con 0x55a6069a6620 2024-04-23T18:47:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa429618640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa420099500 tx=0x7fa42001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8a33d8c4b6910 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42009de60 con 0x55a6069a6620 2024-04-23T18:47:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fa4200ce560 con 0x55a6069a6620 2024-04-23T18:47:20.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa42009cb30 con 0x55a6069a6620 2024-04-23T18:47:20.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa4200b6050 con 0x55a6069a6620 2024-04-23T18:47:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 10 client.17640 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:20.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a6069c02a0 con 0x55a6069a6620 2024-04-23T18:47:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa42009cb30 con 0x55a6069a6620 2024-04-23T18:47:20.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fa420018bb0 con 0x55a6069a6620 2024-04-23T18:47:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 10 client.17640 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 10 client.17640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 20 client.17640 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a6069c15f0 0x55a6069c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:21.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 4 client.17640 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:47:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.975+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a606919390 con 0x55a6069c15f0 2024-04-23T18:47:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.976+0000 7fa428e17640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a6069c15f0 0x55a6069c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.976+0000 7fa428e17640 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a6069c15f0 0x55a6069c39e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342a3e6032222234 server_cookie=82783e603eaf791f in_seq=0 out_seq=0 2024-04-23T18:47:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.976+0000 7fa40cff9640 10 client.17640 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa40cff9640 1 -- 172.21.15.33:0/3981595845 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a606919390 con 0x55a6069c15f0 2024-04-23T18:47:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa40cff9640 10 client.17640 handle_command_reply: tid=0 2024-04-23T18:47:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 client.17640 shutdown 2024-04-23T18:47:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 msgr2=0x55a6069b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa420099500 tx=0x7fa42001f000 comp rx=0 tx=0).stop 2024-04-23T18:47:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 shutdown_connections 2024-04-23T18:47:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a6069c15f0 0x55a6069c39e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:21.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:21.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa40cff9640 0 client.17640 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 --2- 172.21.15.33:0/3981595845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b1100 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 >> 172.21.15.33:0/3981595845 conn(0x55a6069a2b30 msgr2=0x55a6069a88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:21.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 shutdown_connections 2024-04-23T18:47:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.977+0000 7fa42e592740 1 -- 172.21.15.33:0/3981595845 wait complete. 2024-04-23T18:47:22.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.978+0000 7fa42e592740 20 client.17640 trim_cache size 0 max 16384 2024-04-23T18:47:22.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 Processor -- start 2024-04-23T18:47:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 -- start start 2024-04-23T18:47:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069bd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:22.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241ad460 con 0x55a6069a6f40 2024-04-23T18:47:22.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241ab250 con 0x55a6069a6620 2024-04-23T18:47:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa42e592740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241acd90 con 0x55a6069b1800 2024-04-23T18:47:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33280/0 (socket says 172.21.15.33:33280) 2024-04-23T18:47:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 -- 172.21.15.33:0/2650252110 learned_addr learned my addr 172.21.15.33:0/2650252110 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa429618640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069bd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa429e19640 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:22.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 -- 172.21.15.33:0/2650252110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 msgr2=0x55a6069c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 -- 172.21.15.33:0/2650252110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 msgr2=0x55a6069bd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069bd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:22.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 -- 172.21.15.33:0/2650252110 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6069b44f0 con 0x55a6069a6f40 2024-04-23T18:47:22.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa429618640 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:47:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.993+0000 7fa428e17640 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa41400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d4cbf0d3e1bc4f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa40effd640 1 -- 172.21.15.33:0/2650252110 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414012760 con 0x55a6069a6f40 2024-04-23T18:47:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa40effd640 1 -- 172.21.15.33:0/2650252110 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa414012db0 con 0x55a6069a6f40 2024-04-23T18:47:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa40effd640 1 -- 172.21.15.33:0/2650252110 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414021330 con 0x55a6069a6f40 2024-04-23T18:47:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 -- 172.21.15.33:0/2650252110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 msgr2=0x55a6069ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa414000c00 tx=0x7fa41400b390 comp rx=0 tx=0).stop 2024-04-23T18:47:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 -- 172.21.15.33:0/2650252110 shutdown_connections 2024-04-23T18:47:23.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069b1800 0x55a6069c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:23.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6f40 0x55a6069ad460 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:23.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 --2- 172.21.15.33:0/2650252110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069a6620 0x55a6069bd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 -- 172.21.15.33:0/2650252110 >> 172.21.15.33:0/2650252110 conn(0x55a6069a2b30 msgr2=0x55a6069a1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 -- 172.21.15.33:0/2650252110 shutdown_connections 2024-04-23T18:47:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.994+0000 7fa42e592740 1 -- 172.21.15.33:0/2650252110 wait complete. 2024-04-23T18:47:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 -- start start 2024-04-23T18:47:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 Processor -- start 2024-04-23T18:47:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:23.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:23.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241abff0 con 0x55a6069a6620 2024-04-23T18:47:23.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241ab920 con 0x55a6069b1800 2024-04-23T18:47:23.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4241ac6c0 con 0x55a6069a6f40 2024-04-23T18:47:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa42e592740 10 client.0 _ll_register_callbacks cb 0x7fa420009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:24.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:24.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa429618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33290/0 (socket says 172.21.15.33:33290) 2024-04-23T18:47:24.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa429618640 1 -- 172.21.15.33:0/2068898640 learned_addr learned my addr 172.21.15.33:0/2068898640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:24.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa428e17640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.995+0000 7fa429e19640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:24.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 -- 172.21.15.33:0/2068898640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 msgr2=0x55a6069b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:24.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:24.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 -- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 msgr2=0x55a6069b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:24.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 -- 172.21.15.33:0/2068898640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6069b7a90 con 0x55a6069a6620 2024-04-23T18:47:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa428e17640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:24.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429e19640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:47:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa429618640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097b0 tx=0x7fa42000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90c58994eebfa78 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:24.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42009de60 con 0x55a6069a6620 2024-04-23T18:47:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fa420018d30 con 0x55a6069a6620 2024-04-23T18:47:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa414012db0 con 0x55a6069a6620 2024-04-23T18:47:24.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa42009d230 con 0x55a6069a6620 2024-04-23T18:47:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 10 client.17646 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a606a27770 con 0x55a6069a6620 2024-04-23T18:47:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa414012db0 con 0x55a6069a6620 2024-04-23T18:47:25.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fa42009b9f0 con 0x55a6069a6620 2024-04-23T18:47:25.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 10 client.17646 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:25.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 10 client.17646 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 20 client.17646 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.996+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a606a27cf0 0x55a606a2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:25.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.997+0000 7fa42e592740 4 client.17646 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:25.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.997+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a606913200 con 0x55a606a27cf0 2024-04-23T18:47:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.997+0000 7fa428e17640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a606a27cf0 0x55a606a2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.997+0000 7fa40cff9640 10 client.17646 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:25.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:21.997+0000 7fa428e17640 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a606a27cf0 0x55a606a2a0e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c37e375d18d3f0e6 server_cookie=2d8da339a5784d25 in_seq=0 out_seq=0 2024-04-23T18:47:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa40cff9640 1 -- 172.21.15.33:0/2068898640 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a606913200 con 0x55a606a27cf0 2024-04-23T18:47:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa40cff9640 10 client.17646 handle_command_reply: tid=0 2024-04-23T18:47:25.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 client.17646 shutdown 2024-04-23T18:47:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 msgr2=0x55a6069b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa4200097b0 tx=0x7fa42000cf20 comp rx=0 tx=0).stop 2024-04-23T18:47:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 shutdown_connections 2024-04-23T18:47:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a606a27cf0 0x55a606a2a0e0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a6069b1800 0x55a6069b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a6069a6f40 0x55a6069b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa40cff9640 0 client.17646 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:26.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 --2- 172.21.15.33:0/2068898640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6069a6620 0x55a6069b9a80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 >> 172.21.15.33:0/2068898640 conn(0x55a6069a2b30 msgr2=0x55a6069a1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 shutdown_connections 2024-04-23T18:47:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 1 -- 172.21.15.33:0/2068898640 wait complete. 2024-04-23T18:47:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.004+0000 7fa42e592740 20 client.17646 trim_cache size 0 max 16384 2024-04-23T18:47:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:26.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:26.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 msgr2=0x7fa4100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 0x7fa4100a6280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa41c004af0 tx=0x7fa41c0095c0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 msgr2=0x7fa424198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e9f0 tx=0x7fa42000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 shutdown_connections 2024-04-23T18:47:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa4100a3dc0 0x7fa4100a6280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa42419cef0 0x7fa42419d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4241072a0 0x7fa424198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 --2- 172.21.15.33:0/1745082910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4241021d0 0x7fa424198630 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:26.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 >> 172.21.15.33:0/1745082910 conn(0x7fa4240793e0 msgr2=0x7fa4240ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:26.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 shutdown_connections 2024-04-23T18:47:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:22.005+0000 7fa42b87c640 1 -- 172.21.15.33:0/1745082910 wait complete. 2024-04-23T18:47:26.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:22 smithi033 ceph-mon[29274]: pgmap v602: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:47:26.976 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:24 smithi033 ceph-mon[29274]: pgmap v603: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:47:27.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:26 smithi033 ceph-mon[29274]: pgmap v604: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-23T18:47:27.227 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:28 smithi033 ceph-mon[29274]: pgmap v605: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:47:27.271 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:30 smithi033 ceph-mon[29274]: pgmap v606: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:47:27.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:32 smithi033 ceph-mon[29274]: pgmap v607: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-23T18:47:27.351 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:17 smithi106 ceph-mon[32716]: pgmap v665: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:47:27.406 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:19 smithi106 ceph-mon[32716]: pgmap v666: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 826 B/s wr, 11 op/s 2024-04-23T18:47:27.511 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:21 smithi106 ceph-mon[32716]: pgmap v667: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:47:27.578 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:23 smithi106 ceph-mon[32716]: pgmap v668: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:47:27.643 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:25 smithi106 ceph-mon[32716]: pgmap v669: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:47:27.673 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:17 smithi122 ceph-mon[32546]: pgmap v665: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:47:27.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:19 smithi122 ceph-mon[32546]: pgmap v666: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 826 B/s wr, 11 op/s 2024-04-23T18:47:27.758 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:21 smithi122 ceph-mon[32546]: pgmap v667: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:47:27.766 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:23 smithi122 ceph-mon[32546]: pgmap v668: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:47:27.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:25 smithi122 ceph-mon[32546]: pgmap v669: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:47:27.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 -- 172.21.15.33:0/1754634332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 msgr2=0x7f87b810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 --2- 172.21.15.33:0/1754634332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b810ee40 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009ca0 tx=0x7f87b002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:47:27.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 -- 172.21.15.33:0/1754634332 shutdown_connections 2024-04-23T18:47:27.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 --2- 172.21.15.33:0/1754634332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b810ee40 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 --2- 172.21.15.33:0/1754634332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b8102b50 0x7f87b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:28.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 --2- 172.21.15.33:0/1754634332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 0x7f87b8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.104+0000 7f87bfcef640 1 -- 172.21.15.33:0/1754634332 >> 172.21.15.33:0/1754634332 conn(0x7f87b80793e0 msgr2=0x7f87b80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 -- 172.21.15.33:0/1754634332 shutdown_connections 2024-04-23T18:47:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 -- 172.21.15.33:0/1754634332 wait complete. 2024-04-23T18:47:28.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 Processor -- start 2024-04-23T18:47:28.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 -- start start 2024-04-23T18:47:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 0x7f87b81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.105+0000 7f87bfcef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 0x7f87b819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:28.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bfcef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81121c0 con 0x7f87b8107270 2024-04-23T18:47:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bfcef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8112040 con 0x7f87b81021a0 2024-04-23T18:47:28.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bfcef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87b8112340 con 0x7f87b819cf30 2024-04-23T18:47:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60532/0 (socket says 172.21.15.33:60532) 2024-04-23T18:47:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 -- 172.21.15.33:0/3570519006 learned_addr learned my addr 172.21.15.33:0/3570519006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 0x7f87b81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87be28c640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 0x7f87b819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 -- 172.21.15.33:0/3570519006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 msgr2=0x7f87b819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 0x7f87b819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 -- 172.21.15.33:0/3570519006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 msgr2=0x7f87b81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 0x7f87b81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:29.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 -- 172.21.15.33:0/3570519006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b819da20 con 0x7f87b8107270 2024-04-23T18:47:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87be28c640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 0x7f87b819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:29.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f87a8005fd0 tx=0x7f87a800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:29.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87aeffd640 1 -- 172.21.15.33:0/3570519006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8017070 con 0x7f87b8107270 2024-04-23T18:47:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b819dd10 con 0x7f87b8107270 2024-04-23T18:47:29.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87aeffd640 1 -- 172.21.15.33:0/3570519006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8009c30 con 0x7f87b8107270 2024-04-23T18:47:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.106+0000 7f87aeffd640 1 -- 172.21.15.33:0/3570519006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8003d30 con 0x7f87b8107270 2024-04-23T18:47:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.107+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b81a5500 con 0x7f87b8107270 2024-04-23T18:47:29.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.109+0000 7f87aeffd640 1 -- 172.21.15.33:0/3570519006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f87a8003ed0 con 0x7f87b8107270 2024-04-23T18:47:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.109+0000 7f87c2a07740 1 Processor -- start 2024-04-23T18:47:29.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.109+0000 7f87c2a07740 1 -- start start 2024-04-23T18:47:29.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbc2620 0x55f45dbc2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:29.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbcd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f45db27970 con 0x55f45dbcd800 2024-04-23T18:47:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f45dac44a0 con 0x55f45dbc2f40 2024-04-23T18:47:29.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87c2a07740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f45db23930 con 0x55f45dbc2620 2024-04-23T18:47:29.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bd28a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:29.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bd28a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60544/0 (socket says 172.21.15.33:60544) 2024-04-23T18:47:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bd28a640 1 -- 172.21.15.33:0/3073930091 learned_addr learned my addr 172.21.15.33:0/3073930091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:29.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87aeffd640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 0x7f87a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:29.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbcd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87be28c640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbc2620 0x55f45dbc2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87be28c640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 0x7f87a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87aeffd640 1 -- 172.21.15.33:0/3570519006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7f87a80d03f0 con 0x7f87b8107270 2024-04-23T18:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 -- 172.21.15.33:0/3073930091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbc2620 msgr2=0x55f45dbc2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbc2620 0x55f45dbc2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 -- 172.21.15.33:0/3073930091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 msgr2=0x55f45dbcfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bda8b640 1 -- 172.21.15.33:0/3073930091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f45d83a210 con 0x55f45dbc2f40 2024-04-23T18:47:30.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.110+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:47:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.111+0000 7f87be28c640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 0x7f87a40a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005fa0 tx=0x7f87b00095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.111+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbcd2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009c20 tx=0x7f87b4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca57050b04bf7638 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:30.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.111+0000 7f87a3fff640 1 -- 172.21.15.33:0/3073930091 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4004050 con 0x55f45dbc2f40 2024-04-23T18:47:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.111+0000 7f87a3fff640 1 -- 172.21.15.33:0/3073930091 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b40054d0 con 0x55f45dbc2f40 2024-04-23T18:47:30.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.111+0000 7f87a3fff640 1 -- 172.21.15.33:0/3073930091 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4004bf0 con 0x55f45dbc2f40 2024-04-23T18:47:30.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- 172.21.15.33:0/3073930091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 msgr2=0x55f45dbcd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:30.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbcd2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009c20 tx=0x7f87b4011810 comp rx=0 tx=0).stop 2024-04-23T18:47:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- 172.21.15.33:0/3073930091 shutdown_connections 2024-04-23T18:47:30.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbcd800 0x55f45dbcfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:30.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbcd2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:30.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3073930091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbc2620 0x55f45dbc2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:30.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- 172.21.15.33:0/3073930091 >> 172.21.15.33:0/3073930091 conn(0x55f45dbbeb30 msgr2=0x55f45dbbef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- 172.21.15.33:0/3073930091 shutdown_connections 2024-04-23T18:47:30.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- 172.21.15.33:0/3073930091 wait complete. 2024-04-23T18:47:30.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 -- start start 2024-04-23T18:47:30.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.112+0000 7f87c2a07740 1 Processor -- start 2024-04-23T18:47:30.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f45dbd0430 con 0x55f45dbc2620 2024-04-23T18:47:31.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f45dac20b0 con 0x55f45dbc2f40 2024-04-23T18:47:31.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f45dbd05b0 con 0x55f45dbcd800 2024-04-23T18:47:31.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:31.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87c2a07740 10 client.0 _ll_register_callbacks cb 0x55f45daafd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:31.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60560/0 (socket says 172.21.15.33:60560) 2024-04-23T18:47:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 -- 172.21.15.33:0/3180386730 learned_addr learned my addr 172.21.15.33:0/3180386730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87be28c640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:31.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 -- 172.21.15.33:0/3180386730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 msgr2=0x55f45dbca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:31.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 -- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 msgr2=0x55f45dbca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:31.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bda8b640 1 -- 172.21.15.33:0/3180386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f45d906620 con 0x55f45dbc2620 2024-04-23T18:47:31.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.113+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:47:31.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87bda8b640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f87b4005e00 tx=0x7f87b4004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f973849e3a37c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:31.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4029070 con 0x55f45dbc2620 2024-04-23T18:47:32.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7f87b4003a00 con 0x55f45dbc2620 2024-04-23T18:47:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b40054d0 con 0x55f45dbc2620 2024-04-23T18:47:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87b4035050 con 0x55f45dbc2620 2024-04-23T18:47:32.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 10 client.17658 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:32.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f45dbdc340 con 0x55f45dbc2620 2024-04-23T18:47:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b40054d0 con 0x55f45dbc2620 2024-04-23T18:47:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f87b4005180 con 0x55f45dbc2620 2024-04-23T18:47:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 10 client.17658 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 20 client.17658 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dbdd750 0x55f45dbdfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 4 client.17658 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:47:32.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.114+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f45db35390 con 0x55f45dbdd750 2024-04-23T18:47:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.115+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dbdd750 0x55f45dbdfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.115+0000 7f87bd28a640 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dbdd750 0x55f45dbdfb40 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3c36c5797257fc server_cookie=87f7c301166a384d in_seq=0 out_seq=0 2024-04-23T18:47:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.115+0000 7f87a17fa640 10 client.17658 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:32.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87a17fa640 1 -- 172.21.15.33:0/3180386730 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f45db35390 con 0x55f45dbdd750 2024-04-23T18:47:32.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87a17fa640 10 client.17658 handle_command_reply: tid=0 2024-04-23T18:47:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 client.17658 shutdown 2024-04-23T18:47:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 msgr2=0x55f45dbcd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:32.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f87b4005e00 tx=0x7f87b4004520 comp rx=0 tx=0).stop 2024-04-23T18:47:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 shutdown_connections 2024-04-23T18:47:32.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dbdd750 0x55f45dbdfb40 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:32.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 --2- 172.21.15.33:0/3180386730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbcd150 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:32.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87a17fa640 0 client.17658 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 >> 172.21.15.33:0/3180386730 conn(0x55f45dbbeb30 msgr2=0x55f45dbc4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 shutdown_connections 2024-04-23T18:47:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 1 -- 172.21.15.33:0/3180386730 wait complete. 2024-04-23T18:47:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.116+0000 7f87c2a07740 20 client.17658 trim_cache size 0 max 16384 2024-04-23T18:47:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.131+0000 7f87c2a07740 1 Processor -- start 2024-04-23T18:47:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.131+0000 7f87c2a07740 1 -- start start 2024-04-23T18:47:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbbdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:33.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81af6a0 con 0x55f45dbc2620 2024-04-23T18:47:33.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81ad490 con 0x55f45dbc2f40 2024-04-23T18:47:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87c2a07740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81aefd0 con 0x55f45dbcd800 2024-04-23T18:47:33.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60574/0 (socket says 172.21.15.33:60574) 2024-04-23T18:47:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 -- 172.21.15.33:0/580690171 learned_addr learned my addr 172.21.15.33:0/580690171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bd28a640 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbbdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 -- 172.21.15.33:0/580690171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 msgr2=0x55f45dbccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 -- 172.21.15.33:0/580690171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 msgr2=0x55f45dbbdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:33.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbbdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:33.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 -- 172.21.15.33:0/580690171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f45dbc8990 con 0x55f45dbc2620 2024-04-23T18:47:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87bda8b640 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011850 tx=0x7f87b4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90edb094861368a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87a37fe640 1 -- 172.21.15.33:0/580690171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4029070 con 0x55f45dbc2620 2024-04-23T18:47:33.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87a37fe640 1 -- 172.21.15.33:0/580690171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b40238f0 con 0x55f45dbc2620 2024-04-23T18:47:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.132+0000 7f87a37fe640 1 -- 172.21.15.33:0/580690171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4005280 con 0x55f45dbc2620 2024-04-23T18:47:33.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- 172.21.15.33:0/580690171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 msgr2=0x55f45dbd92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f87b4011850 tx=0x7f87b4005bb0 comp rx=0 tx=0).stop 2024-04-23T18:47:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- 172.21.15.33:0/580690171 shutdown_connections 2024-04-23T18:47:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dbccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dbbdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 --2- 172.21.15.33:0/580690171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dbd92f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- 172.21.15.33:0/580690171 >> 172.21.15.33:0/580690171 conn(0x55f45dbbeb30 msgr2=0x55f45dbbd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- 172.21.15.33:0/580690171 shutdown_connections 2024-04-23T18:47:34.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- 172.21.15.33:0/580690171 wait complete. 2024-04-23T18:47:34.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 -- start start 2024-04-23T18:47:34.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.133+0000 7f87c2a07740 1 Processor -- start 2024-04-23T18:47:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dc40540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dc3a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:34.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81ae230 con 0x55f45dbc2620 2024-04-23T18:47:34.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81adb60 con 0x55f45dbc2f40 2024-04-23T18:47:34.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87b81ae900 con 0x55f45dbcd800 2024-04-23T18:47:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87c2a07740 10 client.0 _ll_register_callbacks cb 0x7f87b4009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60590/0 (socket says 172.21.15.33:60590) 2024-04-23T18:47:34.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 -- 172.21.15.33:0/901651305 learned_addr learned my addr 172.21.15.33:0/901651305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bd28a640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dc40540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:34.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87be28c640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dc3a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 -- 172.21.15.33:0/901651305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 msgr2=0x55f45dc3a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:35.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dc3a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:35.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 -- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 msgr2=0x55f45dc40540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dc40540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 -- 172.21.15.33:0/901651305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f45dc3ab00 con 0x55f45dbc2620 2024-04-23T18:47:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87be28c640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dc3a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87bda8b640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4023cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f172d4a58e2a5c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.134+0000 7f87a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4029040 con 0x55f45dbc2620 2024-04-23T18:47:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7f87b4003800 con 0x55f45dbc2620 2024-04-23T18:47:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87b40238f0 con 0x55f45dbc2620 2024-04-23T18:47:35.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87b4003a00 con 0x55f45dbc2620 2024-04-23T18:47:35.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 10 client.17664 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f45dc47810 con 0x55f45dbc2620 2024-04-23T18:47:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b40238f0 con 0x55f45dbc2620 2024-04-23T18:47:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7f87b402ec60 con 0x55f45dbc2620 2024-04-23T18:47:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 10 client.17664 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 10 client.17664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 20 client.17664 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:35.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dc47dc0 0x55f45dc4a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 4 client.17664 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.135+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f45db2f200 con 0x55f45dc47dc0 2024-04-23T18:47:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.136+0000 7f87bd28a640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dc47dc0 0x55f45dc4a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.136+0000 7f87a17fa640 10 client.17664 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.136+0000 7f87bd28a640 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dc47dc0 0x55f45dc4a1b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ab366c565a0760 server_cookie=36ffc2e746ab8922 in_seq=0 out_seq=0 2024-04-23T18:47:36.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.140+0000 7f87a17fa640 1 -- 172.21.15.33:0/901651305 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f45db2f200 con 0x55f45dc47dc0 2024-04-23T18:47:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.140+0000 7f87a17fa640 10 client.17664 handle_command_reply: tid=0 2024-04-23T18:47:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.140+0000 7f87c2a07740 1 client.17664 shutdown 2024-04-23T18:47:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.140+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 msgr2=0x55f45dc40000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.140+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87b4000c00 tx=0x7f87b4023cd0 comp rx=0 tx=0).stop 2024-04-23T18:47:36.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 shutdown_connections 2024-04-23T18:47:36.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f45dc47dc0 0x55f45dc4a1b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f45dbcd800 0x55f45dc3a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:36.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87a17fa640 0 client.17664 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f45dbc2f40 0x55f45dc40540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 --2- 172.21.15.33:0/901651305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f45dbc2620 0x55f45dc40000 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 >> 172.21.15.33:0/901651305 conn(0x55f45dbbeb30 msgr2=0x55f45dbbdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 shutdown_connections 2024-04-23T18:47:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 1 -- 172.21.15.33:0/901651305 wait complete. 2024-04-23T18:47:36.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87c2a07740 20 client.17664 trim_cache size 0 max 16384 2024-04-23T18:47:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:36.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:37.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 msgr2=0x7f87a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:37.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 0x7f87a40a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87b0005fa0 tx=0x7f87b00095c0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 msgr2=0x7f87b8198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:37.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f87a8005fd0 tx=0x7f87a800dde0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 shutdown_connections 2024-04-23T18:47:37.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f87a40a3dc0 0x7f87a40a6280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87b819cf30 0x7f87b819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87b8107270 0x7f87b8198c00 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 --2- 172.21.15.33:0/3570519006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87b81021a0 0x7f87b81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:37.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 >> 172.21.15.33:0/3570519006 conn(0x7f87b80793e0 msgr2=0x7f87b80ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:37.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.141+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 shutdown_connections 2024-04-23T18:47:37.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:35.142+0000 7f87bfcef640 1 -- 172.21.15.33:0/3570519006 wait complete. 2024-04-23T18:47:37.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:34 smithi033 ceph-mon[29274]: pgmap v608: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-23T18:47:37.589 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:36 smithi033 ceph-mon[29274]: pgmap v609: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-23T18:47:37.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:38 smithi033 ceph-mon[29274]: pgmap v610: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:37.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:40 smithi033 ceph-mon[29274]: pgmap v611: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:37.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:42 smithi033 ceph-mon[29274]: pgmap v612: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:37.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:44 smithi033 ceph-mon[29274]: pgmap v613: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:38.017 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:46 smithi033 ceph-mon[29274]: pgmap v614: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:38.088 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:27 smithi122 ceph-mon[32546]: pgmap v670: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 293 B/s wr, 10 op/s 2024-04-23T18:47:38.209 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:29 smithi122 ceph-mon[32546]: pgmap v671: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.285 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:31 smithi122 ceph-mon[32546]: pgmap v672: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.299 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:33 smithi122 ceph-mon[32546]: pgmap v673: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:38.387 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:35 smithi122 ceph-mon[32546]: pgmap v674: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:27 smithi106 ceph-mon[32716]: pgmap v670: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 293 B/s wr, 10 op/s 2024-04-23T18:47:38.525 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:29 smithi106 ceph-mon[32716]: pgmap v671: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.569 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:31 smithi106 ceph-mon[32716]: pgmap v672: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.578 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:33 smithi106 ceph-mon[32716]: pgmap v673: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:38.587 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:35 smithi106 ceph-mon[32716]: pgmap v674: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:38.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:37 smithi122 ceph-mon[32546]: pgmap v675: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:47:38.608 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:37 smithi106 ceph-mon[32716]: pgmap v675: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:47:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- 172.21.15.33:0/3303962906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd01016a0 msgr2=0x7fbbd010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3303962906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd01016a0 0x7fbbd010e3f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc002a410 comp rx=0 tx=0).stop 2024-04-23T18:47:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- 172.21.15.33:0/3303962906 shutdown_connections 2024-04-23T18:47:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3303962906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd010e930 0x7fbbd0110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3303962906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd01016a0 0x7fbbd010e3f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3303962906 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd0100d80 0x7fbbd0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- 172.21.15.33:0/3303962906 >> 172.21.15.33:0/3303962906 conn(0x7fbbd00771d0 msgr2=0x7fbbd00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:39.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- 172.21.15.33:0/3303962906 shutdown_connections 2024-04-23T18:47:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- 172.21.15.33:0/3303962906 wait complete. 2024-04-23T18:47:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 Processor -- start 2024-04-23T18:47:39.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.241+0000 7fbbd730f640 1 -- start start 2024-04-23T18:47:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd0100d80 0x7fbbd01a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:39.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd01016a0 0x7fbbd01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:39.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd0114440 con 0x7fbbd010e930 2024-04-23T18:47:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01142c0 con 0x7fbbd01016a0 2024-04-23T18:47:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd730f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01145c0 con 0x7fbbd0100d80 2024-04-23T18:47:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56144/0 (socket says 172.21.15.33:56144) 2024-04-23T18:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 -- 172.21.15.33:0/3810684924 learned_addr learned my addr 172.21.15.33:0/3810684924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd01016a0 0x7fbbd01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd0100d80 0x7fbbd01a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 -- 172.21.15.33:0/3810684924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd0100d80 msgr2=0x7fbbd01a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd0100d80 0x7fbbd01a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:39.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 -- 172.21.15.33:0/3810684924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd01016a0 msgr2=0x7fbbd01a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd01016a0 0x7fbbd01a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 -- 172.21.15.33:0/3810684924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd019afd0 con 0x7fbbd010e930 2024-04-23T18:47:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.242+0000 7fbbd58ac640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800eb10 tx=0x7fbbc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.243+0000 7fbbc67fc640 1 -- 172.21.15.33:0/3810684924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8004b20 con 0x7fbbd010e930 2024-04-23T18:47:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.243+0000 7fbbc67fc640 1 -- 172.21.15.33:0/3810684924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc8004cc0 con 0x7fbbd010e930 2024-04-23T18:47:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.243+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd019b2c0 con 0x7fbbd010e930 2024-04-23T18:47:40.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.243+0000 7fbbc67fc640 1 -- 172.21.15.33:0/3810684924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80107b0 con 0x7fbbd010e930 2024-04-23T18:47:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.243+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd01a74e0 con 0x7fbbd010e930 2024-04-23T18:47:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.245+0000 7fbbc67fc640 1 -- 172.21.15.33:0/3810684924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbc8005020 con 0x7fbbd010e930 2024-04-23T18:47:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.245+0000 7fbbda021740 1 Processor -- start 2024-04-23T18:47:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 -- start start 2024-04-23T18:47:40.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:40.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b9344d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b933a7970 con 0x560b93442620 2024-04-23T18:47:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b933444a0 con 0x560b9344d800 2024-04-23T18:47:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbda021740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b933a3930 con 0x560b93442f40 2024-04-23T18:47:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbc67fc640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 0x7fbbbc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56150/0 (socket says 172.21.15.33:56150) 2024-04-23T18:47:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 -- 172.21.15.33:0/2143593518 learned_addr learned my addr 172.21.15.33:0/2143593518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b9344d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:40.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd58ac640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 0x7fbbbc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbc67fc640 1 -- 172.21.15.33:0/3810684924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fbbc80d0ee0 con 0x7fbbd010e930 2024-04-23T18:47:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 -- 172.21.15.33:0/2143593518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 msgr2=0x560b9344d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:40.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b9344d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 -- 172.21.15.33:0/2143593518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 msgr2=0x560b9344fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:41.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd48aa640 1 -- 172.21.15.33:0/2143593518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b930ba210 con 0x560b93442620 2024-04-23T18:47:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.246+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b9344d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005c40 tx=0x7fbbc0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538824aed02183de server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbd58ac640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 0x7fbbbc0a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8017000 tx=0x7fbbc8007850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:41.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbbb7fe640 1 -- 172.21.15.33:0/2143593518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0044020 con 0x560b93442620 2024-04-23T18:47:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbbb7fe640 1 -- 172.21.15.33:0/2143593518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc00316a0 con 0x560b93442620 2024-04-23T18:47:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbbb7fe640 1 -- 172.21.15.33:0/2143593518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0035040 con 0x560b93442620 2024-04-23T18:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 -- 172.21.15.33:0/2143593518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 msgr2=0x560b93442a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:41.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0005c40 tx=0x7fbbc0005b50 comp rx=0 tx=0).stop 2024-04-23T18:47:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 -- 172.21.15.33:0/2143593518 shutdown_connections 2024-04-23T18:47:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b9344d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 --2- 172.21.15.33:0/2143593518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93442a00 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.247+0000 7fbbda021740 1 -- 172.21.15.33:0/2143593518 >> 172.21.15.33:0/2143593518 conn(0x560b9343eb30 msgr2=0x560b9343ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:41.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- 172.21.15.33:0/2143593518 shutdown_connections 2024-04-23T18:47:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- 172.21.15.33:0/2143593518 wait complete. 2024-04-23T18:47:41.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- start start 2024-04-23T18:47:41.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 Processor -- start 2024-04-23T18:47:41.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442620 0x560b9344d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560b934503d0 con 0x560b93442f40 2024-04-23T18:47:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b933420b0 con 0x560b9344d800 2024-04-23T18:47:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560b93450550 con 0x560b93442620 2024-04-23T18:47:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.248+0000 7fbbda021740 10 client.0 _ll_register_callbacks cb 0x560b9332fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:42.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56162/0 (socket says 172.21.15.33:56162) 2024-04-23T18:47:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 -- 172.21.15.33:0/1195630410 learned_addr learned my addr 172.21.15.33:0/1195630410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:42.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442620 0x560b9344d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:42.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:42.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 -- 172.21.15.33:0/1195630410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442620 msgr2=0x560b9344d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442620 0x560b9344d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 -- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 msgr2=0x560b9344a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 -- 172.21.15.33:0/1195630410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b93186620 con 0x560b93442f40 2024-04-23T18:47:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00baa0 tx=0x7fbbcc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2829e9d90efca09 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:42.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc01b640 con 0x560b93442f40 2024-04-23T18:47:42.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fbbcc011440 con 0x560b93442f40 2024-04-23T18:47:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc00316a0 con 0x560b93442f40 2024-04-23T18:47:42.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbcc01b7e0 con 0x560b93442f40 2024-04-23T18:47:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbda021740 10 client.17676 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:42.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.249+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b9345c360 con 0x560b93442f40 2024-04-23T18:47:42.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc00316a0 con 0x560b93442f40 2024-04-23T18:47:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fbbcc019020 con 0x560b93442f40 2024-04-23T18:47:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 10 client.17676 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:42.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:42.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 20 client.17676 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:42.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b9345fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:42.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 4 client.17676 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:47:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b933b5390 con 0x560b9345d870 2024-04-23T18:47:43.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.250+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b9345fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b9345fc60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab6891570e9c4529 server_cookie=7002e9db5e84a717 in_seq=0 out_seq=0 2024-04-23T18:47:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbb8ff9640 10 client.17676 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/1195630410 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b933b5390 con 0x560b9345d870 2024-04-23T18:47:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbb8ff9640 10 client.17676 handle_command_reply: tid=0 2024-04-23T18:47:43.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 client.17676 shutdown 2024-04-23T18:47:43.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 msgr2=0x560b9344a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:43.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc00baa0 tx=0x7fbbcc00bf60 comp rx=0 tx=0).stop 2024-04-23T18:47:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 shutdown_connections 2024-04-23T18:47:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b9345fc60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442f40 0x560b9344a440 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbb8ff9640 0 client.17676 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.251+0000 7fbbda021740 1 --2- 172.21.15.33:0/1195630410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442620 0x560b9344d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.252+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 >> 172.21.15.33:0/1195630410 conn(0x560b9343eb30 msgr2=0x560b934448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.252+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 shutdown_connections 2024-04-23T18:47:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.252+0000 7fbbda021740 1 -- 172.21.15.33:0/1195630410 wait complete. 2024-04-23T18:47:43.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.252+0000 7fbbda021740 20 client.17676 trim_cache size 0 max 16384 2024-04-23T18:47:43.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 Processor -- start 2024-04-23T18:47:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 -- start start 2024-04-23T18:47:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b93445970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:43.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:43.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01b1810 con 0x560b93442620 2024-04-23T18:47:43.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01af600 con 0x560b9344d800 2024-04-23T18:47:43.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbda021740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01b1140 con 0x560b93442f40 2024-04-23T18:47:43.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:43.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56168/0 (socket says 172.21.15.33:56168) 2024-04-23T18:47:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbd48aa640 1 -- 172.21.15.33:0/3999841060 learned_addr learned my addr 172.21.15.33:0/3999841060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:44.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.267+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:44.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b93445970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 -- 172.21.15.33:0/3999841060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 msgr2=0x560b93445970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:44.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b93445970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:44.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 -- 172.21.15.33:0/3999841060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 msgr2=0x560b9344cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:44.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 -- 172.21.15.33:0/3999841060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9344ad10 con 0x560b93442620 2024-04-23T18:47:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b93445970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:47:44.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0033000 tx=0x7fbbc0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7231abf66d882e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbbaffd640 1 -- 172.21.15.33:0/3999841060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0044070 con 0x560b93442620 2024-04-23T18:47:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbbaffd640 1 -- 172.21.15.33:0/3999841060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0005420 con 0x560b93442620 2024-04-23T18:47:44.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.268+0000 7fbbbaffd640 1 -- 172.21.15.33:0/3999841060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0035040 con 0x560b93442620 2024-04-23T18:47:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- 172.21.15.33:0/3999841060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 msgr2=0x560b93459290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:44.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0033000 tx=0x7fbbc0005fd0 comp rx=0 tx=0).stop 2024-04-23T18:47:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- 172.21.15.33:0/3999841060 shutdown_connections 2024-04-23T18:47:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b9344d800 0x560b9344cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b93442f40 0x560b93445970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- 172.21.15.33:0/3999841060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b93459290 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- 172.21.15.33:0/3999841060 >> 172.21.15.33:0/3999841060 conn(0x560b9343eb30 msgr2=0x560b93445260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- 172.21.15.33:0/3999841060 shutdown_connections 2024-04-23T18:47:45.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- 172.21.15.33:0/3999841060 wait complete. 2024-04-23T18:47:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- start start 2024-04-23T18:47:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 Processor -- start 2024-04-23T18:47:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b93442f40 0x560b93455ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9344d800 0x560b934466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01b03a0 con 0x560b93442620 2024-04-23T18:47:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01afcd0 con 0x560b93442f40 2024-04-23T18:47:45.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd01b0a70 con 0x560b9344d800 2024-04-23T18:47:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.269+0000 7fbbda021740 10 client.0 _ll_register_callbacks cb 0x7fbbcc004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:45.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56182/0 (socket says 172.21.15.33:56182) 2024-04-23T18:47:45.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 -- 172.21.15.33:0/674674957 learned_addr learned my addr 172.21.15.33:0/674674957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b93442f40 0x560b93455ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9344d800 0x560b934466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 -- 172.21.15.33:0/674674957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9344d800 msgr2=0x560b934466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:45.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9344d800 0x560b934466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 -- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b93442f40 msgr2=0x560b93455ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b93442f40 0x560b93455ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:45.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 -- 172.21.15.33:0/674674957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b93446bf0 con 0x560b93442620 2024-04-23T18:47:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbd48aa640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbbc002a930 tx=0x7fbbc00098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e5e0413834bce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:45.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc005b060 con 0x560b93442620 2024-04-23T18:47:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7fbbc0055de0 con 0x560b93442620 2024-04-23T18:47:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.270+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc0005420 con 0x560b93442620 2024-04-23T18:47:46.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fbbc0044050 con 0x560b93442620 2024-04-23T18:47:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 10 client.17682 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:46.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560b934c3540 con 0x560b93442620 2024-04-23T18:47:46.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0005420 con 0x560b93442620 2024-04-23T18:47:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7fbbc004a760 con 0x560b93442620 2024-04-23T18:47:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 10 client.17682 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 20 client.17682 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b934c5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 4 client.17682 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b933af200 con 0x560b9345d870 2024-04-23T18:47:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.271+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b934c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.272+0000 7fbbd50ab640 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b934c5780 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c703a1f062ffef7 server_cookie=29ee500011d3ce4f in_seq=0 out_seq=0 2024-04-23T18:47:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.272+0000 7fbbb8ff9640 10 client.17682 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbb8ff9640 1 -- 172.21.15.33:0/674674957 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b933af200 con 0x560b9345d870 2024-04-23T18:47:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbb8ff9640 10 client.17682 handle_command_reply: tid=0 2024-04-23T18:47:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 client.17682 shutdown 2024-04-23T18:47:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 msgr2=0x560b934557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbbc002a930 tx=0x7fbbc00098c0 comp rx=0 tx=0).stop 2024-04-23T18:47:46.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 shutdown_connections 2024-04-23T18:47:46.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x560b9345d870 0x560b934c5780 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560b9344d800 0x560b934466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b93442f40 0x560b93455ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbb8ff9640 0 client.17682 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 --2- 172.21.15.33:0/674674957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b93442620 0x560b934557a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 >> 172.21.15.33:0/674674957 conn(0x560b9343eb30 msgr2=0x560b93444860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:47.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 shutdown_connections 2024-04-23T18:47:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 1 -- 172.21.15.33:0/674674957 wait complete. 2024-04-23T18:47:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.276+0000 7fbbda021740 20 client.17682 trim_cache size 0 max 16384 2024-04-23T18:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:47.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 msgr2=0x7fbbbc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 0x7fbbbc0a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8017000 tx=0x7fbbc8007850 comp rx=0 tx=0).stop 2024-04-23T18:47:47.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 msgr2=0x7fbbd019a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fbbc800eb10 tx=0x7fbbc800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 shutdown_connections 2024-04-23T18:47:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbbc0a3dc0 0x7fbbbc0a6280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbd010e930 0x7fbbd019a800 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbd01016a0 0x7fbbd01a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 --2- 172.21.15.33:0/3810684924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbd0100d80 0x7fbbd01a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:48.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 >> 172.21.15.33:0/3810684924 conn(0x7fbbd00771d0 msgr2=0x7fbbd010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 shutdown_connections 2024-04-23T18:47:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:45:48.277+0000 7fbbd730f640 1 -- 172.21.15.33:0/3810684924 wait complete. 2024-04-23T18:47:48.202 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:39 smithi106 ceph-mon[32716]: pgmap v676: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.211 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:41 smithi106 ceph-mon[32716]: pgmap v677: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.288 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:43 smithi106 ceph-mon[32716]: pgmap v678: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:48.331 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:45 smithi106 ceph-mon[32716]: pgmap v679: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.340 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:39 smithi122 ceph-mon[32546]: pgmap v676: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.348 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:41 smithi122 ceph-mon[32546]: pgmap v677: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:43 smithi122 ceph-mon[32546]: pgmap v678: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:48.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:45 smithi122 ceph-mon[32546]: pgmap v679: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:48.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:48 smithi033 ceph-mon[29274]: pgmap v615: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:47:48.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:50 smithi033 ceph-mon[29274]: pgmap v616: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 8 op/s 2024-04-23T18:47:48.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:52 smithi033 ceph-mon[29274]: pgmap v617: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:48.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:47:48.524 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:47:48.571 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:47:48.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:53 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:47:48.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:54 smithi033 ceph-mon[29274]: pgmap v618: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:48.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:56 smithi033 ceph-mon[29274]: pgmap v619: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:48.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:45:58 smithi033 ceph-mon[29274]: pgmap v620: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:48.760 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-23T18:47:48.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-23T18:47:48.801 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:00 smithi033 ceph-mon[29274]: pgmap v621: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-23T18:47:48.873 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:47 smithi106 ceph-mon[32716]: pgmap v680: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:48.881 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:47 smithi122 ceph-mon[32546]: pgmap v680: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:48.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- 172.21.15.33:0/4043037244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 msgr2=0x7ff8a006a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:48.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/4043037244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a006a780 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff888009900 tx=0x7ff88802a410 comp rx=0 tx=0).stop 2024-04-23T18:47:48.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- 172.21.15.33:0/4043037244 shutdown_connections 2024-04-23T18:47:48.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/4043037244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 0x7ff8a0110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/4043037244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a006b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/4043037244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a006a780 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:49.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- 172.21.15.33:0/4043037244 >> 172.21.15.33:0/4043037244 conn(0x7ff8a0076c50 msgr2=0x7ff8a0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- 172.21.15.33:0/4043037244 shutdown_connections 2024-04-23T18:47:49.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- 172.21.15.33:0/4043037244 wait complete. 2024-04-23T18:47:49.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 Processor -- start 2024-04-23T18:47:49.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.373+0000 7ff8a65c6640 1 -- start start 2024-04-23T18:47:49.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a01a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:49.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 0x7ff8a019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0114470 con 0x7ff8a0102f00 2024-04-23T18:47:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01142f0 con 0x7ff8a010e8a0 2024-04-23T18:47:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a65c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01145f0 con 0x7ff8a006acc0 2024-04-23T18:47:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:49.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59190/0 (socket says 172.21.15.33:59190) 2024-04-23T18:47:49.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2287197514 learned_addr learned my addr 172.21.15.33:0/2287197514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 0x7ff8a019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:49.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89ffff640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a01a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:49.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2287197514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 msgr2=0x7ff8a01a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a01a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:49.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2287197514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 msgr2=0x7ff8a019a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:49.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 0x7ff8a019a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2287197514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a019aff0 con 0x7ff8a0102f00 2024-04-23T18:47:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.374+0000 7ff89ffff640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:49.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff89000d950 tx=0x7ff89000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff89d7fa640 1 -- 172.21.15.33:0/2287197514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890017030 con 0x7ff8a0102f00 2024-04-23T18:47:49.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff89d7fa640 1 -- 172.21.15.33:0/2287197514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff890004020 con 0x7ff8a0102f00 2024-04-23T18:47:49.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a019b2e0 con 0x7ff8a0102f00 2024-04-23T18:47:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff89d7fa640 1 -- 172.21.15.33:0/2287197514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900047a0 con 0x7ff8a0102f00 2024-04-23T18:47:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.375+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a01a75f0 con 0x7ff8a0102f00 2024-04-23T18:47:50.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.377+0000 7ff89d7fa640 1 -- 172.21.15.33:0/2287197514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff890009a30 con 0x7ff8a0102f00 2024-04-23T18:47:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.377+0000 7ff8a93b9740 1 Processor -- start 2024-04-23T18:47:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.377+0000 7ff8a93b9740 1 -- start start 2024-04-23T18:47:50.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:50.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7f40 0x5642f37c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:50.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:50.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642f371c970 con 0x5642f37b7620 2024-04-23T18:47:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5642f36b94a0 con 0x5642f37b7f40 2024-04-23T18:47:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5642f3718930 con 0x5642f37c2800 2024-04-23T18:47:50.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59200/0 (socket says 172.21.15.33:59200) 2024-04-23T18:47:50.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 -- 172.21.15.33:0/3766323668 learned_addr learned my addr 172.21.15.33:0/3766323668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff89ffff640 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7f40 0x5642f37c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff89d7fa640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 0x7ff88c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 -- 172.21.15.33:0/3766323668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 msgr2=0x5642f37c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:50.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 -- 172.21.15.33:0/3766323668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7f40 msgr2=0x5642f37c22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7f40 0x5642f37c22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 -- 172.21.15.33:0/3766323668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f342f210 con 0x5642f37b7620 2024-04-23T18:47:50.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 0x7ff88c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:50.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff89d7fa640 1 -- 172.21.15.33:0/2287197514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7ff8900d0330 con 0x7ff8a0102f00 2024-04-23T18:47:50.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.378+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff898010440 tx=0x7ff898010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4be05fe1c04a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8967fc640 1 -- 172.21.15.33:0/3766323668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89801d400 con 0x5642f37b7620 2024-04-23T18:47:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8967fc640 1 -- 172.21.15.33:0/3766323668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89801da50 con 0x5642f37b7620 2024-04-23T18:47:50.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8967fc640 1 -- 172.21.15.33:0/3766323668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89801c760 con 0x5642f37b7620 2024-04-23T18:47:50.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 -- 172.21.15.33:0/3766323668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 msgr2=0x5642f37b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff898010440 tx=0x7ff898010900 comp rx=0 tx=0).stop 2024-04-23T18:47:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 0x7ff88c0a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff898021000 tx=0x7ff89801e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 -- 172.21.15.33:0/3766323668 shutdown_connections 2024-04-23T18:47:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7f40 0x5642f37c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:51.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/3766323668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7620 0x5642f37b7a00 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:51.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 -- 172.21.15.33:0/3766323668 >> 172.21.15.33:0/3766323668 conn(0x5642f37b3b30 msgr2=0x5642f37b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 -- 172.21.15.33:0/3766323668 shutdown_connections 2024-04-23T18:47:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.379+0000 7ff8a93b9740 1 -- 172.21.15.33:0/3766323668 wait complete. 2024-04-23T18:47:51.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 -- start start 2024-04-23T18:47:51.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 Processor -- start 2024-04-23T18:47:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7620 0x5642f37c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37c2800 0x5642f37bf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642f37c5430 con 0x5642f37b7f40 2024-04-23T18:47:51.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5642f36b70b0 con 0x5642f37c2800 2024-04-23T18:47:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5642f37c55b0 con 0x5642f37b7620 2024-04-23T18:47:51.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff8a93b9740 10 client.0 _ll_register_callbacks cb 0x5642f36a4d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:51.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59216/0 (socket says 172.21.15.33:59216) 2024-04-23T18:47:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2349059637 learned_addr learned my addr 172.21.15.33:0/2349059637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.380+0000 7ff89ffff640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7620 0x5642f37c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37c2800 0x5642f37bf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2349059637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7620 msgr2=0x5642f37c2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7620 0x5642f37c2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:52.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37c2800 msgr2=0x5642f37bf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:52.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37c2800 0x5642f37bf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 -- 172.21.15.33:0/2349059637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f34fb620 con 0x5642f37b7f40 2024-04-23T18:47:52.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff89009c1f0 tx=0x7ff89009bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcbd83b7896b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:52.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff87f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8900a4ed0 con 0x5642f37b7f40 2024-04-23T18:47:52.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89801da50 con 0x5642f37b7f40 2024-04-23T18:47:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7ff8900041d0 con 0x5642f37b7f40 2024-04-23T18:47:52.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff8900988d0 con 0x5642f37b7f40 2024-04-23T18:47:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff8a93b9740 10 client.17694 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5642f37d13a0 con 0x5642f37b7f40 2024-04-23T18:47:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.381+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89801da50 con 0x5642f37b7f40 2024-04-23T18:47:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7ff8900a0150 con 0x5642f37b7f40 2024-04-23T18:47:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 10 client.17694 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:52.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 20 client.17694 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f37d2860 0x5642f37d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 4 client.17694 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:47:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642f372a390 con 0x5642f37d2860 2024-04-23T18:47:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff89ffff640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f37d2860 0x5642f37d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff87f7fe640 10 client.17694 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:52.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.382+0000 7ff89ffff640 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f37d2860 0x5642f37d4c50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f538b83b68af815f server_cookie=d9286428f0f15e17 in_seq=0 out_seq=0 2024-04-23T18:47:53.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff87f7fe640 1 -- 172.21.15.33:0/2349059637 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642f372a390 con 0x5642f37d2860 2024-04-23T18:47:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff87f7fe640 10 client.17694 handle_command_reply: tid=0 2024-04-23T18:47:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 client.17694 shutdown 2024-04-23T18:47:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 msgr2=0x5642f37bf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff89009c1f0 tx=0x7ff89009bcf0 comp rx=0 tx=0).stop 2024-04-23T18:47:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 shutdown_connections 2024-04-23T18:47:53.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f37d2860 0x5642f37d4c50 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37c2800 0x5642f37bf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:53.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bf430 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff87f7fe640 0 client.17694 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/2349059637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7620 0x5642f37c2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 >> 172.21.15.33:0/2349059637 conn(0x5642f37b3b30 msgr2=0x5642f37b98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:53.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 shutdown_connections 2024-04-23T18:47:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.383+0000 7ff8a93b9740 1 -- 172.21.15.33:0/2349059637 wait complete. 2024-04-23T18:47:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.384+0000 7ff8a93b9740 20 client.17694 trim_cache size 0 max 16384 2024-04-23T18:47:53.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.398+0000 7ff8a93b9740 1 Processor -- start 2024-04-23T18:47:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 -- start start 2024-04-23T18:47:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7f40 0x5642f37b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b17c0 con 0x5642f37c2800 2024-04-23T18:47:53.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01af5b0 con 0x5642f37b7620 2024-04-23T18:47:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b10f0 con 0x5642f37b7f40 2024-04-23T18:47:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59218/0 (socket says 172.21.15.33:59218) 2024-04-23T18:47:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 -- 172.21.15.33:0/1296901530 learned_addr learned my addr 172.21.15.33:0/1296901530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff89ffff640 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:54.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 -- 172.21.15.33:0/1296901530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7f40 msgr2=0x5642f37b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7f40 0x5642f37b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 -- 172.21.15.33:0/1296901530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 msgr2=0x5642f37ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 -- 172.21.15.33:0/1296901530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f37bd9c0 con 0x5642f37c2800 2024-04-23T18:47:54.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff898010440 tx=0x7ff898028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15014817ba437623 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff895ffb640 1 -- 172.21.15.33:0/1296901530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898035020 con 0x5642f37c2800 2024-04-23T18:47:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.399+0000 7ff895ffb640 1 -- 172.21.15.33:0/1296901530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89802b070 con 0x5642f37c2800 2024-04-23T18:47:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff895ffb640 1 -- 172.21.15.33:0/1296901530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898003bd0 con 0x5642f37c2800 2024-04-23T18:47:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- 172.21.15.33:0/1296901530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 msgr2=0x5642f37c1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff898010440 tx=0x7ff898028000 comp rx=0 tx=0).stop 2024-04-23T18:47:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- 172.21.15.33:0/1296901530 shutdown_connections 2024-04-23T18:47:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37c2800 0x5642f37c1c70 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37b7f40 0x5642f37b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/1296901530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:54.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- 172.21.15.33:0/1296901530 >> 172.21.15.33:0/1296901530 conn(0x5642f37b3b30 msgr2=0x5642f37b9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- 172.21.15.33:0/1296901530 shutdown_connections 2024-04-23T18:47:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- 172.21.15.33:0/1296901530 wait complete. 2024-04-23T18:47:54.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.400+0000 7ff8a93b9740 1 -- start start 2024-04-23T18:47:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 Processor -- start 2024-04-23T18:47:54.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:54.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b0350 con 0x5642f37b7f40 2024-04-23T18:47:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01afc80 con 0x5642f37b7620 2024-04-23T18:47:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a01b0a20 con 0x5642f37c2800 2024-04-23T18:47:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a93b9740 10 client.0 _ll_register_callbacks cb 0x7ff89009be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:47:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:55.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59234/0 (socket says 172.21.15.33:59234) 2024-04-23T18:47:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 -- 172.21.15.33:0/734883812 learned_addr learned my addr 172.21.15.33:0/734883812 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:47:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89ffff640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff8a4b63640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:55.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 -- 172.21.15.33:0/734883812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 msgr2=0x5642f37bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:55.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37bc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:55.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 -- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 msgr2=0x5642f37cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:47:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 -- 172.21.15.33:0/734883812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f37bcca0 con 0x5642f37b7f40 2024-04-23T18:47:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89ffff640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:47:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff89f7fe640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff890098c30 tx=0x7ff89009bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8f84a6e6608d85 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:47:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.401+0000 7ff87f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:47:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890098a70 con 0x5642f37b7f40 2024-04-23T18:47:55.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 8542+0+0 (secure 0 0 0) 0x7ff8900041d0 con 0x5642f37b7f40 2024-04-23T18:47:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89802b070 con 0x5642f37b7f40 2024-04-23T18:47:55.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff89006a020 con 0x5642f37b7f40 2024-04-23T18:47:55.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 10 client.17700 fetch_fsmap learned FSMap version 39 2024-04-23T18:47:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5642f3838df0 con 0x5642f37b7f40 2024-04-23T18:47:55.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff89802b070 con 0x5642f37b7f40 2024-04-23T18:47:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) ==== 7749+0+0 (secure 0 0 0) 0x7ff8900cdc90 con 0x5642f37b7f40 2024-04-23T18:47:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 10 client.17700 fetch_fsmap finished waiting for FSMap version 39 2024-04-23T18:47:55.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 10 client.17700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:47:55.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 20 client.17700 populate_metadata read hostname 'smithi033' 2024-04-23T18:47:56.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f3839510 0x5642f383b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:47:56.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 4 client.17700 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:47:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.402+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642f37bfd00 con 0x5642f3839510 2024-04-23T18:47:56.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.403+0000 7ff89ffff640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f3839510 0x5642f383b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:47:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.403+0000 7ff87f7fe640 10 client.17700 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:01.403+0000 7ff89ffff640 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f3839510 0x5642f383b930 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8a1478006388949 server_cookie=7783bb1bf4d52cee in_seq=0 out_seq=0 2024-04-23T18:47:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff87f7fe640 1 -- 172.21.15.33:0/734883812 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642f37bfd00 con 0x5642f3839510 2024-04-23T18:47:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff87f7fe640 10 client.17700 handle_command_reply: tid=0 2024-04-23T18:47:56.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 client.17700 shutdown 2024-04-23T18:47:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 msgr2=0x5642f37bbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff890098c30 tx=0x7ff89009bbc0 comp rx=0 tx=0).stop 2024-04-23T18:47:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 shutdown_connections 2024-04-23T18:47:56.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5642f3839510 0x5642f383b930 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:56.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642f37c2800 0x5642f37bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642f37b7f40 0x5642f37bbef0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff87f7fe640 0 client.17700 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:47:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 --2- 172.21.15.33:0/734883812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642f37b7620 0x5642f37cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 >> 172.21.15.33:0/734883812 conn(0x5642f37b3b30 msgr2=0x5642f37b2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 shutdown_connections 2024-04-23T18:47:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 1 -- 172.21.15.33:0/734883812 wait complete. 2024-04-23T18:47:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.188+0000 7ff8a93b9740 20 client.17700 trim_cache size 0 max 16384 2024-04-23T18:47:56.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:47:56.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:47:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:47:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:47:57.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 msgr2=0x7ff88c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 0x7ff88c0a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff898021000 tx=0x7ff89801e000 comp rx=0 tx=0).stop 2024-04-23T18:47:57.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 msgr2=0x7ff8a01a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:47:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff89000d950 tx=0x7ff89000de10 comp rx=0 tx=0).stop 2024-04-23T18:47:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 shutdown_connections 2024-04-23T18:47:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff88c0a3dc0 0x7ff88c0a6280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8a010e8a0 0x7ff8a019a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:57.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8a0102f00 0x7ff8a01a0bb0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:57.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 --2- 172.21.15.33:0/2287197514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a006acc0 0x7ff8a01a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:47:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.189+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 >> 172.21.15.33:0/2287197514 conn(0x7ff8a0076c50 msgr2=0x7ff8a010f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:47:57.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.190+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 shutdown_connections 2024-04-23T18:47:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:03.190+0000 7ff8a65c6640 1 -- 172.21.15.33:0/2287197514 wait complete. 2024-04-23T18:47:57.532 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:02 smithi033 ceph-mon[29274]: pgmap v622: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-23T18:47:57.608 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:49 smithi122 ceph-mon[32546]: pgmap v681: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:57.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:51 smithi122 ceph-mon[32546]: pgmap v682: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:57.627 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:54 smithi122 ceph-mon[32546]: pgmap v683: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:57.634 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:56 smithi122 ceph-mon[32546]: pgmap v684: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:57.643 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:57 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:47:57.742 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:49 smithi106 ceph-mon[32716]: pgmap v681: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:57.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:51 smithi106 ceph-mon[32716]: pgmap v682: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:57.958 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:54 smithi106 ceph-mon[32716]: pgmap v683: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:47:58.009 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:56 smithi106 ceph-mon[32716]: pgmap v684: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:47:58.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:57 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:47:58.026 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:47:58.056 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: in thread 7f9b55e3e700 thread_name:md_log_replay 2024-04-23T18:47:58.106 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:47:58.114 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9b671ffd20] 2024-04-23T18:47:58.122 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56309cbcd38f] 2024-04-23T18:47:58.173 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56309cbd0b23] 2024-04-23T18:47:58.181 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 4: (MDLog::_replay_thread()+0x617) [0x56309cc2b647] 2024-04-23T18:47:58.190 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56309cc34d31] 2024-04-23T18:47:58.198 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f9b671f51ca] 2024-04-23T18:47:58.207 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 7: clone() 2024-04-23T18:47:58.285 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:46:04.132+0000 7f9b55e3e700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:47:58.294 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: in thread 7f9b55e3e700 thread_name:md_log_replay 2024-04-23T18:47:58.302 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:58.311 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:47:58.368 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9b671ffd20] 2024-04-23T18:47:58.397 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56309cbcd38f] 2024-04-23T18:47:58.404 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56309cbd0b23] 2024-04-23T18:47:58.451 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 4: (MDLog::_replay_thread()+0x617) [0x56309cc2b647] 2024-04-23T18:47:58.459 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56309cc34d31] 2024-04-23T18:47:58.540 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f9b671f51ca] 2024-04-23T18:47:58.548 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 7: clone() 2024-04-23T18:47:58.616 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:47:58.624 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:58.632 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 0> 2024-04-23T18:46:04.132+0000 7f9b55e3e700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:47:58.640 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: in thread 7f9b55e3e700 thread_name:md_log_replay 2024-04-23T18:47:58.649 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:58.711 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:47:58.774 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9b671ffd20] 2024-04-23T18:47:58.783 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56309cbcd38f] 2024-04-23T18:47:58.829 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56309cbd0b23] 2024-04-23T18:47:58.868 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 4: (MDLog::_replay_thread()+0x617) [0x56309cc2b647] 2024-04-23T18:47:58.929 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56309cc34d31] 2024-04-23T18:47:58.962 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f9b671f51ca] 2024-04-23T18:47:58.971 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 7: clone() 2024-04-23T18:47:59.061 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:47:59.102 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:59.152 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 0> 2024-04-23T18:46:04.132+0000 7f9b55e3e700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:47:59.207 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: in thread 7f9b55e3e700 thread_name:md_log_replay 2024-04-23T18:47:59.215 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:59.225 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:47:59.234 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9b671ffd20] 2024-04-23T18:47:59.242 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2: /usr/bin/ceph-mds(+0x54938f) [0x56309cbcd38f] 2024-04-23T18:47:59.336 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x56309cbd0b23] 2024-04-23T18:47:59.346 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 4: (MDLog::_replay_thread()+0x617) [0x56309cc2b647] 2024-04-23T18:47:59.355 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 5: (MDLog::ReplayThread::entry()+0x11) [0x56309cc34d31] 2024-04-23T18:47:59.363 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f9b671f51ca] 2024-04-23T18:47:59.446 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 7: clone() 2024-04-23T18:47:59.525 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:47:59.623 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[93448]: 2024-04-23T18:47:59.702 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 podman[94214]: 2024-04-23 18:46:04.189216957 +0000 UTC m=+0.023007286 container died eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:47:59.709 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 podman[94214]: 2024-04-23 18:46:04.732272773 +0000 UTC m=+0.566063114 container remove eb3e69313b246ecca4e8607c2553718017aa823fd7bb83c673db26cbc8b564b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T18:47:59.768 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:47:59.795 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-mon[29274]: pgmap v623: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:47:59.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:04 smithi033 ceph-mon[29274]: osdmap e122: 12 total, 12 up, 12 in 2024-04-23T18:47:59.949 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:47:59.989 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: pgmap v685: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:00.051 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.114 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.123 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:48:00.188 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:48:00.195 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.203 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:59 smithi122 ceph-mon[32546]: pgmap v686: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 270 B/s wr, 9 op/s 2024-04-23T18:48:00.211 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:47:59 smithi122 ceph-mon[32546]: pgmap v687: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:48:00.220 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: pgmap v685: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:00.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.423 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.436 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:48:00.454 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:48:00.549 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:59 smithi106 ceph-mon[32716]: pgmap v686: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 270 B/s wr, 9 op/s 2024-04-23T18:48:00.650 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:47:59 smithi106 ceph-mon[32716]: pgmap v687: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:48:00.656 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:06 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:00.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:07 smithi033 ceph-mon[29274]: pgmap v625: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:48:00.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:09 smithi033 ceph-mon[29274]: pgmap v626: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:48:00.773 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:11 smithi033 ceph-mon[29274]: pgmap v627: 97 pgs: 97 active+clean; 15 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 62 KiB/s wr, 33 op/s 2024-04-23T18:48:00.780 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:11 smithi033 ceph-mon[29274]: osdmap e123: 12 total, 12 up, 12 in 2024-04-23T18:48:00.839 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:13 smithi033 ceph-mon[29274]: pgmap v629: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 78 KiB/s wr, 40 op/s 2024-04-23T18:48:00.845 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:15 smithi033 ceph-mon[29274]: pgmap v630: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-04-23T18:48:00.854 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 3. 2024-04-23T18:48:00.959 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:48:01.044 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:48:01.054 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 podman[94309]: 2024-04-23 18:46:15.382651412 +0000 UTC m=+0.019074675 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:48:01.120 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 podman[94309]: 2024-04-23 18:46:15.508127207 +0000 UTC m=+0.144550449 container create 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T18:48:01.192 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 podman[94309]: 2024-04-23 18:46:15.74458107 +0000 UTC m=+0.381004310 container init 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T18:48:01.201 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 podman[94309]: 2024-04-23 18:46:15.747466593 +0000 UTC m=+0.383889835 container start 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-23T18:48:01.210 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 ceph-mds[94329]: -- 172.21.15.33:0/2596364268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f17f1efa40 con 0x55f17f1c2c00 2024-04-23T18:48:01.321 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: starting mds.d at 2024-04-23T18:48:01.412 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 bash[94309]: 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe 2024-04-23T18:48:01.421 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:46:15 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:48:01.495 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:01 smithi122 ceph-mon[32546]: pgmap v688: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:48:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 -- 172.21.15.33:0/3999455135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34701010f0 msgr2=0x7f34701014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:01.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 --2- 172.21.15.33:0/3999455135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34701010f0 0x7f34701014d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f346c009900 tx=0x7f346c02a3f0 comp rx=0 tx=0).stop 2024-04-23T18:48:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 -- 172.21.15.33:0/3999455135 shutdown_connections 2024-04-23T18:48:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 --2- 172.21.15.33:0/3999455135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470106170 0x7f347010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 --2- 172.21.15.33:0/3999455135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3470101aa0 0x7f3470105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 --2- 172.21.15.33:0/3999455135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34701010f0 0x7f34701014d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 -- 172.21.15.33:0/3999455135 >> 172.21.15.33:0/3999455135 conn(0x7f3470077fd0 msgr2=0x7f34700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:01.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 -- 172.21.15.33:0/3999455135 shutdown_connections 2024-04-23T18:48:01.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.282+0000 7f347858a640 1 -- 172.21.15.33:0/3999455135 wait complete. 2024-04-23T18:48:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 Processor -- start 2024-04-23T18:48:01.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 -- start start 2024-04-23T18:48:01.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 0x7f34701972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:01.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:02.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470197d60 0x7f34701a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34701110b0 con 0x7f3470106170 2024-04-23T18:48:02.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3470110f30 con 0x7f3470197d60 2024-04-23T18:48:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f347858a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3470111230 con 0x7f34701010f0 2024-04-23T18:48:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:02.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59836/0 (socket says 172.21.15.33:59836) 2024-04-23T18:48:02.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 -- 172.21.15.33:0/3381420394 learned_addr learned my addr 172.21.15.33:0/3381420394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3476b27640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470197d60 0x7f34701a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3476326640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 0x7f34701972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 -- 172.21.15.33:0/3381420394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 msgr2=0x7f34701972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:02.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 0x7f34701972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 -- 172.21.15.33:0/3381420394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470197d60 msgr2=0x7f34701a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470197d60 0x7f34701a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 -- 172.21.15.33:0/3381420394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34701a6f20 con 0x7f3470106170 2024-04-23T18:48:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3476326640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 0x7f34701972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:02.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.283+0000 7f3475b25640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f346000c9a0 tx=0x7f346000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.284+0000 7f34677fe640 1 -- 172.21.15.33:0/3381420394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460014070 con 0x7f3470106170 2024-04-23T18:48:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.284+0000 7f34677fe640 1 -- 172.21.15.33:0/3381420394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3460004440 con 0x7f3470106170 2024-04-23T18:48:02.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.284+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34701a7210 con 0x7f3470106170 2024-04-23T18:48:02.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.284+0000 7f34677fe640 1 -- 172.21.15.33:0/3381420394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460003de0 con 0x7f3470106170 2024-04-23T18:48:02.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.284+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34701a77b0 con 0x7f3470106170 2024-04-23T18:48:02.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.286+0000 7f34677fe640 1 -- 172.21.15.33:0/3381420394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3460017070 con 0x7f3470106170 2024-04-23T18:48:02.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.286+0000 7f347b377740 1 Processor -- start 2024-04-23T18:48:02.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 -- start start 2024-04-23T18:48:02.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026620 0x55b6ef026a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef033bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6eef8b970 con 0x55b6ef026f40 2024-04-23T18:48:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b6eef284a0 con 0x55b6ef026620 2024-04-23T18:48:02.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f347b377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6eef87930 con 0x55b6ef031800 2024-04-23T18:48:03.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59844/0 (socket says 172.21.15.33:59844) 2024-04-23T18:48:03.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 -- 172.21.15.33:0/1186032421 learned_addr learned my addr 172.21.15.33:0/1186032421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:03.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f34677fe640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 0x7f345c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476b27640 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026620 0x55b6ef026a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:03.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3475b25640 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef033bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 -- 172.21.15.33:0/1186032421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 msgr2=0x55b6ef033bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:03.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef033bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:03.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 -- 172.21.15.33:0/1186032421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026620 msgr2=0x55b6ef026a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:03.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026620 0x55b6ef026a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:03.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476326640 1 -- 172.21.15.33:0/1186032421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6eec9e210 con 0x55b6ef026f40 2024-04-23T18:48:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f3476b27640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 0x7f345c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.287+0000 7f34677fe640 1 -- 172.21.15.33:0/3381420394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f34600d0270 con 0x7f3470106170 2024-04-23T18:48:03.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.288+0000 7f3476326640 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f346c000c00 tx=0x7f346c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a94fb2fcb9c43ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:03.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.288+0000 7f3464bfe640 1 -- 172.21.15.33:0/1186032421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c03f070 con 0x55b6ef026f40 2024-04-23T18:48:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.288+0000 7f3464bfe640 1 -- 172.21.15.33:0/1186032421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346c032e50 con 0x55b6ef026f40 2024-04-23T18:48:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.288+0000 7f3464bfe640 1 -- 172.21.15.33:0/1186032421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c035070 con 0x55b6ef026f40 2024-04-23T18:48:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- 172.21.15.33:0/1186032421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 msgr2=0x55b6ef0312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f3476b27640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 0x7f345c0a6300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3468005fd0 tx=0x7f3468014040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f346c000c00 tx=0x7f346c032c30 comp rx=0 tx=0).stop 2024-04-23T18:48:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- 172.21.15.33:0/1186032421 shutdown_connections 2024-04-23T18:48:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef033bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026f40 0x55b6ef0312c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:03.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- 172.21.15.33:0/1186032421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026620 0x55b6ef026a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- 172.21.15.33:0/1186032421 >> 172.21.15.33:0/1186032421 conn(0x55b6ef022b30 msgr2=0x55b6ef022f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:03.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- 172.21.15.33:0/1186032421 shutdown_connections 2024-04-23T18:48:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- 172.21.15.33:0/1186032421 wait complete. 2024-04-23T18:48:03.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- start start 2024-04-23T18:48:03.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 Processor -- start 2024-04-23T18:48:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:03.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef02e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef02e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6ef0343d0 con 0x55b6ef026620 2024-04-23T18:48:03.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b6eef260b0 con 0x55b6ef026f40 2024-04-23T18:48:04.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6ef034550 con 0x55b6ef031800 2024-04-23T18:48:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f347b377740 10 client.0 _ll_register_callbacks cb 0x55b6eef13d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:04.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.289+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:04.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59856/0 (socket says 172.21.15.33:59856) 2024-04-23T18:48:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 -- 172.21.15.33:0/2297438913 learned_addr learned my addr 172.21.15.33:0/2297438913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476b27640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef02e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3475b25640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef02e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 -- 172.21.15.33:0/2297438913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 msgr2=0x55b6ef02e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef02e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 -- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 msgr2=0x55b6ef02e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:04.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef02e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:04.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 -- 172.21.15.33:0/2297438913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6eed6a620 con 0x55b6ef026620 2024-04-23T18:48:04.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3476326640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f346c02a430 tx=0x7f346c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15919b45bec9560 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:04.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:04.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c03f070 con 0x55b6ef026620 2024-04-23T18:48:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f346c004250 con 0x55b6ef026620 2024-04-23T18:48:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346c032e50 con 0x55b6ef026620 2024-04-23T18:48:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f346c035040 con 0x55b6ef026620 2024-04-23T18:48:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f347b377740 10 client.17718 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.290+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b6ef040270 con 0x55b6ef026620 2024-04-23T18:48:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346c032e50 con 0x55b6ef026620 2024-04-23T18:48:05.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f346c003710 con 0x55b6ef026620 2024-04-23T18:48:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 10 client.17718 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 10 client.17718 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:05.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 20 client.17718 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0415c0 0x55b6ef0439b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 4 client.17718 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6eef99390 con 0x55b6ef0415c0 2024-04-23T18:48:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f3475b25640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0415c0 0x55b6ef0439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f3475b25640 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0415c0 0x55b6ef0439b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbd8e5bd8a70a69 server_cookie=8d28aa021cc286ea in_seq=0 out_seq=0 2024-04-23T18:48:05.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.291+0000 7f3459ffb640 10 client.17718 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:05.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f3459ffb640 1 -- 172.21.15.33:0/2297438913 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6eef99390 con 0x55b6ef0415c0 2024-04-23T18:48:05.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f3459ffb640 10 client.17718 handle_command_reply: tid=0 2024-04-23T18:48:05.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 client.17718 shutdown 2024-04-23T18:48:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 msgr2=0x55b6ef0310d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f346c02a430 tx=0x7f346c0314a0 comp rx=0 tx=0).stop 2024-04-23T18:48:05.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 shutdown_connections 2024-04-23T18:48:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0415c0 0x55b6ef0439b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef02e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef02e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f3459ffb640 0 client.17718 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:05.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 --2- 172.21.15.33:0/2297438913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef0310d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:05.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 >> 172.21.15.33:0/2297438913 conn(0x55b6ef022b30 msgr2=0x55b6ef0288b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 shutdown_connections 2024-04-23T18:48:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.292+0000 7f347b377740 1 -- 172.21.15.33:0/2297438913 wait complete. 2024-04-23T18:48:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.293+0000 7f347b377740 20 client.17718 trim_cache size 0 max 16384 2024-04-23T18:48:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 Processor -- start 2024-04-23T18:48:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 -- start start 2024-04-23T18:48:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef026620 0x55b6ef030c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef048fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:06.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34701b1830 con 0x55b6ef031800 2024-04-23T18:48:06.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34701af5f0 con 0x55b6ef026f40 2024-04-23T18:48:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.307+0000 7f347b377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34701b1160 con 0x55b6ef026620 2024-04-23T18:48:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:06.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59866/0 (socket says 172.21.15.33:59866) 2024-04-23T18:48:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 -- 172.21.15.33:0/1300566710 learned_addr learned my addr 172.21.15.33:0/1300566710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:06.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3475b25640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef048fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476326640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef026620 0x55b6ef030c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 -- 172.21.15.33:0/1300566710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef026620 msgr2=0x55b6ef030c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef026620 0x55b6ef030c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:06.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 -- 172.21.15.33:0/1300566710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 msgr2=0x55b6ef048fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef048fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 -- 172.21.15.33:0/1300566710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ef02d370 con 0x55b6ef031800 2024-04-23T18:48:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3475b25640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef048fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:48:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f3476b27640 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3468004960 tx=0x7f3468017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbd8af96301cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f345bfff640 1 -- 172.21.15.33:0/1300566710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468010d30 con 0x55b6ef031800 2024-04-23T18:48:06.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f345bfff640 1 -- 172.21.15.33:0/1300566710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346801c930 con 0x55b6ef031800 2024-04-23T18:48:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.308+0000 7f345bfff640 1 -- 172.21.15.33:0/1300566710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346801d950 con 0x55b6ef031800 2024-04-23T18:48:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- 172.21.15.33:0/1300566710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 msgr2=0x55b6ef049520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3468004960 tx=0x7f3468017000 comp rx=0 tx=0).stop 2024-04-23T18:48:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- 172.21.15.33:0/1300566710 shutdown_connections 2024-04-23T18:48:06.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef031800 0x55b6ef049520 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef048fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 --2- 172.21.15.33:0/1300566710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef026620 0x55b6ef030c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- 172.21.15.33:0/1300566710 >> 172.21.15.33:0/1300566710 conn(0x55b6ef022b30 msgr2=0x55b6ef02e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- 172.21.15.33:0/1300566710 shutdown_connections 2024-04-23T18:48:07.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- 172.21.15.33:0/1300566710 wait complete. 2024-04-23T18:48:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 -- start start 2024-04-23T18:48:07.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.309+0000 7f347b377740 1 Processor -- start 2024-04-23T18:48:07.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:07.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef0a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef036a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:07.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34701b03c0 con 0x55b6ef026620 2024-04-23T18:48:07.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f34701afcc0 con 0x55b6ef026f40 2024-04-23T18:48:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34701b0a90 con 0x55b6ef031800 2024-04-23T18:48:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f347b377740 10 client.0 _ll_register_callbacks cb 0x7f346c0053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59882/0 (socket says 172.21.15.33:59882) 2024-04-23T18:48:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 -- 172.21.15.33:0/2411227184 learned_addr learned my addr 172.21.15.33:0/2411227184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476b27640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef036a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:07.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3475b25640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef0a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 -- 172.21.15.33:0/2411227184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 msgr2=0x55b6ef036a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef036a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 -- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 msgr2=0x55b6ef0a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef0a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 -- 172.21.15.33:0/2411227184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ef037100 con 0x55b6ef026620 2024-04-23T18:48:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3476326640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f346c0163c0 tx=0x7f346c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b8e7a09bfb7365 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:07.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3459ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.310+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c05b030 con 0x55b6ef026620 2024-04-23T18:48:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f346c0048d0 con 0x55b6ef026620 2024-04-23T18:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346801c930 con 0x55b6ef026620 2024-04-23T18:48:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f346c065050 con 0x55b6ef026620 2024-04-23T18:48:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 10 client.17724 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b6ef0a7380 con 0x55b6ef026620 2024-04-23T18:48:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f346801c930 con 0x55b6ef026620 2024-04-23T18:48:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f346c003cc0 con 0x55b6ef026620 2024-04-23T18:48:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 10 client.17724 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 10 client.17724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 20 client.17724 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0a7a60 0x55b6ef0a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 4 client.17724 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:48:08.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b6eef93200 con 0x55b6ef0a7a60 2024-04-23T18:48:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.311+0000 7f3475b25640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0a7a60 0x55b6ef0a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.312+0000 7f3475b25640 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0a7a60 0x55b6ef0a9e80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27209b55feea944b server_cookie=be4b5ee9935dfe07 in_seq=0 out_seq=0 2024-04-23T18:48:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.312+0000 7f3459ffb640 10 client.17724 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:08.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f3459ffb640 1 -- 172.21.15.33:0/2411227184 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b6eef93200 con 0x55b6ef0a7a60 2024-04-23T18:48:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f3459ffb640 10 client.17724 handle_command_reply: tid=0 2024-04-23T18:48:08.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 client.17724 shutdown 2024-04-23T18:48:08.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 msgr2=0x55b6ef09fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f346c0163c0 tx=0x7f346c001e70 comp rx=0 tx=0).stop 2024-04-23T18:48:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 shutdown_connections 2024-04-23T18:48:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b6ef0a7a60 0x55b6ef0a9e80 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ef031800 0x55b6ef036a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b6ef026f40 0x55b6ef0a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 --2- 172.21.15.33:0/2411227184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6ef026620 0x55b6ef09fbd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 >> 172.21.15.33:0/2411227184 conn(0x55b6ef022b30 msgr2=0x55b6ef021420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:09.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.322+0000 7f3459ffb640 0 client.17724 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 shutdown_connections 2024-04-23T18:48:09.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347b377740 1 -- 172.21.15.33:0/2411227184 wait complete. 2024-04-23T18:48:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347b377740 20 client.17724 trim_cache size 0 max 16384 2024-04-23T18:48:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:48:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:48:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:48:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:48:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 msgr2=0x7f345c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 0x7f345c0a6300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3468005fd0 tx=0x7f3468014040 comp rx=0 tx=0).stop 2024-04-23T18:48:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 msgr2=0x7f3470197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:09.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f346000c9a0 tx=0x7f346000ce60 comp rx=0 tx=0).stop 2024-04-23T18:48:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 shutdown_connections 2024-04-23T18:48:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f345c0a3e40 0x7f345c0a6300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3470197d60 0x7f34701a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:10.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3470106170 0x7f3470197820 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 --2- 172.21.15.33:0/3381420394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34701010f0 0x7f34701972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.323+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 >> 172.21.15.33:0/3381420394 conn(0x7f3470077fd0 msgr2=0x7f34700fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:10.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.324+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 shutdown_connections 2024-04-23T18:48:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:16.324+0000 7f347858a640 1 -- 172.21.15.33:0/3381420394 wait complete. 2024-04-23T18:48:10.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:16 smithi033 ceph-mon[29274]: daemon mds.d restarted 2024-04-23T18:48:10.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:16 smithi033 ceph-mon[29274]: osdmap e124: 12 total, 12 up, 12 in 2024-04-23T18:48:10.259 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/1458865300,v1:172.21.15.33:6837/1458865300] up:boot 2024-04-23T18:48:10.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:10.322 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:48:10.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:17 smithi033 ceph-mon[29274]: pgmap v631: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-23T18:48:10.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:19 smithi033 ceph-mon[29274]: pgmap v633: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-04-23T18:48:10.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:21 smithi033 ceph-mon[29274]: pgmap v634: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.7 KiB/s wr, 4 op/s 2024-04-23T18:48:10.695 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:23 smithi033 ceph-mon[29274]: pgmap v635: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:48:10.702 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:25 smithi033 ceph-mon[29274]: pgmap v636: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 4.3 KiB/s wr, 12 op/s 2024-04-23T18:48:10.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:27 smithi033 ceph-mon[29274]: pgmap v637: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-23T18:48:10.796 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:03 smithi122 ceph-mon[32546]: pgmap v689: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:48:10.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:05 smithi122 ceph-mon[32546]: pgmap v690: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:48:10.817 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:05 smithi122 ceph-mon[32546]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:48:10.838 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:07 smithi122 ceph-mon[32546]: pgmap v692: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:48:10.895 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:09 smithi122 ceph-mon[32546]: pgmap v693: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:48:10.905 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:01 smithi106 ceph-mon[32716]: pgmap v688: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:48:10.914 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:03 smithi106 ceph-mon[32716]: pgmap v689: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:48:10.938 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:05 smithi106 ceph-mon[32716]: pgmap v690: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:48:11.058 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:05 smithi106 ceph-mon[32716]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:48:11.067 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:07 smithi106 ceph-mon[32716]: pgmap v692: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:48:11.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:09 smithi106 ceph-mon[32716]: pgmap v693: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:48:11.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 -- 172.21.15.33:0/3932139596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8470101a70 msgr2=0x7f8470105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:11.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 --2- 172.21.15.33:0/3932139596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8470101a70 0x7f8470105ab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402a410 comp rx=0 tx=0).stop 2024-04-23T18:48:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 -- 172.21.15.33:0/3932139596 shutdown_connections 2024-04-23T18:48:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 --2- 172.21.15.33:0/3932139596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8470106190 0x7f847010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 --2- 172.21.15.33:0/3932139596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8470101a70 0x7f8470105ab0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:11.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 --2- 172.21.15.33:0/3932139596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84701010c0 0x7f84701014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:11.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.423+0000 7f8476976640 1 -- 172.21.15.33:0/3932139596 >> 172.21.15.33:0/3932139596 conn(0x7f8470077fd0 msgr2=0x7f84700783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:11.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.424+0000 7f8476976640 1 -- 172.21.15.33:0/3932139596 shutdown_connections 2024-04-23T18:48:11.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.424+0000 7f8476976640 1 -- 172.21.15.33:0/3932139596 wait complete. 2024-04-23T18:48:11.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.424+0000 7f8476976640 1 Processor -- start 2024-04-23T18:48:11.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.424+0000 7f8476976640 1 -- start start 2024-04-23T18:48:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84701010c0 0x7f8470102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8470106190 0x7f8470103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84701110d0 con 0x7f84701035d0 2024-04-23T18:48:12.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8470110f50 con 0x7f8470106190 2024-04-23T18:48:12.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8476976640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8470111250 con 0x7f84701010c0 2024-04-23T18:48:12.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52192/0 (socket says 172.21.15.33:52192) 2024-04-23T18:48:12.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 -- 172.21.15.33:0/1708053878 learned_addr learned my addr 172.21.15.33:0/1708053878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:12.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8470106190 0x7f8470103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f846ffff640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84701010c0 0x7f8470102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 -- 172.21.15.33:0/1708053878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84701010c0 msgr2=0x7f8470102b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84701010c0 0x7f8470102b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 -- 172.21.15.33:0/1708053878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8470106190 msgr2=0x7f8470103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:12.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8470106190 0x7f8470103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:12.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 -- 172.21.15.33:0/1708053878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84701a6f80 con 0x7f84701035d0 2024-04-23T18:48:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.425+0000 7f8474f13640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f846800eaa0 tx=0x7f846800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.426+0000 7f846d7fa640 1 -- 172.21.15.33:0/1708053878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846800cdb0 con 0x7f84701035d0 2024-04-23T18:48:12.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.426+0000 7f846d7fa640 1 -- 172.21.15.33:0/1708053878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8468005020 con 0x7f84701035d0 2024-04-23T18:48:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.426+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84701a7270 con 0x7f84701035d0 2024-04-23T18:48:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.426+0000 7f846d7fa640 1 -- 172.21.15.33:0/1708053878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680116d0 con 0x7f84701035d0 2024-04-23T18:48:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.426+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84701a77e0 con 0x7f84701035d0 2024-04-23T18:48:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f846d7fa640 1 -- 172.21.15.33:0/1708053878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84680051d0 con 0x7f84701035d0 2024-04-23T18:48:12.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 Processor -- start 2024-04-23T18:48:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 -- start start 2024-04-23T18:48:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc511dfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:13.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511ea800 0x55bc511ecbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc51144970 con 0x55bc511dff40 2024-04-23T18:48:13.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bc510e14a0 con 0x55bc511ea800 2024-04-23T18:48:13.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.428+0000 7f847976a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc51140930 con 0x55bc511df620 2024-04-23T18:48:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:13.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846d7fa640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 0x7f84580a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52196/0 (socket says 172.21.15.33:52196) 2024-04-23T18:48:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 -- 172.21.15.33:0/365131863 learned_addr learned my addr 172.21.15.33:0/365131863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846f7fe640 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc511dfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846f7fe640 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511ea800 0x55bc511ecbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:13.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f8474f13640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 0x7f84580a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:13.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 -- 172.21.15.33:0/365131863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 msgr2=0x55bc511dfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc511dfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 -- 172.21.15.33:0/365131863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511ea800 msgr2=0x55bc511ecbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511ea800 0x55bc511ecbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 -- 172.21.15.33:0/365131863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc50e57210 con 0x55bc511dff40 2024-04-23T18:48:13.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846d7fa640 1 -- 172.21.15.33:0/1708053878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f84680d07b0 con 0x7f84701035d0 2024-04-23T18:48:13.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f846ffff640 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f846000ac20 tx=0x7f846000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ca60cda80b76a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:13.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f8474f13640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 0x7f84580a6330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8468098c70 tx=0x7f8468098410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:13.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.429+0000 7f84567fc640 1 -- 172.21.15.33:0/365131863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846001d070 con 0x55bc511dff40 2024-04-23T18:48:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f84567fc640 1 -- 172.21.15.33:0/365131863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460018af0 con 0x55bc511dff40 2024-04-23T18:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f84567fc640 1 -- 172.21.15.33:0/365131863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460010ad0 con 0x55bc511dff40 2024-04-23T18:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 -- 172.21.15.33:0/365131863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 msgr2=0x55bc511ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:13.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f846000ac20 tx=0x7f846000f790 comp rx=0 tx=0).stop 2024-04-23T18:48:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 -- 172.21.15.33:0/365131863 shutdown_connections 2024-04-23T18:48:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511ea800 0x55bc511ecbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:13.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511dff40 0x55bc511ea2c0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:13.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 --2- 172.21.15.33:0/365131863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc511dfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 -- 172.21.15.33:0/365131863 >> 172.21.15.33:0/365131863 conn(0x55bc511dbb30 msgr2=0x55bc511dbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 -- 172.21.15.33:0/365131863 shutdown_connections 2024-04-23T18:48:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.430+0000 7f847976a740 1 -- 172.21.15.33:0/365131863 wait complete. 2024-04-23T18:48:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 -- start start 2024-04-23T18:48:14.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 Processor -- start 2024-04-23T18:48:14.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511df620 0x55bc511ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 0x55bc511e7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:14.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc511ed430 con 0x55bc511ea800 2024-04-23T18:48:14.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bc510df0b0 con 0x55bc511df620 2024-04-23T18:48:14.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc511ed5b0 con 0x55bc511dff40 2024-04-23T18:48:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f847976a740 10 client.0 _ll_register_callbacks cb 0x55bc510ccd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:14.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52204/0 (socket says 172.21.15.33:52204) 2024-04-23T18:48:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f846f7fe640 1 -- 172.21.15.33:0/1019146539 learned_addr learned my addr 172.21.15.33:0/1019146539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.431+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511df620 0x55bc511ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:14.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846ffff640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 0x55bc511e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:14.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 -- 172.21.15.33:0/1019146539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 msgr2=0x55bc511e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 0x55bc511e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:14.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 -- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511df620 msgr2=0x55bc511ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511df620 0x55bc511ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:15.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 -- 172.21.15.33:0/1019146539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc50f23620 con 0x55bc511ea800 2024-04-23T18:48:15.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846ffff640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 0x55bc511e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:15.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f8464009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de90c119e8ab944d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:15.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f844f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035040 con 0x55bc511ea800 2024-04-23T18:48:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f8464004450 con 0x55bc511ea800 2024-04-23T18:48:15.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8460018af0 con 0x55bc511ea800 2024-04-23T18:48:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f84640035f0 con 0x55bc511ea800 2024-04-23T18:48:15.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f847976a740 10 client.17736 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bc511f9420 con 0x55bc511ea800 2024-04-23T18:48:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.432+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460018af0 con 0x55bc511ea800 2024-04-23T18:48:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f84640548f0 con 0x55bc511ea800 2024-04-23T18:48:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 10 client.17736 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 10 client.17736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 20 client.17736 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc511fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:15.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 4 client.17736 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:15.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc51152390 con 0x55bc511fa890 2024-04-23T18:48:15.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f846ffff640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc511fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:15.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.433+0000 7f846ffff640 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc511fcca0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3244f66bd4ce86 server_cookie=1d96f7aa4e6036ca in_seq=0 out_seq=0 2024-04-23T18:48:16.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f844f7fe640 10 client.17736 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f844f7fe640 1 -- 172.21.15.33:0/1019146539 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc51152390 con 0x55bc511fa890 2024-04-23T18:48:16.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f844f7fe640 10 client.17736 handle_command_reply: tid=0 2024-04-23T18:48:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f847976a740 1 client.17736 shutdown 2024-04-23T18:48:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 msgr2=0x55bc511e7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:16.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f8464009260 comp rx=0 tx=0).stop 2024-04-23T18:48:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 shutdown_connections 2024-04-23T18:48:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.434+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc511fcca0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:16.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511e7b10 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511dff40 0x55bc511e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:16.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f844f7fe640 0 client.17736 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:16.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 --2- 172.21.15.33:0/1019146539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511df620 0x55bc511ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 >> 172.21.15.33:0/1019146539 conn(0x55bc511dbb30 msgr2=0x55bc511e1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 shutdown_connections 2024-04-23T18:48:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 1 -- 172.21.15.33:0/1019146539 wait complete. 2024-04-23T18:48:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.435+0000 7f847976a740 20 client.17736 trim_cache size 0 max 16384 2024-04-23T18:48:16.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 Processor -- start 2024-04-23T18:48:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 -- start start 2024-04-23T18:48:16.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc512025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511ea800 0x55bc51202af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:17.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b18e0 con 0x55bc511df620 2024-04-23T18:48:17.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84701af6a0 con 0x55bc511dff40 2024-04-23T18:48:17.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f847976a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b1210 con 0x55bc511ea800 2024-04-23T18:48:17.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:17.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52210/0 (socket says 172.21.15.33:52210) 2024-04-23T18:48:17.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 -- 172.21.15.33:0/2549109025 learned_addr learned my addr 172.21.15.33:0/2549109025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:17.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846ffff640 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc512025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 -- 172.21.15.33:0/2549109025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511ea800 msgr2=0x55bc51202af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:17.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511ea800 0x55bc51202af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:17.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 -- 172.21.15.33:0/2549109025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 msgr2=0x55bc512025b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:17.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc512025b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 -- 172.21.15.33:0/2549109025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc511e7d50 con 0x55bc511df620 2024-04-23T18:48:17.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846ffff640 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc512025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:17.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.450+0000 7f846f7fe640 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8464033000 tx=0x7f8464031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c100dba43aa7bfde server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f8455ffb640 1 -- 172.21.15.33:0/2549109025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035040 con 0x55bc511df620 2024-04-23T18:48:17.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f8455ffb640 1 -- 172.21.15.33:0/2549109025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004250 con 0x55bc511df620 2024-04-23T18:48:17.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f8455ffb640 1 -- 172.21.15.33:0/2549109025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464005560 con 0x55bc511df620 2024-04-23T18:48:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- 172.21.15.33:0/2549109025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 msgr2=0x55bc511e9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:17.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f8464033000 tx=0x7f8464031400 comp rx=0 tx=0).stop 2024-04-23T18:48:17.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- 172.21.15.33:0/2549109025 shutdown_connections 2024-04-23T18:48:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511ea800 0x55bc51202af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:18.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc512025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:18.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 --2- 172.21.15.33:0/2549109025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511df620 0x55bc511e9d40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- 172.21.15.33:0/2549109025 >> 172.21.15.33:0/2549109025 conn(0x55bc511dbb30 msgr2=0x55bc511e5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:18.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- 172.21.15.33:0/2549109025 shutdown_connections 2024-04-23T18:48:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- 172.21.15.33:0/2549109025 wait complete. 2024-04-23T18:48:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.451+0000 7f847976a740 1 -- start start 2024-04-23T18:48:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 Processor -- start 2024-04-23T18:48:18.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc51258120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc51258660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:18.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b0470 con 0x55bc511ea800 2024-04-23T18:48:18.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84701afd70 con 0x55bc511dff40 2024-04-23T18:48:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84701b0b40 con 0x55bc511df620 2024-04-23T18:48:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f847976a740 10 client.0 _ll_register_callbacks cb 0x7f84640314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:18.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:18.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f846f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52222/0 (socket says 172.21.15.33:52222) 2024-04-23T18:48:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f846f7fe640 1 -- 172.21.15.33:0/1790428990 learned_addr learned my addr 172.21.15.33:0/1790428990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc51258120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:18.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.452+0000 7f846ffff640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc51258660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 -- 172.21.15.33:0/1790428990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 msgr2=0x55bc51258120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc51258120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 -- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 msgr2=0x55bc51258660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc51258660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 -- 172.21.15.33:0/1790428990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc511f0370 con 0x55bc511ea800 2024-04-23T18:48:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f846f7fe640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8464054d40 tx=0x7f8464055f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e792aaa4fdbf5b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:18.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f844f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035070 con 0x55bc511ea800 2024-04-23T18:48:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f8464005480 con 0x55bc511ea800 2024-04-23T18:48:19.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8464004250 con 0x55bc511ea800 2024-04-23T18:48:19.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8464070050 con 0x55bc511ea800 2024-04-23T18:48:19.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f847976a740 10 client.17742 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:19.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bc512608a0 con 0x55bc511ea800 2024-04-23T18:48:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.453+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004250 con 0x55bc511ea800 2024-04-23T18:48:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f84640546a0 con 0x55bc511ea800 2024-04-23T18:48:19.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 10 client.17742 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 10 client.17742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 20 client.17742 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc51262f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 4 client.17742 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:48:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc5114c200 con 0x55bc511fa890 2024-04-23T18:48:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f846ffff640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc51262f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f844f7fe640 10 client.17742 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.454+0000 7f846ffff640 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc51262f70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb417b95232e06fc server_cookie=4f7837e17d5c32b in_seq=0 out_seq=0 2024-04-23T18:48:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.458+0000 7f844f7fe640 1 -- 172.21.15.33:0/1790428990 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc5114c200 con 0x55bc511fa890 2024-04-23T18:48:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.458+0000 7f844f7fe640 10 client.17742 handle_command_reply: tid=0 2024-04-23T18:48:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.458+0000 7f847976a740 1 client.17742 shutdown 2024-04-23T18:48:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.458+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 msgr2=0x55bc511efbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.458+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f8464054d40 tx=0x7f8464055f60 comp rx=0 tx=0).stop 2024-04-23T18:48:20.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 shutdown_connections 2024-04-23T18:48:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bc511fa890 0x55bc51262f70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:20.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc511ea800 0x55bc511efbe0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bc511dff40 0x55bc51258660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 --2- 172.21.15.33:0/1790428990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc511df620 0x55bc51258120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:20.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f844f7fe640 0 client.17742 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:20.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 >> 172.21.15.33:0/1790428990 conn(0x55bc511dbb30 msgr2=0x55bc511da420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:20.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 shutdown_connections 2024-04-23T18:48:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 1 -- 172.21.15.33:0/1790428990 wait complete. 2024-04-23T18:48:20.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f847976a740 20 client.17742 trim_cache size 0 max 16384 2024-04-23T18:48:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:48:20.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:48:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:48:20.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:48:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 msgr2=0x7f84580a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:20.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 0x7f84580a6330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8468098c70 tx=0x7f8468098410 comp rx=0 tx=0).stop 2024-04-23T18:48:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 msgr2=0x7f84701a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f846800eaa0 tx=0x7f846800ef60 comp rx=0 tx=0).stop 2024-04-23T18:48:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 shutdown_connections 2024-04-23T18:48:20.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84580a3e70 0x7f84580a6330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:20.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84701035d0 0x7f84701a6880 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8470106190 0x7f8470103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 --2- 172.21.15.33:0/1708053878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84701010c0 0x7f8470102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.459+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 >> 172.21.15.33:0/1708053878 conn(0x7f8470077fd0 msgr2=0x7f847006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.460+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 shutdown_connections 2024-04-23T18:48:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:29.460+0000 7f8476976640 1 -- 172.21.15.33:0/1708053878 wait complete. 2024-04-23T18:48:21.378 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:29 smithi033 ceph-mon[29274]: pgmap v638: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:48:21.409 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:31 smithi033 ceph-mon[29274]: pgmap v639: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-23T18:48:21.448 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:33 smithi033 ceph-mon[29274]: pgmap v640: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 KiB/s wr, 15 op/s 2024-04-23T18:48:21.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:35 smithi033 ceph-mon[29274]: pgmap v641: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:21.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:37 smithi033 ceph-mon[29274]: pgmap v642: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:21.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:39 smithi033 ceph-mon[29274]: pgmap v643: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:21.504 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:11 smithi106 ceph-mon[32716]: pgmap v694: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:48:21.611 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:11 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:21.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:11 smithi106 ceph-mon[32716]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:48:21.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:13 smithi106 ceph-mon[32716]: pgmap v696: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-23T18:48:21.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:15 smithi106 ceph-mon[32716]: pgmap v697: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 957 B/s wr, 10 op/s 2024-04-23T18:48:21.740 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:16 smithi106 ceph-mon[32716]: daemon mds.d restarted 2024-04-23T18:48:21.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:16 smithi106 ceph-mon[32716]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:48:21.811 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/1517776929,v1:172.21.15.33:6837/1517776929] up:boot 2024-04-23T18:48:21.922 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:22.034 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:48:22.181 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:17 smithi106 ceph-mon[32716]: pgmap v698: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:48:22.251 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:19 smithi106 ceph-mon[32716]: pgmap v700: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:48:22.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:11 smithi122 ceph-mon[32546]: pgmap v694: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:48:22.315 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:11 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:22.323 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:11 smithi122 ceph-mon[32546]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:48:22.400 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:13 smithi122 ceph-mon[32546]: pgmap v696: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-23T18:48:22.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:15 smithi122 ceph-mon[32546]: pgmap v697: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 957 B/s wr, 10 op/s 2024-04-23T18:48:22.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:16 smithi122 ceph-mon[32546]: daemon mds.d restarted 2024-04-23T18:48:22.479 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:16 smithi122 ceph-mon[32546]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:48:22.509 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/1517776929,v1:172.21.15.33:6837/1517776929] up:boot 2024-04-23T18:48:22.549 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:48:22.557 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:48:22.566 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:17 smithi122 ceph-mon[32546]: pgmap v698: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:48:22.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:19 smithi122 ceph-mon[32546]: pgmap v700: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:48:22.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:21 smithi122 ceph-mon[32546]: pgmap v701: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:48:22.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:21 smithi106 ceph-mon[32716]: pgmap v701: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:48:22.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.391+0000 7f5202e79640 1 -- 172.21.15.33:0/739742752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc06a840 msgr2=0x7f51fc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.391+0000 7f5202e79640 1 --2- 172.21.15.33:0/739742752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc06a840 0x7f51fc06ac20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009900 tx=0x7f51ec02a410 comp rx=0 tx=0).stop 2024-04-23T18:48:23.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- 172.21.15.33:0/739742752 shutdown_connections 2024-04-23T18:48:23.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- 172.21.15.33:0/739742752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc107270 0x7f51fc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:23.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- 172.21.15.33:0/739742752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51fc06b1f0 0x7f51fc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- 172.21.15.33:0/739742752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc06a840 0x7f51fc06ac20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- 172.21.15.33:0/739742752 >> 172.21.15.33:0/739742752 conn(0x7f51fc077eb0 msgr2=0x7f51fc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:23.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- 172.21.15.33:0/739742752 shutdown_connections 2024-04-23T18:48:23.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- 172.21.15.33:0/739742752 wait complete. 2024-04-23T18:48:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 Processor -- start 2024-04-23T18:48:23.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- start start 2024-04-23T18:48:23.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc06a840 0x7f51fc198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc107270 0x7f51fc198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:23.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51fc19d010 0x7f51fc19d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:23.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1121d0 con 0x7f51fc107270 2024-04-23T18:48:23.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc112050 con 0x7f51fc06a840 2024-04-23T18:48:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.392+0000 7f5202e79640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc112350 con 0x7f51fc19d010 2024-04-23T18:48:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc107270 0x7f51fc198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:23.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc107270 0x7f51fc198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44630/0 (socket says 172.21.15.33:44630) 2024-04-23T18:48:23.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 -- 172.21.15.33:0/4054290333 learned_addr learned my addr 172.21.15.33:0/4054290333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:23.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f5200c15640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc06a840 0x7f51fc198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f5201416640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51fc19d010 0x7f51fc19d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:23.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 -- 172.21.15.33:0/4054290333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51fc19d010 msgr2=0x7f51fc19d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:24.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51fc19d010 0x7f51fc19d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:24.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 -- 172.21.15.33:0/4054290333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc06a840 msgr2=0x7f51fc198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:24.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc06a840 0x7f51fc198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:24.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 -- 172.21.15.33:0/4054290333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc19db00 con 0x7f51fc107270 2024-04-23T18:48:24.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f5200c15640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51fc06a840 0x7f51fc198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:48:24.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51fbfff640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51fc107270 0x7f51fc198c90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f51f000c970 tx=0x7f51f000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:24.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51f9ffb640 1 -- 172.21.15.33:0/4054290333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0007bb0 con 0x7f51fc107270 2024-04-23T18:48:24.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51f9ffb640 1 -- 172.21.15.33:0/4054290333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f0007d50 con 0x7f51fc107270 2024-04-23T18:48:24.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f51f9ffb640 1 -- 172.21.15.33:0/4054290333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0018730 con 0x7f51fc107270 2024-04-23T18:48:24.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.393+0000 7f5202e79640 1 -- 172.21.15.33:0/4054290333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc19ddf0 con 0x7f51fc107270 2024-04-23T18:48:24.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.394+0000 7f5202e79640 1 -- 172.21.15.33:0/4054290333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a5590 con 0x7f51fc107270 2024-04-23T18:48:24.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.396+0000 7f51f9ffb640 1 -- 172.21.15.33:0/4054290333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51f000f990 con 0x7f51fc107270 2024-04-23T18:48:24.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 Processor -- start 2024-04-23T18:48:24.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 -- start start 2024-04-23T18:48:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 0x558b3ace8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:24.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:24.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:24.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b3ac4d6d0 con 0x558b3acf3480 2024-04-23T18:48:24.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558b3abea1a0 con 0x558b3ace8db0 2024-04-23T18:48:24.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5205b8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b3ac49690 con 0x558b3ace8490 2024-04-23T18:48:24.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51f9ffb640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f51e40a3d20 0x7f51e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:24.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:24.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44640/0 (socket says 172.21.15.33:44640) 2024-04-23T18:48:24.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 -- 172.21.15.33:0/2859248820 learned_addr learned my addr 172.21.15.33:0/2859248820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:24.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5200c15640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:24.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5201416640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f51e40a3d20 0x7f51e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:24.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51f9ffb640 1 -- 172.21.15.33:0/4054290333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f51f00d0950 con 0x7f51fc107270 2024-04-23T18:48:25.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f5201416640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 0x558b3ace8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:25.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 -- 172.21.15.33:0/2859248820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 msgr2=0x558b3ace8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:25.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 0x558b3ace8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 -- 172.21.15.33:0/2859248820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 msgr2=0x558b3acf2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:25.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.397+0000 7f51fbfff640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:25.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f51fbfff640 1 -- 172.21.15.33:0/2859248820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3abf45b0 con 0x558b3acf3480 2024-04-23T18:48:25.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f5201416640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 0x558b3ace8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:48:25.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f51fbfff640 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cdfc0 tx=0x7f51f0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b4bddd196ab59d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:25.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f5201416640 1 --2- 172.21.15.33:0/4054290333 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f51e40a3d20 0x7f51e40a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005fd0 tx=0x7f51f4009330 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:25.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f51e2ffd640 1 -- 172.21.15.33:0/2859248820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00a0c50 con 0x558b3acf3480 2024-04-23T18:48:25.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f51e2ffd640 1 -- 172.21.15.33:0/2859248820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f00a5510 con 0x558b3acf3480 2024-04-23T18:48:25.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.398+0000 7f51e2ffd640 1 -- 172.21.15.33:0/2859248820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000f450 con 0x558b3acf3480 2024-04-23T18:48:25.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- 172.21.15.33:0/2859248820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 msgr2=0x558b3acf5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:25.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f51f00cdfc0 tx=0x7f51f0021000 comp rx=0 tx=0).stop 2024-04-23T18:48:25.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- 172.21.15.33:0/2859248820 shutdown_connections 2024-04-23T18:48:25.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acf5870 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:25.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:25.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 --2- 172.21.15.33:0/2859248820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8490 0x558b3ace8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:25.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- 172.21.15.33:0/2859248820 >> 172.21.15.33:0/2859248820 conn(0x558b3ace47b0 msgr2=0x558b3ace4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- 172.21.15.33:0/2859248820 shutdown_connections 2024-04-23T18:48:25.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- 172.21.15.33:0/2859248820 wait complete. 2024-04-23T18:48:25.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 -- start start 2024-04-23T18:48:25.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.399+0000 7f5205b8c740 1 Processor -- start 2024-04-23T18:48:25.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:26.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:26.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558b3acf62b0 con 0x558b3ace8490 2024-04-23T18:48:26.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558b3abe7b20 con 0x558b3ace8db0 2024-04-23T18:48:26.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b3acf6430 con 0x558b3acf3480 2024-04-23T18:48:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 10 client.0 _ll_register_callbacks cb 0x558b3abe7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:26.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:26.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44646/0 (socket says 172.21.15.33:44646) 2024-04-23T18:48:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 -- 172.21.15.33:0/3247420158 learned_addr learned my addr 172.21.15.33:0/3247420158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5201416640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 -- 172.21.15.33:0/3247420158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 msgr2=0x558b3acf08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:26.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:26.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 -- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 msgr2=0x558b3acf0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:26.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:26.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 -- 172.21.15.33:0/3247420158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3a95f920 con 0x558b3ace8490 2024-04-23T18:48:26.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5200c15640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f51ec02a930 tx=0x7f51ec0318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f67afda4319bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:26.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f51dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:26.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec035040 con 0x558b3ace8490 2024-04-23T18:48:26.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f51ec031d10 con 0x558b3ace8490 2024-04-23T18:48:26.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.400+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f00a5510 con 0x558b3ace8490 2024-04-23T18:48:26.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f51ec004100 con 0x558b3ace8490 2024-04-23T18:48:26.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 10 client.17754 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:26.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558b3aa2bd30 con 0x558b3ace8490 2024-04-23T18:48:26.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f00a5510 con 0x558b3ace8490 2024-04-23T18:48:26.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f51ec004730 con 0x558b3ace8490 2024-04-23T18:48:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 10 client.17754 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:26.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 10 client.17754 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:48:26.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 20 client.17754 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:27.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad03580 0x558b3ad059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 4 client.17754 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b3ac0b500 con 0x558b3ad03580 2024-04-23T18:48:27.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.401+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad03580 0x558b3ad059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:27.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.402+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad03580 0x558b3ad059b0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268260a6d925ce6d server_cookie=7555b768c7a37ff4 in_seq=0 out_seq=0 2024-04-23T18:48:27.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.402+0000 7f51dbfff640 10 client.17754 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:27.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f51dbfff640 1 -- 172.21.15.33:0/3247420158 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b3ac0b500 con 0x558b3ad03580 2024-04-23T18:48:27.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f51dbfff640 10 client.17754 handle_command_reply: tid=0 2024-04-23T18:48:27.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 client.17754 shutdown 2024-04-23T18:48:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 msgr2=0x558b3acf2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:27.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f51ec02a930 tx=0x7f51ec0318a0 comp rx=0 tx=0).stop 2024-04-23T18:48:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 shutdown_connections 2024-04-23T18:48:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad03580 0x558b3ad059b0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:27.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8db0 0x558b3acf0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:27.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3247420158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8490 0x558b3acf2f10 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f51dbfff640 0 client.17754 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 >> 172.21.15.33:0/3247420158 conn(0x558b3ace47b0 msgr2=0x558b3acea5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:27.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 shutdown_connections 2024-04-23T18:48:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 1 -- 172.21.15.33:0/3247420158 wait complete. 2024-04-23T18:48:28.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.403+0000 7f5205b8c740 20 client.17754 trim_cache size 0 max 16384 2024-04-23T18:48:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 Processor -- start 2024-04-23T18:48:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 -- start start 2024-04-23T18:48:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acff170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:28.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1af640 con 0x558b3ace8db0 2024-04-23T18:48:28.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ad400 con 0x558b3ace8490 2024-04-23T18:48:28.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f5205b8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1aef70 con 0x558b3acf3480 2024-04-23T18:48:28.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:28.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44650/0 (socket says 172.21.15.33:44650) 2024-04-23T18:48:28.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.420+0000 7f51fbfff640 1 -- 172.21.15.33:0/3219074678 learned_addr learned my addr 172.21.15.33:0/3219074678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:28.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f5201416640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:28.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f5200c15640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:28.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 -- 172.21.15.33:0/3219074678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 msgr2=0x558b3acf2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:28.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:28.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 -- 172.21.15.33:0/3219074678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 msgr2=0x558b3acff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:28.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 -- 172.21.15.33:0/3219074678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3acf0c60 con 0x558b3ace8db0 2024-04-23T18:48:28.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f5201416640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:28.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f5200c15640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51fbfff640 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f51f0098db0 tx=0x7f51f000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a85baebd9509224 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:28.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51e27fc640 1 -- 172.21.15.33:0/3219074678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00ce2d0 con 0x558b3ace8db0 2024-04-23T18:48:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51e27fc640 1 -- 172.21.15.33:0/3219074678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f009dbd0 con 0x558b3ace8db0 2024-04-23T18:48:29.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.421+0000 7f51e27fc640 1 -- 172.21.15.33:0/3219074678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000f450 con 0x558b3ace8db0 2024-04-23T18:48:29.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- 172.21.15.33:0/3219074678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 msgr2=0x558b3acef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:29.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f51f0098db0 tx=0x7f51f000af90 comp rx=0 tx=0).stop 2024-04-23T18:48:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- 172.21.15.33:0/3219074678 shutdown_connections 2024-04-23T18:48:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3acf3480 0x558b3acf2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3ace8db0 0x558b3acef370 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- 172.21.15.33:0/3219074678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:29.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- 172.21.15.33:0/3219074678 >> 172.21.15.33:0/3219074678 conn(0x558b3ace47b0 msgr2=0x558b3acea470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:29.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- 172.21.15.33:0/3219074678 shutdown_connections 2024-04-23T18:48:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- 172.21.15.33:0/3219074678 wait complete. 2024-04-23T18:48:29.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- start start 2024-04-23T18:48:29.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 Processor -- start 2024-04-23T18:48:29.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acfbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:29.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8db0 0x558b3aced880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:29.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ae1d0 con 0x558b3acf3480 2024-04-23T18:48:29.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1adad0 con 0x558b3ace8490 2024-04-23T18:48:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc1ae8a0 con 0x558b3ace8db0 2024-04-23T18:48:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.422+0000 7f5205b8c740 10 client.0 _ll_register_callbacks cb 0x7f51ec032f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44654/0 (socket says 172.21.15.33:44654) 2024-04-23T18:48:29.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 -- 172.21.15.33:0/4141216695 learned_addr learned my addr 172.21.15.33:0/4141216695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:29.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f51fbfff640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8db0 0x558b3aced880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:29.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5200c15640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acfbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 -- 172.21.15.33:0/4141216695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8db0 msgr2=0x558b3aced880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:30.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8db0 0x558b3aced880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:30.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 -- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 msgr2=0x558b3acfbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acfbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 -- 172.21.15.33:0/4141216695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b3acf4760 con 0x558b3acf3480 2024-04-23T18:48:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5201416640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f51f40039a0 tx=0x7f51f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db0c7b22b4b8836 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f51dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4004250 con 0x558b3acf3480 2024-04-23T18:48:30.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f51f40043f0 con 0x558b3acf3480 2024-04-23T18:48:30.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f009dbd0 con 0x558b3acf3480 2024-04-23T18:48:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f51f40045f0 con 0x558b3acf3480 2024-04-23T18:48:30.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5205b8c740 10 client.17760 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:30.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.423+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558b3ad6a010 con 0x558b3acf3480 2024-04-23T18:48:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f009dbd0 con 0x558b3acf3480 2024-04-23T18:48:30.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f51f4027070 con 0x558b3acf3480 2024-04-23T18:48:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 10 client.17760 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 10 client.17760 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:48:30.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 20 client.17760 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:30.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad6a810 0x558b3ad6cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:30.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 4 client.17760 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6bf9e7c1", "release": true, "await": true}] 2024-04-23T18:48:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6bf9e7c1", "release": true, "await": true}) -- 0x558b3ac5b0f0 con 0x558b3ad6a810 2024-04-23T18:48:30.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.424+0000 7f5200c15640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad6a810 0x558b3ad6cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:30.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f51dbfff640 10 client.17760 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:30.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5200c15640 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad6a810 0x558b3ad6cc30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54476b8ee8dc39a server_cookie=3bb999013cf591b in_seq=0 out_seq=0 2024-04-23T18:48:30.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f51dbfff640 1 -- 172.21.15.33:0/4141216695 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: -1 ) ==== 8+0+592 (crc 0 0 0) 0x558b3ac5b0f0 con 0x558b3ad6a810 2024-04-23T18:48:30.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f51dbfff640 10 client.17760 handle_command_reply: tid=0 2024-04-23T18:48:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 client.17760 shutdown 2024-04-23T18:48:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 msgr2=0x558b3acedeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:31.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f51f40039a0 tx=0x7f51f4004a40 comp rx=0 tx=0).stop 2024-04-23T18:48:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 shutdown_connections 2024-04-23T18:48:31.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558b3ad6a810 0x558b3ad6cc30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558b3acf3480 0x558b3acedeb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:31.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f51dbfff640 0 client.17760 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b3ace8db0 0x558b3aced880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:31.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 --2- 172.21.15.33:0/4141216695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558b3ace8490 0x558b3acfbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:31.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 >> 172.21.15.33:0/4141216695 conn(0x558b3ace47b0 msgr2=0x558b3acea470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:31.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 shutdown_connections 2024-04-23T18:48:31.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 1 -- 172.21.15.33:0/4141216695 wait complete. 2024-04-23T18:48:31.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:46:40.425+0000 7f5205b8c740 20 client.17760 trim_cache size 0 max 16384 2024-04-23T18:48:31.453 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:48:31.507 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 41, 2024-04-23T18:48:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:48:31.526 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 35, 2024-04-23T18:48:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": { 2024-04-23T18:48:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: "6bf9e7c1": { 2024-04-23T18:48:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: "version": 35, 2024-04-23T18:48:31.702 INFO:teuthology.orchestra.run.smithi033.stdout: "age_ref": 406.4, 2024-04-23T18:48:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:48:31.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "EXPIRED", 2024-04-23T18:48:31.824 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 0.0 2024-04-23T18:48:31.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:48:31.993 INFO:teuthology.orchestra.run.smithi033.stdout: "timeout": 60.0, 2024-04-23T18:48:32.047 INFO:teuthology.orchestra.run.smithi033.stdout: "expiration": 82.9, 2024-04-23T18:48:32.163 INFO:teuthology.orchestra.run.smithi033.stdout: "members": { 2024-04-23T18:48:32.263 INFO:teuthology.orchestra.run.smithi033.stdout: "file:/": { 2024-04-23T18:48:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: "excluded": false, 2024-04-23T18:48:32.390 INFO:teuthology.orchestra.run.smithi033.stdout: "state": { 2024-04-23T18:48:32.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "QUIESCED", 2024-04-23T18:48:32.405 INFO:teuthology.orchestra.run.smithi033.stdout: "age": 82.9 2024-04-23T18:48:32.412 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:48:32.419 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:48:32.427 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:48:32.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:48:32.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:48:32.505 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:48:32.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T18:48:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:Error EPERM: 2024-04-23T18:48:32.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:41 smithi033 ceph-mon[29274]: pgmap v644: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:32.651 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't release set '6bf9e7c1' with rc: 1 (EPERM), stdout: { "epoch": 41, "leader": 24551, "set_version": 35, "sets": { "6bf9e7c1": { "version": 35, "age_ref": 406.4, "state": { "name": "EXPIRED", "age": 0.0 }, "timeout": 60.0, "expiration": 82.9, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCED", "age": 82.9 } } } } } } 2024-04-23T18:48:32.720 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 337, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 313, in do_quiesce raise RuntimeError(f"Error releasing set '{set_id}': {rcinfo}") RuntimeError: Error releasing set '6bf9e7c1': 1 (EPERM) 2024-04-23T18:48:32.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:23 smithi106 ceph-mon[32716]: pgmap v702: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:48:32.837 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:25 smithi106 ceph-mon[32716]: pgmap v703: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:48:32.896 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:27 smithi106 ceph-mon[32716]: pgmap v704: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 409 B/s wr, 11 op/s 2024-04-23T18:48:32.906 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:29 smithi106 ceph-mon[32716]: pgmap v705: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 345 B/s wr, 9 op/s 2024-04-23T18:48:32.942 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:31 smithi106 ceph-mon[32716]: pgmap v706: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:48:32.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:23 smithi122 ceph-mon[32546]: pgmap v702: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:48:32.958 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:25 smithi122 ceph-mon[32546]: pgmap v703: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:48:32.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:27 smithi122 ceph-mon[32546]: pgmap v704: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 409 B/s wr, 11 op/s 2024-04-23T18:48:33.035 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:29 smithi122 ceph-mon[32546]: pgmap v705: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 345 B/s wr, 9 op/s 2024-04-23T18:48:33.113 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:31 smithi122 ceph-mon[32546]: pgmap v706: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:48:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 -- 172.21.15.33:0/3479206707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f81021a0 msgr2=0x7fe6f8102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:33.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3479206707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f81021a0 0x7fe6f8102580 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009900 tx=0x7fe6e802a410 comp rx=0 tx=0).stop 2024-04-23T18:48:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 -- 172.21.15.33:0/3479206707 shutdown_connections 2024-04-23T18:48:33.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3479206707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f8107270 0x7fe6f810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3479206707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f8102b50 0x7fe6f8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3479206707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f81021a0 0x7fe6f8102580 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 -- 172.21.15.33:0/3479206707 >> 172.21.15.33:0/3479206707 conn(0x7fe6f80793e0 msgr2=0x7fe6f80797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 -- 172.21.15.33:0/3479206707 shutdown_connections 2024-04-23T18:48:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.559+0000 7fe6fe035640 1 -- 172.21.15.33:0/3479206707 wait complete. 2024-04-23T18:48:33.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.560+0000 7fe6fe035640 1 Processor -- start 2024-04-23T18:48:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.560+0000 7fe6fe035640 1 -- start start 2024-04-23T18:48:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f81021a0 0x7fe6f8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:33.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f819cfd0 0x7fe6f819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81121b0 con 0x7fe6f8107270 2024-04-23T18:48:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8112030 con 0x7fe6f819cfd0 2024-04-23T18:48:33.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6fe035640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8112330 con 0x7fe6f81021a0 2024-04-23T18:48:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44672/0 (socket says 172.21.15.33:44672) 2024-04-23T18:48:34.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/3872845622 learned_addr learned my addr 172.21.15.33:0/3872845622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f81021a0 0x7fe6f8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f819cfd0 0x7fe6f819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/3872845622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f81021a0 msgr2=0x7fe6f8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:34.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f81021a0 0x7fe6f8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/3872845622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f819cfd0 msgr2=0x7fe6f819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f819cfd0 0x7fe6f819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:34.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/3872845622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f819dac0 con 0x7fe6f8107270 2024-04-23T18:48:34.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000d770 tx=0x7fe6e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f4ff9640 1 -- 172.21.15.33:0/3872845622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0017070 con 0x7fe6f8107270 2024-04-23T18:48:34.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.561+0000 7fe6f4ff9640 1 -- 172.21.15.33:0/3872845622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e0009c30 con 0x7fe6f8107270 2024-04-23T18:48:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.562+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f819ddb0 con 0x7fe6f8107270 2024-04-23T18:48:34.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.562+0000 7fe6f4ff9640 1 -- 172.21.15.33:0/3872845622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000f620 con 0x7fe6f8107270 2024-04-23T18:48:34.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.562+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f81a55a0 con 0x7fe6f8107270 2024-04-23T18:48:34.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe6f4ff9640 1 -- 172.21.15.33:0/3872845622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6e001a070 con 0x7fe6f8107270 2024-04-23T18:48:34.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 Processor -- start 2024-04-23T18:48:34.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 -- start start 2024-04-23T18:48:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:34.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe81000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aacbdd4970 con 0x55aacbe6f520 2024-04-23T18:48:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55aacbd714a0 con 0x55aacbe7ec10 2024-04-23T18:48:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.564+0000 7fe700e2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aacbdd0930 con 0x55aacbe6fe40 2024-04-23T18:48:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f4ff9640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 0x7fe6e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44688/0 (socket says 172.21.15.33:44688) 2024-04-23T18:48:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 -- 172.21.15.33:0/1907784278 learned_addr learned my addr 172.21.15.33:0/1907784278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 0x7fe6e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f4ff9640 1 -- 172.21.15.33:0/3872845622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7fe6e00d02b0 con 0x7fe6f8107270 2024-04-23T18:48:35.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe81000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:35.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 -- 172.21.15.33:0/1907784278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 msgr2=0x55aacbe7e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:35.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 -- 172.21.15.33:0/1907784278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 msgr2=0x55aacbe81000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:35.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe81000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 -- 172.21.15.33:0/1907784278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacbae7210 con 0x55aacbe6f520 2024-04-23T18:48:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0013dc0 tx=0x7fe6f00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c105aa49b1ca2c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 0x7fe6e40a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0011cb0 tx=0x7fe6f001d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:35.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6edffb640 1 -- 172.21.15.33:0/1907784278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0022070 con 0x55aacbe6f520 2024-04-23T18:48:35.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6edffb640 1 -- 172.21.15.33:0/1907784278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f0003e20 con 0x55aacbe6f520 2024-04-23T18:48:35.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.565+0000 7fe6edffb640 1 -- 172.21.15.33:0/1907784278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0004580 con 0x55aacbe6f520 2024-04-23T18:48:35.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 -- 172.21.15.33:0/1907784278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 msgr2=0x55aacbe6f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0013dc0 tx=0x7fe6f00117b0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 -- 172.21.15.33:0/1907784278 shutdown_connections 2024-04-23T18:48:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe81000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1907784278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe6f900 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 -- 172.21.15.33:0/1907784278 >> 172.21.15.33:0/1907784278 conn(0x55aacbe6bb30 msgr2=0x55aacbe6bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:36.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 -- 172.21.15.33:0/1907784278 shutdown_connections 2024-04-23T18:48:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.566+0000 7fe700e2f740 1 -- 172.21.15.33:0/1907784278 wait complete. 2024-04-23T18:48:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 -- start start 2024-04-23T18:48:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 Processor -- start 2024-04-23T18:48:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6fe40 0x55aacbe7b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe7ec10 0x55aacbe7bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:36.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aacbe817e0 con 0x55aacbe6f520 2024-04-23T18:48:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55aacbd6f0b0 con 0x55aacbe6fe40 2024-04-23T18:48:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aacbe81960 con 0x55aacbe7ec10 2024-04-23T18:48:36.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe700e2f740 10 client.0 _ll_register_callbacks cb 0x55aacbd5cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:36.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe6f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:36.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe6f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44700/0 (socket says 172.21.15.33:44700) 2024-04-23T18:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe6f77fe640 1 -- 172.21.15.33:0/3270997503 learned_addr learned my addr 172.21.15.33:0/3270997503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe7ec10 0x55aacbe7bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:36.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.567+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6fe40 0x55aacbe7b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:36.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 -- 172.21.15.33:0/3270997503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe7ec10 msgr2=0x55aacbe7bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe7ec10 0x55aacbe7bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 -- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6fe40 msgr2=0x55aacbe7b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6fe40 0x55aacbe7b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 -- 172.21.15.33:0/3270997503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacbbb3620 con 0x55aacbe6f520 2024-04-23T18:48:36.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009900 tx=0x7fe6e80316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2357d6432a08dfeb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8035040 con 0x55aacbe6f520 2024-04-23T18:48:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7fe6e8031d10 con 0x55aacbe6f520 2024-04-23T18:48:37.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6f0003e20 con 0x55aacbe6f520 2024-04-23T18:48:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe6e8004040 con 0x55aacbe6f520 2024-04-23T18:48:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe700e2f740 10 client.17772 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:37.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aacbe8d790 con 0x55aacbe6f520 2024-04-23T18:48:37.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.568+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f0003e20 con 0x55aacbe6f520 2024-04-23T18:48:37.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fe6e80047e0 con 0x55aacbe6f520 2024-04-23T18:48:37.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 10 client.17772 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:37.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 20 client.17772 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:37.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbe8ec00 0x55aacbe91010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 4 client.17772 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aacbde2390 con 0x55aacbe8ec00 2024-04-23T18:48:37.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbe8ec00 0x55aacbe91010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe6d6ffd640 10 client.17772 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.569+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbe8ec00 0x55aacbe91010 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec15ed4b76c7fea0 server_cookie=ee4fcfe1ba764244 in_seq=0 out_seq=0 2024-04-23T18:48:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3270997503 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aacbde2390 con 0x55aacbe8ec00 2024-04-23T18:48:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe6d6ffd640 10 client.17772 handle_command_reply: tid=0 2024-04-23T18:48:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 client.17772 shutdown 2024-04-23T18:48:38.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 msgr2=0x55aacbe7e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:38.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009900 tx=0x7fe6e80316d0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 shutdown_connections 2024-04-23T18:48:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbe8ec00 0x55aacbe91010 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe7ec10 0x55aacbe7bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6fe40 0x55aacbe7b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe6d6ffd640 0 client.17772 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3270997503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6f520 0x55aacbe7e560 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 >> 172.21.15.33:0/3270997503 conn(0x55aacbe6bb30 msgr2=0x55aacbe719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:38.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 shutdown_connections 2024-04-23T18:48:38.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 1 -- 172.21.15.33:0/3270997503 wait complete. 2024-04-23T18:48:38.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.570+0000 7fe700e2f740 20 client.17772 trim_cache size 0 max 16384 2024-04-23T18:48:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.585+0000 7fe700e2f740 1 Processor -- start 2024-04-23T18:48:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.585+0000 7fe700e2f740 1 -- start start 2024-04-23T18:48:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6f520 0x55aacbe8a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:38.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81ad4a0 con 0x55aacbe6fe40 2024-04-23T18:48:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81ab260 con 0x55aacbe7ec10 2024-04-23T18:48:38.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe700e2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81acdd0 con 0x55aacbe6f520 2024-04-23T18:48:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44702/0 (socket says 172.21.15.33:44702) 2024-04-23T18:48:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/1411137267 learned_addr learned my addr 172.21.15.33:0/1411137267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/1411137267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6f520 msgr2=0x55aacbe8a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6f520 0x55aacbe8a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/1411137267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 msgr2=0x55aacbe96920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe96920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 -- 172.21.15.33:0/1411137267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacbe6fc70 con 0x55aacbe6fe40 2024-04-23T18:48:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:48:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0098ce0 tx=0x7fe6e00985f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d37c7349d48d0b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6ed7fa640 1 -- 172.21.15.33:0/1411137267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00b5070 con 0x55aacbe6fe40 2024-04-23T18:48:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6ed7fa640 1 -- 172.21.15.33:0/1411137267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e009c310 con 0x55aacbe6fe40 2024-04-23T18:48:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.586+0000 7fe6ed7fa640 1 -- 172.21.15.33:0/1411137267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0085070 con 0x55aacbe6fe40 2024-04-23T18:48:39.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- 172.21.15.33:0/1411137267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 msgr2=0x55aacbe7e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0098ce0 tx=0x7fe6e00985f0 comp rx=0 tx=0).stop 2024-04-23T18:48:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- 172.21.15.33:0/1411137267 shutdown_connections 2024-04-23T18:48:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe7ec10 0x55aacbe96920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:39.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe6fe40 0x55aacbe7e090 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 --2- 172.21.15.33:0/1411137267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6f520 0x55aacbe8a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:39.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- 172.21.15.33:0/1411137267 >> 172.21.15.33:0/1411137267 conn(0x55aacbe6bb30 msgr2=0x55aacbe76f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- 172.21.15.33:0/1411137267 shutdown_connections 2024-04-23T18:48:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- 172.21.15.33:0/1411137267 wait complete. 2024-04-23T18:48:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.587+0000 7fe700e2f740 1 -- start start 2024-04-23T18:48:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 Processor -- start 2024-04-23T18:48:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6f520 0x55aacbe78bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81ac030 con 0x55aacbe7ec10 2024-04-23T18:48:39.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81ab930 con 0x55aacbe6f520 2024-04-23T18:48:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f81ac700 con 0x55aacbe6fe40 2024-04-23T18:48:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe700e2f740 10 client.0 _ll_register_callbacks cb 0x7fe6e8032dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:40.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:40.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44718/0 (socket says 172.21.15.33:44718) 2024-04-23T18:48:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 -- 172.21.15.33:0/3681815885 learned_addr learned my addr 172.21.15.33:0/3681815885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6f520 0x55aacbe78bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f6ffd640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:40.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 -- 172.21.15.33:0/3681815885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 msgr2=0x55aacbe790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:40.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 -- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6f520 msgr2=0x55aacbe78bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6f520 0x55aacbe78bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 -- 172.21.15.33:0/3681815885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacbef46d0 con 0x55aacbe7ec10 2024-04-23T18:48:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6f7fff640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000a340 tx=0x7fe6f0013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9620278fd9c1172 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.588+0000 7fe6d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:40.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0022040 con 0x55aacbe7ec10 2024-04-23T18:48:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7fe6f0013860 con 0x55aacbe7ec10 2024-04-23T18:48:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6e009c310 con 0x55aacbe7ec10 2024-04-23T18:48:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe6f000cd10 con 0x55aacbe7ec10 2024-04-23T18:48:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 10 client.17778 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aacbef58b0 con 0x55aacbe7ec10 2024-04-23T18:48:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e009c310 con 0x55aacbe7ec10 2024-04-23T18:48:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fe6f0004410 con 0x55aacbe7ec10 2024-04-23T18:48:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 10 client.17778 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 10 client.17778 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 20 client.17778 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbef5ec0 0x55aacbef82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 4 client.17778 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:48:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aacbddc200 con 0x55aacbef5ec0 2024-04-23T18:48:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.589+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbef5ec0 0x55aacbef82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.590+0000 7fe6f77fe640 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbef5ec0 0x55aacbef82e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a66a68808e3eae89 server_cookie=138942a762c9509c in_seq=0 out_seq=0 2024-04-23T18:48:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.590+0000 7fe6d6ffd640 10 client.17778 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe6d6ffd640 1 -- 172.21.15.33:0/3681815885 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aacbddc200 con 0x55aacbef5ec0 2024-04-23T18:48:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe6d6ffd640 10 client.17778 handle_command_reply: tid=0 2024-04-23T18:48:41.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 client.17778 shutdown 2024-04-23T18:48:41.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 msgr2=0x55aacbef4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000a340 tx=0x7fe6f0013c70 comp rx=0 tx=0).stop 2024-04-23T18:48:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 shutdown_connections 2024-04-23T18:48:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aacbef5ec0 0x55aacbef82e0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aacbe7ec10 0x55aacbef4190 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aacbe6fe40 0x55aacbe790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe6d6ffd640 0 client.17778 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:41.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 --2- 172.21.15.33:0/3681815885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aacbe6f520 0x55aacbe78bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 >> 172.21.15.33:0/3681815885 conn(0x55aacbe6bb30 msgr2=0x55aacbe71070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 shutdown_connections 2024-04-23T18:48:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 1 -- 172.21.15.33:0/3681815885 wait complete. 2024-04-23T18:48:41.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.594+0000 7fe700e2f740 20 client.17778 trim_cache size 0 max 16384 2024-04-23T18:48:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:48:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:48:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:48:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:48:41.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 msgr2=0x7fe6e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:41.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 0x7fe6e40a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0011cb0 tx=0x7fe6f001d000 comp rx=0 tx=0).stop 2024-04-23T18:48:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 msgr2=0x7fe6f8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000d770 tx=0x7fe6e000dc30 comp rx=0 tx=0).stop 2024-04-23T18:48:41.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 shutdown_connections 2024-04-23T18:48:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6e40a3dc0 0x7fe6e40a6280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6f819cfd0 0x7fe6f819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f8107270 0x7fe6f8198c50 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 --2- 172.21.15.33:0/3872845622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6f81021a0 0x7fe6f8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 >> 172.21.15.33:0/3872845622 conn(0x7fe6f80793e0 msgr2=0x7fe6f80ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 shutdown_connections 2024-04-23T18:48:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:42.595+0000 7fe6fe035640 1 -- 172.21.15.33:0/3872845622 wait complete. 2024-04-23T18:48:42.010 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:43 smithi033 ceph-mon[29274]: pgmap v645: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:42.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:45 smithi033 ceph-mon[29274]: pgmap v646: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:48:42.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:47 smithi033 ceph-mon[29274]: pgmap v647: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:42.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:49 smithi033 ceph-mon[29274]: pgmap v648: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:51 smithi033 ceph-mon[29274]: pgmap v649: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:53 smithi033 ceph-mon[29274]: pgmap v650: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:42.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:54 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:48:42.487 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:55 smithi033 ceph-mon[29274]: pgmap v651: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.551 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:33 smithi122 ceph-mon[32546]: pgmap v707: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:48:42.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:35 smithi122 ceph-mon[32546]: pgmap v708: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.571 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:37 smithi122 ceph-mon[32546]: pgmap v709: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:42.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:39 smithi122 ceph-mon[32546]: pgmap v710: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.666 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:41 smithi122 ceph-mon[32546]: pgmap v711: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.676 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:33 smithi106 ceph-mon[32716]: pgmap v707: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:48:42.683 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:35 smithi106 ceph-mon[32716]: pgmap v708: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.817 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:37 smithi106 ceph-mon[32716]: pgmap v709: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:42.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:39 smithi106 ceph-mon[32716]: pgmap v710: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:41 smithi106 ceph-mon[32716]: pgmap v711: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 -- 172.21.15.33:0/721342213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 msgr2=0x7f8c5810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 --2- 172.21.15.33:0/721342213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c5810dd40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009ca0 tx=0x7f8c5002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:48:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 -- 172.21.15.33:0/721342213 shutdown_connections 2024-04-23T18:48:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 --2- 172.21.15.33:0/721342213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c5810dd40 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:43.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 --2- 172.21.15.33:0/721342213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c58101aa0 0x7f8c58105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 --2- 172.21.15.33:0/721342213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581010f0 0x7f8c581014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 -- 172.21.15.33:0/721342213 >> 172.21.15.33:0/721342213 conn(0x7f8c58077fd0 msgr2=0x7f8c580783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 -- 172.21.15.33:0/721342213 shutdown_connections 2024-04-23T18:48:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.690+0000 7f8c5e619640 1 -- 172.21.15.33:0/721342213 wait complete. 2024-04-23T18:48:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 Processor -- start 2024-04-23T18:48:43.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 -- start start 2024-04-23T18:48:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c581010f0 0x7f8c58102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:43.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581035b0 0x7f8c581a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581110e0 con 0x7f8c58106170 2024-04-23T18:48:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58110f60 con 0x7f8c581035b0 2024-04-23T18:48:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5e619640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c58111260 con 0x7f8c581010f0 2024-04-23T18:48:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:43.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50962/0 (socket says 172.21.15.33:50962) 2024-04-23T18:48:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c577fe640 1 -- 172.21.15.33:0/4150363176 learned_addr learned my addr 172.21.15.33:0/4150363176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581035b0 0x7f8c581a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.691+0000 7f8c57fff640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c581010f0 0x7f8c58102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:43.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 -- 172.21.15.33:0/4150363176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c581010f0 msgr2=0x7f8c58102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c581010f0 0x7f8c58102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:43.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 -- 172.21.15.33:0/4150363176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581035b0 msgr2=0x7f8c581a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:43.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581035b0 0x7f8c581a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:43.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 -- 172.21.15.33:0/4150363176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c581a6fe0 con 0x7f8c58106170 2024-04-23T18:48:43.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c577fe640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00e9f0 tx=0x7f8c4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:43.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c557fa640 1 -- 172.21.15.33:0/4150363176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00ccc0 con 0x7f8c58106170 2024-04-23T18:48:43.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c557fa640 1 -- 172.21.15.33:0/4150363176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c4c0054d0 con 0x7f8c58106170 2024-04-23T18:48:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c581a72d0 con 0x7f8c58106170 2024-04-23T18:48:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c557fa640 1 -- 172.21.15.33:0/4150363176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c010430 con 0x7f8c58106170 2024-04-23T18:48:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.692+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c581a7820 con 0x7f8c58106170 2024-04-23T18:48:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c557fa640 1 -- 172.21.15.33:0/4150363176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c4c0046c0 con 0x7f8c58106170 2024-04-23T18:48:44.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 Processor -- start 2024-04-23T18:48:44.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 -- start start 2024-04-23T18:48:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 0x55aed6a3d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:44.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a3d800 0x55aed6a3fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aed6997970 con 0x55aed6a32620 2024-04-23T18:48:44.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55aed69344a0 con 0x55aed6a3d800 2024-04-23T18:48:44.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c61405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aed6993930 con 0x55aed6a32f40 2024-04-23T18:48:44.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50976/0 (socket says 172.21.15.33:50976) 2024-04-23T18:48:44.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/1921078161 learned_addr learned my addr 172.21.15.33:0/1921078161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:44.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c57fff640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 0x55aed6a3d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c577fe640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a3d800 0x55aed6a3fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:44.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.695+0000 7f8c557fa640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 0x7f8c400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 -- 172.21.15.33:0/1921078161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a3d800 msgr2=0x55aed6a3fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a3d800 0x55aed6a3fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:44.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 -- 172.21.15.33:0/1921078161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 msgr2=0x55aed6a32a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:44.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c557fa640 1 -- 172.21.15.33:0/4150363176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f8c4c0d0a60 con 0x7f8c58106170 2024-04-23T18:48:44.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 -- 172.21.15.33:0/1921078161 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed66aa210 con 0x55aed6a32f40 2024-04-23T18:48:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:48:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 0x7f8c400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c57fff640 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 0x55aed6a3d2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009c20 tx=0x7f8c48011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=521f1694a58bdf42 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:45.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1921078161 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004050 con 0x55aed6a32f40 2024-04-23T18:48:45.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1921078161 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c480054d0 con 0x55aed6a32f40 2024-04-23T18:48:45.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.696+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1921078161 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004cd0 con 0x55aed6a32f40 2024-04-23T18:48:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- 172.21.15.33:0/1921078161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 msgr2=0x55aed6a3d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:45.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 0x7f8c400a6300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c5002f500 tx=0x7f8c500095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:45.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 0x55aed6a3d2c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009c20 tx=0x7f8c48011810 comp rx=0 tx=0).stop 2024-04-23T18:48:45.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- 172.21.15.33:0/1921078161 shutdown_connections 2024-04-23T18:48:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a3d800 0x55aed6a3fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:45.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32f40 0x55aed6a3d2c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:45.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 --2- 172.21.15.33:0/1921078161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a32620 0x55aed6a32a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- 172.21.15.33:0/1921078161 >> 172.21.15.33:0/1921078161 conn(0x55aed6a2eb30 msgr2=0x55aed6a2ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- 172.21.15.33:0/1921078161 shutdown_connections 2024-04-23T18:48:45.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- 172.21.15.33:0/1921078161 wait complete. 2024-04-23T18:48:45.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 -- start start 2024-04-23T18:48:45.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.697+0000 7f8c61405740 1 Processor -- start 2024-04-23T18:48:45.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:45.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:45.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aed6a40430 con 0x55aed6a3d800 2024-04-23T18:48:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55aed69320b0 con 0x55aed6a32f40 2024-04-23T18:48:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aed6a405b0 con 0x55aed6a32620 2024-04-23T18:48:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c61405740 10 client.0 _ll_register_callbacks cb 0x55aed691fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:46.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50978/0 (socket says 172.21.15.33:50978) 2024-04-23T18:48:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:51252/0 (socket says 172.21.15.33:51252) 2024-04-23T18:48:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:57008/0 (socket says 172.21.15.33:57008) 2024-04-23T18:48:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/2776774254 learned_addr learned my addr 172.21.15.33:0/2776774254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 -- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 msgr2=0x55aed6a3a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 -- 172.21.15.33:0/2776774254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 msgr2=0x55aed6a3a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 -- 172.21.15.33:0/2776774254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed6776620 con 0x55aed6a32620 2024-04-23T18:48:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:48:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:48:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000c00 tx=0x7f8c48005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1080151c5001a7c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:46.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.698+0000 7f8c377fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:48:46.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c480046c0 con 0x55aed6a32620 2024-04-23T18:48:46.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f8c48003800 con 0x55aed6a32620 2024-04-23T18:48:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c480054d0 con 0x55aed6a32620 2024-04-23T18:48:47.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c48033050 con 0x55aed6a32620 2024-04-23T18:48:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 10 client.27070 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aed6a4c2c0 con 0x55aed6a32620 2024-04-23T18:48:47.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c480054d0 con 0x55aed6a32620 2024-04-23T18:48:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f8c480051a0 con 0x55aed6a32620 2024-04-23T18:48:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 10 client.27070 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:47.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 10 client.27070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 20 client.27070 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6a4d6c0 0x55aed6a4fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 4 client.27070 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.699+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aed69a5390 con 0x55aed6a4d6c0 2024-04-23T18:48:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.700+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6a4d6c0 0x55aed6a4fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:47.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.700+0000 7f8c377fe640 10 client.27070 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.700+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6a4d6c0 0x55aed6a4fab0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe047883b6fa38da server_cookie=b6ea734158c29e40 in_seq=0 out_seq=0 2024-04-23T18:48:47.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c377fe640 1 -- 172.21.15.33:0/2776774254 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aed69a5390 con 0x55aed6a4d6c0 2024-04-23T18:48:47.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c377fe640 10 client.27070 handle_command_reply: tid=0 2024-04-23T18:48:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 client.27070 shutdown 2024-04-23T18:48:47.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 msgr2=0x55aed6a3d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000c00 tx=0x7f8c48005700 comp rx=0 tx=0).stop 2024-04-23T18:48:47.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 shutdown_connections 2024-04-23T18:48:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6a4d6c0 0x55aed6a4fab0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a3a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c377fe640 0 client.27070 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:47.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a3a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:47.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 --2- 172.21.15.33:0/2776774254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a3d0c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 >> 172.21.15.33:0/2776774254 conn(0x55aed6a2eb30 msgr2=0x55aed6a348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:47.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 shutdown_connections 2024-04-23T18:48:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 1 -- 172.21.15.33:0/2776774254 wait complete. 2024-04-23T18:48:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.701+0000 7f8c61405740 20 client.27070 trim_cache size 0 max 16384 2024-04-23T18:48:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.716+0000 7f8c61405740 1 Processor -- start 2024-04-23T18:48:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.716+0000 7f8c61405740 1 -- start start 2024-04-23T18:48:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.716+0000 7f8c61405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c61405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a55230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c61405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c61405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581b1850 con 0x55aed6a3d800 2024-04-23T18:48:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c61405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581af610 con 0x55aed6a32f40 2024-04-23T18:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c61405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581b1180 con 0x55aed6a32620 2024-04-23T18:48:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:48.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50990/0 (socket says 172.21.15.33:50990) 2024-04-23T18:48:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/3365763531 learned_addr learned my addr 172.21.15.33:0/3365763531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c577fe640 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a55230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/3365763531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 msgr2=0x55aed6a492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/3365763531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 msgr2=0x55aed6a55230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a55230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/3365763531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aed6a38920 con 0x55aed6a3d800 2024-04-23T18:48:48.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009430 tx=0x7f8c5003e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7597e564b84cbf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:48.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.717+0000 7f8c3dffb640 1 -- 172.21.15.33:0/3365763531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50058070 con 0x55aed6a3d800 2024-04-23T18:48:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c3dffb640 1 -- 172.21.15.33:0/3365763531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c50036040 con 0x55aed6a3d800 2024-04-23T18:48:48.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c3dffb640 1 -- 172.21.15.33:0/3365763531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5002f9e0 con 0x55aed6a3d800 2024-04-23T18:48:48.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 -- 172.21.15.33:0/3365763531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 msgr2=0x55aed6a55770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:48.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009430 tx=0x7f8c5003e030 comp rx=0 tx=0).stop 2024-04-23T18:48:48.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 -- 172.21.15.33:0/3365763531 shutdown_connections 2024-04-23T18:48:49.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6a55770 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a55230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:49.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 --2- 172.21.15.33:0/3365763531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 -- 172.21.15.33:0/3365763531 >> 172.21.15.33:0/3365763531 conn(0x55aed6a2eb30 msgr2=0x55aed6a39910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 -- 172.21.15.33:0/3365763531 shutdown_connections 2024-04-23T18:48:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.718+0000 7f8c61405740 1 -- 172.21.15.33:0/3365763531 wait complete. 2024-04-23T18:48:49.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 -- start start 2024-04-23T18:48:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 Processor -- start 2024-04-23T18:48:49.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:49.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581b03e0 con 0x55aed6a3d800 2024-04-23T18:48:49.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581afce0 con 0x55aed6a32f40 2024-04-23T18:48:49.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c581b0ab0 con 0x55aed6a32620 2024-04-23T18:48:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c61405740 10 client.0 _ll_register_callbacks cb 0x7f8c48009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c5cbb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51002/0 (socket says 172.21.15.33:51002) 2024-04-23T18:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c5cbb6640 1 -- 172.21.15.33:0/2140814767 learned_addr learned my addr 172.21.15.33:0/2140814767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.719+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:49.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 -- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 msgr2=0x55aed6a42f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:50.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a42f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 -- 172.21.15.33:0/2140814767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 msgr2=0x55aed6ab1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 -- 172.21.15.33:0/2140814767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aed6ab1fe0 con 0x55aed6a32620 2024-04-23T18:48:50.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c5cbb6640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:48:50.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c57fff640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a429c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802e540 tx=0x7f8c48005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4d42bf9d7d86af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:50.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c377fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-23T18:48:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004610 con 0x55aed6a32620 2024-04-23T18:48:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c50036040 con 0x55aed6a32620 2024-04-23T18:48:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 8498+0+0 (secure 0 0 0) 0x7f8c48003800 con 0x55aed6a32620 2024-04-23T18:48:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8c480047b0 con 0x55aed6a32620 2024-04-23T18:48:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c61405740 10 client.27076 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:50.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aed6ab32c0 con 0x55aed6a32620 2024-04-23T18:48:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.720+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c50036040 con 0x55aed6a32620 2024-04-23T18:48:50.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f8c48023a80 con 0x55aed6a32620 2024-04-23T18:48:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 10 client.27076 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 10 client.27076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 20 client.27076 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:50.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6ab3940 0x55aed6ab5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 4 client.27076 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:48:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aed699f200 con 0x55aed6ab3940 2024-04-23T18:48:50.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6ab3940 0x55aed6ab5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c377fe640 10 client.27076 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:51.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.721+0000 7f8c577fe640 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6ab3940 0x55aed6ab5d60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=955ee5bcbd008465 server_cookie=9313388bd9805ee0 in_seq=0 out_seq=0 2024-04-23T18:48:51.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c377fe640 1 -- 172.21.15.33:0/2140814767 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aed699f200 con 0x55aed6ab3940 2024-04-23T18:48:51.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c377fe640 10 client.27076 handle_command_reply: tid=0 2024-04-23T18:48:51.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 client.27076 shutdown 2024-04-23T18:48:51.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 msgr2=0x55aed6a429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:51.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a429c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802e540 tx=0x7f8c48005f80 comp rx=0 tx=0).stop 2024-04-23T18:48:51.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 shutdown_connections 2024-04-23T18:48:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55aed6ab3940 0x55aed6ab5d60 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:51.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aed6a3d800 0x55aed6ab1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c377fe640 0 client.27076 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55aed6a32f40 0x55aed6a42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 --2- 172.21.15.33:0/2140814767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aed6a32620 0x55aed6a429c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 >> 172.21.15.33:0/2140814767 conn(0x55aed6a2eb30 msgr2=0x55aed6a3a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 shutdown_connections 2024-04-23T18:48:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 1 -- 172.21.15.33:0/2140814767 wait complete. 2024-04-23T18:48:51.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.725+0000 7f8c61405740 20 client.27076 trim_cache size 0 max 16384 2024-04-23T18:48:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:48:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:48:52.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:48:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:48:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 msgr2=0x7f8c400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 0x7f8c400a6300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8c5002f500 tx=0x7f8c500095c0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 msgr2=0x7f8c58103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c00e9f0 tx=0x7f8c4c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 shutdown_connections 2024-04-23T18:48:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c400a3e40 0x7f8c400a6300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c581035b0 0x7f8c581a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c58106170 0x7f8c58103070 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 --2- 172.21.15.33:0/4150363176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c581010f0 0x7f8c58102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 >> 172.21.15.33:0/4150363176 conn(0x7f8c58077fd0 msgr2=0x7f8c58069970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 shutdown_connections 2024-04-23T18:48:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:46:55.726+0000 7f8c5e619640 1 -- 172.21.15.33:0/4150363176 wait complete. 2024-04-23T18:48:52.681 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:43 smithi122 ceph-mon[32546]: pgmap v712: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:52.690 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:45 smithi122 ceph-mon[32546]: pgmap v713: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:48:52.779 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:47 smithi122 ceph-mon[32546]: pgmap v714: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:52.788 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:49 smithi122 ceph-mon[32546]: pgmap v715: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:52.848 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:51 smithi122 ceph-mon[32546]: pgmap v716: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:52.901 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:43 smithi106 ceph-mon[32716]: pgmap v712: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:52.909 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:45 smithi106 ceph-mon[32716]: pgmap v713: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:48:52.922 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:47 smithi106 ceph-mon[32716]: pgmap v714: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:52.930 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:49 smithi106 ceph-mon[32716]: pgmap v715: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:52.938 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:51 smithi106 ceph-mon[32716]: pgmap v716: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:48:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:52.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:52.964 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:53.161 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:53.227 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: pgmap v652: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:53.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:53.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:53.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.448 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.509 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:48:53.592 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:48:53.618 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.626 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.785 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:48:53.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:48:53.801 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:53.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:48:54.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:48:54.063 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:48:54.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:56 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:48:54.082 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:57 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:48:54.091 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:57 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:48:54.138 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:57 smithi033 ceph-mon[29274]: pgmap v653: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:48:54.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:57 smithi033 ceph-mon[29274]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:48:54.259 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:46:59 smithi033 ceph-mon[29274]: pgmap v654: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:48:54.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:01 smithi033 ceph-mon[29274]: pgmap v655: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-23T18:48:54.413 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:03 smithi033 ceph-mon[29274]: pgmap v656: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:48:54.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:05 smithi033 ceph-mon[29274]: pgmap v657: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:48:54.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:07 smithi033 ceph-mon[29274]: pgmap v658: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 51 KiB/s wr, 29 op/s 2024-04-23T18:48:54.480 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:53 smithi122 ceph-mon[32546]: pgmap v717: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:54.537 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:53 smithi106 ceph-mon[32716]: pgmap v717: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:48:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 -- 172.21.15.33:0/1855569861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c1021c0 msgr2=0x7f9d7c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 --2- 172.21.15.33:0/1855569861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1025a0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009ca0 tx=0x7f9d7802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:48:54.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 -- 172.21.15.33:0/1855569861 shutdown_connections 2024-04-23T18:48:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 --2- 172.21.15.33:0/1855569861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c107290 0x7f9d7c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 --2- 172.21.15.33:0/1855569861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c102b70 0x7f9d7c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:54.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 --2- 172.21.15.33:0/1855569861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1025a0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 -- 172.21.15.33:0/1855569861 >> 172.21.15.33:0/1855569861 conn(0x7f9d7c0fda20 msgr2=0x7f9d7c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 -- 172.21.15.33:0/1855569861 shutdown_connections 2024-04-23T18:48:55.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.826+0000 7f9d84744640 1 -- 172.21.15.33:0/1855569861 wait complete. 2024-04-23T18:48:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 Processor -- start 2024-04-23T18:48:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 -- start start 2024-04-23T18:48:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c19cfb0 0x7f9d7c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1121f0 con 0x7f9d7c107290 2024-04-23T18:48:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c112070 con 0x7f9d7c1021c0 2024-04-23T18:48:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d84744640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c112370 con 0x7f9d7c19cfb0 2024-04-23T18:48:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d81cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.827+0000 7f9d81cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42930/0 (socket says 172.21.15.33:42930) 2024-04-23T18:48:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 -- 172.21.15.33:0/879356025 learned_addr learned my addr 172.21.15.33:0/879356025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c19cfb0 0x7f9d7c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d824e0640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:55.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 -- 172.21.15.33:0/879356025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c19cfb0 msgr2=0x7f9d7c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c19cfb0 0x7f9d7c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 -- 172.21.15.33:0/879356025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 msgr2=0x7f9d7c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:55.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 -- 172.21.15.33:0/879356025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c19daa0 con 0x7f9d7c107290 2024-04-23T18:48:55.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d824e0640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:48:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e9f0 tx=0x7f9d6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:55.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d737fe640 1 -- 172.21.15.33:0/879356025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c00ccc0 con 0x7f9d7c107290 2024-04-23T18:48:55.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c19dd90 con 0x7f9d7c107290 2024-04-23T18:48:55.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d737fe640 1 -- 172.21.15.33:0/879356025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6c0054d0 con 0x7f9d7c107290 2024-04-23T18:48:55.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d737fe640 1 -- 172.21.15.33:0/879356025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c010430 con 0x7f9d7c107290 2024-04-23T18:48:55.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.828+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a5530 con 0x7f9d7c107290 2024-04-23T18:48:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.831+0000 7f9d737fe640 1 -- 172.21.15.33:0/879356025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d6c0046c0 con 0x7f9d7c107290 2024-04-23T18:48:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.831+0000 7f9d8745d740 1 Processor -- start 2024-04-23T18:48:56.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 -- start start 2024-04-23T18:48:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b31f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:56.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31fc800 0x5591b31febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591b3156970 con 0x5591b31f1f40 2024-04-23T18:48:56.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5591b30f34a0 con 0x5591b31fc800 2024-04-23T18:48:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d8745d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5591b3152930 con 0x5591b31f1620 2024-04-23T18:48:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d737fe640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 0x7f9d680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42938/0 (socket says 172.21.15.33:42938) 2024-04-23T18:48:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 -- 172.21.15.33:0/3771296911 learned_addr learned my addr 172.21.15.33:0/3771296911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b31f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31fc800 0x5591b31febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 0x7f9d680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d737fe640 1 -- 172.21.15.33:0/879356025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 8586+0+0 (secure 0 0 0) 0x7f9d6c0d0ca0 con 0x7f9d7c107290 2024-04-23T18:48:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 -- 172.21.15.33:0/3771296911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 msgr2=0x5591b31f1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b31f1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:56.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 -- 172.21.15.33:0/3771296911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31fc800 msgr2=0x5591b31febf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31fc800 0x5591b31febf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.832+0000 7f9d824e0640 1 -- 172.21.15.33:0/3771296911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591b2e69210 con 0x5591b31f1f40 2024-04-23T18:48:56.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.833+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9d78035000 tx=0x7f9d780093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb7d0ad32ee9cee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:56.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.833+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 0x7f9d680a6280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005f60 tx=0x7f9d74005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:56.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.833+0000 7f9d70ff9640 1 -- 172.21.15.33:0/3771296911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78031cf0 con 0x5591b31f1f40 2024-04-23T18:48:56.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.833+0000 7f9d70ff9640 1 -- 172.21.15.33:0/3771296911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d78036040 con 0x5591b31f1f40 2024-04-23T18:48:57.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.833+0000 7f9d70ff9640 1 -- 172.21.15.33:0/3771296911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78003e60 con 0x5591b31f1f40 2024-04-23T18:48:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- 172.21.15.33:0/3771296911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 msgr2=0x5591b31fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9d78035000 tx=0x7f9d780093e0 comp rx=0 tx=0).stop 2024-04-23T18:48:57.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- 172.21.15.33:0/3771296911 shutdown_connections 2024-04-23T18:48:57.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31fc800 0x5591b31febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:57.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b31fc2c0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3771296911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b31f1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- 172.21.15.33:0/3771296911 >> 172.21.15.33:0/3771296911 conn(0x5591b31edb30 msgr2=0x5591b31edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- 172.21.15.33:0/3771296911 shutdown_connections 2024-04-23T18:48:57.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- 172.21.15.33:0/3771296911 wait complete. 2024-04-23T18:48:57.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 -- start start 2024-04-23T18:48:57.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 Processor -- start 2024-04-23T18:48:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.834+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b31f9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b31f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591b31ff3d0 con 0x5591b31f1620 2024-04-23T18:48:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5591b30f10b0 con 0x5591b31f1f40 2024-04-23T18:48:57.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5591b31ff550 con 0x5591b31fc800 2024-04-23T18:48:57.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 10 client.0 _ll_register_callbacks cb 0x5591b30ded30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:48:58.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:58.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42948/0 (socket says 172.21.15.33:42948) 2024-04-23T18:48:58.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 -- 172.21.15.33:0/2041857606 learned_addr learned my addr 172.21.15.33:0/2041857606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:48:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b31f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b31f9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 -- 172.21.15.33:0/2041857606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 msgr2=0x5591b31f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b31f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:58.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 -- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 msgr2=0x5591b31f9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:48:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b31f9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 -- 172.21.15.33:0/2041857606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591b2f35620 con 0x5591b31f1620 2024-04-23T18:48:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d824e0640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009a60 tx=0x7f9d7802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd69210f4c51f095 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:48:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:48:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78052020 con 0x5591b31f1620 2024-04-23T18:48:58.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8586+0+0 (secure 0 0 0) 0x7f9d78031cf0 con 0x5591b31f1620 2024-04-23T18:48:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.835+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d78036040 con 0x5591b31f1620 2024-04-23T18:48:58.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9d7805d050 con 0x5591b31f1620 2024-04-23T18:48:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 10 client.17808 fetch_fsmap learned FSMap version 41 2024-04-23T18:48:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5591b31ff3d0 con 0x5591b31f1620 2024-04-23T18:48:58.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d78036040 con 0x5591b31f1620 2024-04-23T18:48:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f9d7804d030 con 0x5591b31f1620 2024-04-23T18:48:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 10 client.17808 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:48:58.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:48:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 20 client.17808 populate_metadata read hostname 'smithi033' 2024-04-23T18:48:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b320c3a0 0x5591b320e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 4 client.17808 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:48:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591b3164390 con 0x5591b320c3a0 2024-04-23T18:48:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.836+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b320c3a0 0x5591b320e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:48:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.837+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b320c3a0 0x5591b320e790 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a6837f37700f78 server_cookie=830152218b6b341b in_seq=0 out_seq=0 2024-04-23T18:48:59.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.837+0000 7f9d65ffb640 10 client.17808 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.837+0000 7f9d65ffb640 1 -- 172.21.15.33:0/2041857606 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591b3164390 con 0x5591b320c3a0 2024-04-23T18:48:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d65ffb640 10 client.17808 handle_command_reply: tid=0 2024-04-23T18:48:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 client.17808 shutdown 2024-04-23T18:48:59.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 msgr2=0x5591b31fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:48:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009a60 tx=0x7f9d7802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:48:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 shutdown_connections 2024-04-23T18:48:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b320c3a0 0x5591b320e790 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b31f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:59.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b31f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d65ffb640 0 client.17808 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:48:59.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2041857606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1620 0x5591b31fc110 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:48:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 >> 172.21.15.33:0/2041857606 conn(0x5591b31edb30 msgr2=0x5591b31f3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:48:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 shutdown_connections 2024-04-23T18:48:59.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 1 -- 172.21.15.33:0/2041857606 wait complete. 2024-04-23T18:48:59.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.838+0000 7f9d8745d740 20 client.17808 trim_cache size 0 max 16384 2024-04-23T18:48:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 Processor -- start 2024-04-23T18:48:59.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 -- start start 2024-04-23T18:48:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1620 0x5591b31fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:48:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b3214480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1af710 con 0x5591b31f1f40 2024-04-23T18:49:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ad4d0 con 0x5591b31f1620 2024-04-23T18:49:00.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d8745d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1af040 con 0x5591b31fc800 2024-04-23T18:49:00.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d81cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d81cdf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42960/0 (socket says 172.21.15.33:42960) 2024-04-23T18:49:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d81cdf640 1 -- 172.21.15.33:0/2080421934 learned_addr learned my addr 172.21.15.33:0/2080421934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.853+0000 7f9d824e0640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1620 0x5591b31fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b3214480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 -- 172.21.15.33:0/2080421934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 msgr2=0x5591b3214480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:00.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b3214480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 -- 172.21.15.33:0/2080421934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1620 msgr2=0x5591b31fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1620 0x5591b31fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:00.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 -- 172.21.15.33:0/2080421934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591b31f28e0 con 0x5591b31f1f40 2024-04-23T18:49:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b3214480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c099000 tx=0x7f9d6c0671a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd965a853e14a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d67fff640 1 -- 172.21.15.33:0/2080421934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c09d2a0 con 0x5591b31f1f40 2024-04-23T18:49:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d67fff640 1 -- 172.21.15.33:0/2080421934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6c0a0f50 con 0x5591b31f1f40 2024-04-23T18:49:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d67fff640 1 -- 172.21.15.33:0/2080421934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0ce5d0 con 0x5591b31f1f40 2024-04-23T18:49:00.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d8745d740 1 -- 172.21.15.33:0/2080421934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 msgr2=0x5591b3213f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.854+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c099000 tx=0x7f9d6c0671a0 comp rx=0 tx=0).stop 2024-04-23T18:49:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- 172.21.15.33:0/2080421934 shutdown_connections 2024-04-23T18:49:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31fc800 0x5591b3214480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31f1f40 0x5591b3213f40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- 172.21.15.33:0/2080421934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1620 0x5591b31fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:01.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- 172.21.15.33:0/2080421934 >> 172.21.15.33:0/2080421934 conn(0x5591b31edb30 msgr2=0x5591b31eccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- 172.21.15.33:0/2080421934 shutdown_connections 2024-04-23T18:49:01.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- 172.21.15.33:0/2080421934 wait complete. 2024-04-23T18:49:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- start start 2024-04-23T18:49:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 Processor -- start 2024-04-23T18:49:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b32049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b3201d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ae2a0 con 0x5591b31fc800 2024-04-23T18:49:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1adba0 con 0x5591b31f1f40 2024-04-23T18:49:01.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c1ae970 con 0x5591b31f1620 2024-04-23T18:49:01.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d8745d740 10 client.0 _ll_register_callbacks cb 0x7f9d78009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d82ce1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d82ce1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42966/0 (socket says 172.21.15.33:42966) 2024-04-23T18:49:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d82ce1640 1 -- 172.21.15.33:0/3167255320 learned_addr learned my addr 172.21.15.33:0/3167255320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.855+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b32049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d81cdf640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b3201d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 -- 172.21.15.33:0/3167255320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 msgr2=0x5591b32049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b32049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:01.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 -- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 msgr2=0x5591b3201d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b3201d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 -- 172.21.15.33:0/3167255320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591b3202920 con 0x5591b31fc800 2024-04-23T18:49:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d82ce1640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9d74011f80 tx=0x7f9d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb6167198437e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:02.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:02.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400fc60 con 0x5591b31fc800 2024-04-23T18:49:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 8586+0+0 (secure 0 0 0) 0x7f9d7400fe00 con 0x5591b31fc800 2024-04-23T18:49:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d6c0a0f50 con 0x5591b31fc800 2024-04-23T18:49:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9d74005020 con 0x5591b31fc800 2024-04-23T18:49:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d8745d740 10 client.17814 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:02.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5591b32725d0 con 0x5591b31fc800 2024-04-23T18:49:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.856+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6c0a0f50 con 0x5591b31fc800 2024-04-23T18:49:02.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f9d74032070 con 0x5591b31fc800 2024-04-23T18:49:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 10 client.17814 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 10 client.17814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 20 client.17814 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:02.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b3272dd0 0x5591b32751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 4 client.17814 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:49:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591b315e200 con 0x5591b3272dd0 2024-04-23T18:49:02.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b3272dd0 0x5591b32751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d824e0640 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b3272dd0 0x5591b32751f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7543279bc2b0079f server_cookie=eabf77bd55a25835 in_seq=0 out_seq=0 2024-04-23T18:49:02.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.857+0000 7f9d65ffb640 10 client.17814 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:02.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.864+0000 7f9d65ffb640 1 -- 172.21.15.33:0/3167255320 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591b315e200 con 0x5591b3272dd0 2024-04-23T18:49:02.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.864+0000 7f9d65ffb640 10 client.17814 handle_command_reply: tid=0 2024-04-23T18:49:02.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.864+0000 7f9d8745d740 1 client.17814 shutdown 2024-04-23T18:49:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 msgr2=0x5591b32022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:02.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9d74011f80 tx=0x7f9d740046c0 comp rx=0 tx=0).stop 2024-04-23T18:49:02.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 shutdown_connections 2024-04-23T18:49:02.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5591b3272dd0 0x5591b32751f0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:02.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591b31fc800 0x5591b32022b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591b31f1f40 0x5591b3201d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 --2- 172.21.15.33:0/3167255320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591b31f1620 0x5591b32049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d65ffb640 0 client.17814 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:03.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 >> 172.21.15.33:0/3167255320 conn(0x5591b31edb30 msgr2=0x5591b31f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:03.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 shutdown_connections 2024-04-23T18:49:03.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 1 -- 172.21.15.33:0/3167255320 wait complete. 2024-04-23T18:49:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.865+0000 7f9d8745d740 20 client.17814 trim_cache size 0 max 16384 2024-04-23T18:49:03.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:49:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:49:03.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:49:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:49:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 msgr2=0x7f9d680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 0x7f9d680a6280 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d74005f60 tx=0x7f9d74005f90 comp rx=0 tx=0).stop 2024-04-23T18:49:03.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 msgr2=0x7f9d7c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00e9f0 tx=0x7f9d6c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 shutdown_connections 2024-04-23T18:49:03.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f9d680a3dc0 0x7f9d680a6280 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d7c19cfb0 0x7f9d7c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d7c107290 0x7f9d7c198c30 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 --2- 172.21.15.33:0/879356025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9d7c1021c0 0x7f9d7c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 >> 172.21.15.33:0/879356025 conn(0x7f9d7c0fda20 msgr2=0x7f9d7c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:03.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 shutdown_connections 2024-04-23T18:49:03.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:08.866+0000 7f9d84744640 1 -- 172.21.15.33:0/879356025 wait complete. 2024-04-23T18:49:03.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:09 smithi033 ceph-mon[29274]: pgmap v659: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-23T18:49:03.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:09 smithi033 ceph-mon[29274]: osdmap e125: 12 total, 12 up, 12 in 2024-04-23T18:49:03.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:11 smithi033 ceph-mon[29274]: pgmap v661: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:49:03.986 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:13 smithi033 ceph-mon[29274]: pgmap v662: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:03.994 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:15 smithi033 ceph-mon[29274]: pgmap v663: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:04.070 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:16 smithi033 ceph-mon[29274]: osdmap e126: 12 total, 12 up, 12 in 2024-04-23T18:49:04.077 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:17 smithi033 ceph-mon[29274]: pgmap v665: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-23T18:49:04.153 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:19 smithi033 ceph-mon[29274]: pgmap v666: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 826 B/s wr, 11 op/s 2024-04-23T18:49:04.165 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:55 smithi122 ceph-mon[32546]: pgmap v718: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:04.174 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:57 smithi122 ceph-mon[32546]: pgmap v719: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:04.183 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:58 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:49:04.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: pgmap v720: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:04.265 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.273 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.471 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:49:04.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:49:04.567 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:48:59 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.646 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:00 smithi122 ceph-mon[32546]: pgmap v721: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-23T18:49:04.711 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:00 smithi122 ceph-mon[32546]: pgmap v722: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:49:04.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:02 smithi122 ceph-mon[32546]: pgmap v723: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:04.821 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:55 smithi106 ceph-mon[32716]: pgmap v718: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:04.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:57 smithi106 ceph-mon[32716]: pgmap v719: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:04.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:58 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:49:04.849 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: pgmap v720: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:04.945 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.954 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:04.963 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:05.086 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:05.095 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:49:05.214 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:49:05.279 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:48:59 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:05.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:00 smithi106 ceph-mon[32716]: pgmap v721: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-23T18:49:05.476 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:00 smithi106 ceph-mon[32716]: pgmap v722: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:49:05.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:02 smithi106 ceph-mon[32716]: pgmap v723: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:05.550 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:04 smithi122 ceph-mon[32546]: pgmap v724: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:05.598 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:04 smithi106 ceph-mon[32716]: pgmap v724: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:05.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 -- 172.21.15.33:0/1962011523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854106170 msgr2=0x7f185410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 --2- 172.21.15.33:0/1962011523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854106170 0x7f185410dd40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f184c009ca0 tx=0x7f184c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:49:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 -- 172.21.15.33:0/1962011523 shutdown_connections 2024-04-23T18:49:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 --2- 172.21.15.33:0/1962011523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854106170 0x7f185410dd40 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 --2- 172.21.15.33:0/1962011523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1854101aa0 0x7f1854105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 --2- 172.21.15.33:0/1962011523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18541010f0 0x7f18541014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 -- 172.21.15.33:0/1962011523 >> 172.21.15.33:0/1962011523 conn(0x7f1854077fd0 msgr2=0x7f18540783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 -- 172.21.15.33:0/1962011523 shutdown_connections 2024-04-23T18:49:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.967+0000 7f1859f7d640 1 -- 172.21.15.33:0/1962011523 wait complete. 2024-04-23T18:49:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 Processor -- start 2024-04-23T18:49:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 -- start start 2024-04-23T18:49:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18541010f0 0x7f18541994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1854106170 0x7f1854199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18541110c0 con 0x7f1854199fb0 2024-04-23T18:49:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1854110f40 con 0x7f1854106170 2024-04-23T18:49:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1859f7d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1854111240 con 0x7f18541010f0 2024-04-23T18:49:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.968+0000 7f1853fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37410/0 (socket says 172.21.15.33:37410) 2024-04-23T18:49:06.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 -- 172.21.15.33:0/3645451397 learned_addr learned my addr 172.21.15.33:0/3645451397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:06.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1852ffd640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1854106170 0x7f1854199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f18537fe640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18541010f0 0x7f18541994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 -- 172.21.15.33:0/3645451397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18541010f0 msgr2=0x7f18541994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:06.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18541010f0 0x7f18541994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 -- 172.21.15.33:0/3645451397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1854106170 msgr2=0x7f1854199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1854106170 0x7f1854199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 -- 172.21.15.33:0/3645451397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18541a4d60 con 0x7f1854199fb0 2024-04-23T18:49:06.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1853fff640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f184c005cc0 tx=0x7f184c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1850ff9640 1 -- 172.21.15.33:0/3645451397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c003bf0 con 0x7f1854199fb0 2024-04-23T18:49:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1850ff9640 1 -- 172.21.15.33:0/3645451397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f184c003d90 con 0x7f1854199fb0 2024-04-23T18:49:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1850ff9640 1 -- 172.21.15.33:0/3645451397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184c02fbc0 con 0x7f1854199fb0 2024-04-23T18:49:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18541a4ff0 con 0x7f1854199fb0 2024-04-23T18:49:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.969+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18541a54c0 con 0x7f1854199fb0 2024-04-23T18:49:07.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f1850ff9640 1 -- 172.21.15.33:0/3645451397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f184c02fd60 con 0x7f1854199fb0 2024-04-23T18:49:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 Processor -- start 2024-04-23T18:49:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 -- start start 2024-04-23T18:49:07.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cf540 0x55ded17cf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 0x55ded17e1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ded1734970 con 0x55ded17cfe60 2024-04-23T18:49:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ded16d14a0 con 0x55ded17dec10 2024-04-23T18:49:07.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f185cd71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ded1730930 con 0x55ded17cf540 2024-04-23T18:49:07.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f18537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f18537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37412/0 (socket says 172.21.15.33:37412) 2024-04-23T18:49:07.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f18537fe640 1 -- 172.21.15.33:0/3869202292 learned_addr learned my addr 172.21.15.33:0/3869202292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:07.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f1850ff9640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 0x7f183c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.972+0000 7f1852ffd640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cf540 0x55ded17cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:07.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1852ffd640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 0x55ded17e1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:07.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1853fff640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 0x7f183c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 -- 172.21.15.33:0/3869202292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cf540 msgr2=0x55ded17cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cf540 0x55ded17cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 -- 172.21.15.33:0/3869202292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 msgr2=0x55ded17e1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:07.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 0x55ded17e1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 -- 172.21.15.33:0/3869202292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded1447210 con 0x55ded17cfe60 2024-04-23T18:49:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1852ffd640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 0x55ded17e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1850ff9640 1 -- 172.21.15.33:0/3645451397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f184c0f1f70 con 0x7f1854199fb0 2024-04-23T18:49:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f18537fe640 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f184400bd10 tx=0x7f18440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdb7f9547bf33b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:07.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1853fff640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 0x7f183c0a6280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098d0 tx=0x7f184c08fc90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:07.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1839ffb640 1 -- 172.21.15.33:0/3869202292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844004030 con 0x55ded17cfe60 2024-04-23T18:49:07.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1839ffb640 1 -- 172.21.15.33:0/3869202292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440035f0 con 0x55ded17cfe60 2024-04-23T18:49:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.973+0000 7f1839ffb640 1 -- 172.21.15.33:0/3869202292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844011770 con 0x55ded17cfe60 2024-04-23T18:49:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- 172.21.15.33:0/3869202292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 msgr2=0x55ded17de6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f184400bd10 tx=0x7f18440107a0 comp rx=0 tx=0).stop 2024-04-23T18:49:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- 172.21.15.33:0/3869202292 shutdown_connections 2024-04-23T18:49:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17dec10 0x55ded17e1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17de6d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 --2- 172.21.15.33:0/3869202292 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cf540 0x55ded17cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- 172.21.15.33:0/3869202292 >> 172.21.15.33:0/3869202292 conn(0x55ded17cbb30 msgr2=0x55ded17cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- 172.21.15.33:0/3869202292 shutdown_connections 2024-04-23T18:49:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- 172.21.15.33:0/3869202292 wait complete. 2024-04-23T18:49:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.974+0000 7f185cd71740 1 -- start start 2024-04-23T18:49:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 Processor -- start 2024-04-23T18:49:08.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:08.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ded17e1840 con 0x55ded17cfe60 2024-04-23T18:49:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ded16cf0b0 con 0x55ded17cf540 2024-04-23T18:49:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ded17e19c0 con 0x55ded17dec10 2024-04-23T18:49:08.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f185cd71740 10 client.0 _ll_register_callbacks cb 0x55ded16bcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:08.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:08.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37426/0 (socket says 172.21.15.33:37426) 2024-04-23T18:49:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 -- 172.21.15.33:0/2995172498 learned_addr learned my addr 172.21.15.33:0/2995172498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f1852ffd640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f1852ffd640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 -- 172.21.15.33:0/2995172498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 msgr2=0x55ded17dbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17dbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 -- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 msgr2=0x55ded17de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.975+0000 7f18537fe640 1 -- 172.21.15.33:0/2995172498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded1513620 con 0x55ded17cfe60 2024-04-23T18:49:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f18537fe640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1844005d00 tx=0x7f184400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad347353e830113 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:09.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844029060 con 0x55ded17cfe60 2024-04-23T18:49:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18440035f0 con 0x55ded17cfe60 2024-04-23T18:49:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f1844003a30 con 0x55ded17cfe60 2024-04-23T18:49:09.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1844003fc0 con 0x55ded17cfe60 2024-04-23T18:49:09.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f185cd71740 10 client.17826 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ded17ed790 con 0x55ded17cfe60 2024-04-23T18:49:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440035f0 con 0x55ded17cfe60 2024-04-23T18:49:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.976+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f1844004700 con 0x55ded17cfe60 2024-04-23T18:49:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 10 client.17826 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:09.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 20 client.17826 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded17eeb90 0x55ded17f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:09.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 4 client.17826 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:49:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded1742390 con 0x55ded17eeb90 2024-04-23T18:49:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f1852ffd640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded17eeb90 0x55ded17f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:10.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f1852ffd640 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded17eeb90 0x55ded17f0f80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e2e5525eef76f98 server_cookie=e88fcda4cfd3739 in_seq=0 out_seq=0 2024-04-23T18:49:10.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.977+0000 7f1832ffd640 10 client.17826 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f1832ffd640 1 -- 172.21.15.33:0/2995172498 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ded1742390 con 0x55ded17eeb90 2024-04-23T18:49:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f1832ffd640 10 client.17826 handle_command_reply: tid=0 2024-04-23T18:49:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 client.17826 shutdown 2024-04-23T18:49:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 msgr2=0x55ded17db8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f1844005d00 tx=0x7f184400b950 comp rx=0 tx=0).stop 2024-04-23T18:49:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 shutdown_connections 2024-04-23T18:49:10.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded17eeb90 0x55ded17f0f80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:10.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:10.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cfe60 0x55ded17db8a0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:10.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 --2- 172.21.15.33:0/2995172498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f1832ffd640 0 client.17826 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 >> 172.21.15.33:0/2995172498 conn(0x55ded17cbb30 msgr2=0x55ded17d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:10.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 shutdown_connections 2024-04-23T18:49:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.978+0000 7f185cd71740 1 -- 172.21.15.33:0/2995172498 wait complete. 2024-04-23T18:49:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.979+0000 7f185cd71740 20 client.17826 trim_cache size 0 max 16384 2024-04-23T18:49:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.993+0000 7f185cd71740 1 Processor -- start 2024-04-23T18:49:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.993+0000 7f185cd71740 1 -- start start 2024-04-23T18:49:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:10.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cfe60 0x55ded17cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:10.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17de0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ad540 con 0x55ded17cf540 2024-04-23T18:49:10.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ab300 con 0x55ded17cfe60 2024-04-23T18:49:10.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f185cd71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ace70 con 0x55ded17dec10 2024-04-23T18:49:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37442/0 (socket says 172.21.15.33:37442) 2024-04-23T18:49:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 -- 172.21.15.33:0/1294922508 learned_addr learned my addr 172.21.15.33:0/1294922508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f18537fe640 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cfe60 0x55ded17cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:11.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17de0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 -- 172.21.15.33:0/1294922508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 msgr2=0x55ded17de0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17de0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 -- 172.21.15.33:0/1294922508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cfe60 msgr2=0x55ded17cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cfe60 0x55ded17cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.994+0000 7f1852ffd640 1 -- 172.21.15.33:0/1294922508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded17da850 con 0x55ded17cf540 2024-04-23T18:49:11.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f1852ffd640 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f18480137f0 tx=0x7f184800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4380f76681e2016 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f18397fa640 1 -- 172.21.15.33:0/1294922508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004020 con 0x55ded17cf540 2024-04-23T18:49:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f18397fa640 1 -- 172.21.15.33:0/1294922508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18480041c0 con 0x55ded17cf540 2024-04-23T18:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f18397fa640 1 -- 172.21.15.33:0/1294922508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480117f0 con 0x55ded17cf540 2024-04-23T18:49:11.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 -- 172.21.15.33:0/1294922508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 msgr2=0x55ded17ea700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:11.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f18480137f0 tx=0x7f184800b410 comp rx=0 tx=0).stop 2024-04-23T18:49:11.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 -- 172.21.15.33:0/1294922508 shutdown_connections 2024-04-23T18:49:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17dec10 0x55ded17de0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cfe60 0x55ded17cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:11.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 --2- 172.21.15.33:0/1294922508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17cf540 0x55ded17ea700 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 -- 172.21.15.33:0/1294922508 >> 172.21.15.33:0/1294922508 conn(0x55ded17cbb30 msgr2=0x55ded17d6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 -- 172.21.15.33:0/1294922508 shutdown_connections 2024-04-23T18:49:11.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.995+0000 7f185cd71740 1 -- 172.21.15.33:0/1294922508 wait complete. 2024-04-23T18:49:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 -- start start 2024-04-23T18:49:11.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 Processor -- start 2024-04-23T18:49:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17e7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cfe60 0x55ded17e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:12.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ac0d0 con 0x55ded17dec10 2024-04-23T18:49:12.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ab9d0 con 0x55ded17cf540 2024-04-23T18:49:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18541ac7a0 con 0x55ded17cfe60 2024-04-23T18:49:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f185cd71740 10 client.0 _ll_register_callbacks cb 0x7f1844003d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f1852ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f1852ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37450/0 (socket says 172.21.15.33:37450) 2024-04-23T18:49:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f1852ffd640 1 -- 172.21.15.33:0/159971632 learned_addr learned my addr 172.21.15.33:0/159971632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:12.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f18537fe640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cfe60 0x55ded17e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:12.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.996+0000 7f1852ffd640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 -- 172.21.15.33:0/159971632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cfe60 msgr2=0x55ded17e24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cfe60 0x55ded17e24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 -- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 msgr2=0x55ded17e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17e7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 -- 172.21.15.33:0/159971632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded17cfc90 con 0x55ded17dec10 2024-04-23T18:49:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1852ffd640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f18480114e0 tx=0x7f1848015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dbfbe0397c1e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1832ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004370 con 0x55ded17dec10 2024-04-23T18:49:12.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f18480035f0 con 0x55ded17dec10 2024-04-23T18:49:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18480041c0 con 0x55ded17dec10 2024-04-23T18:49:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1848003830 con 0x55ded17dec10 2024-04-23T18:49:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f185cd71740 10 client.17832 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ded1854ed0 con 0x55ded17dec10 2024-04-23T18:49:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.997+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18480041c0 con 0x55ded17dec10 2024-04-23T18:49:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f1848021040 con 0x55ded17dec10 2024-04-23T18:49:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 10 client.17832 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 20 client.17832 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:13.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded1855510 0x55ded1857930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 4 client.17832 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:49:13.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ded173c200 con 0x55ded1855510 2024-04-23T18:49:13.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f18537fe640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded1855510 0x55ded1857930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:13.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f1832ffd640 10 client.17832 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:21.998+0000 7f18537fe640 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded1855510 0x55ded1857930 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7817a6b4d152eff server_cookie=78a12c15582782c in_seq=0 out_seq=0 2024-04-23T18:49:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f1832ffd640 1 -- 172.21.15.33:0/159971632 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ded173c200 con 0x55ded1855510 2024-04-23T18:49:13.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f1832ffd640 10 client.17832 handle_command_reply: tid=0 2024-04-23T18:49:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 client.17832 shutdown 2024-04-23T18:49:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 msgr2=0x55ded17e2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f18480114e0 tx=0x7f1848015040 comp rx=0 tx=0).stop 2024-04-23T18:49:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 shutdown_connections 2024-04-23T18:49:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ded1855510 0x55ded1857930 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:13.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ded17dec10 0x55ded17e2be0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ded17cfe60 0x55ded17e24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f1832ffd640 0 client.17832 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 --2- 172.21.15.33:0/159971632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ded17cf540 0x55ded17e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:13.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 >> 172.21.15.33:0/159971632 conn(0x55ded17cbb30 msgr2=0x55ded17d6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.002+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 shutdown_connections 2024-04-23T18:49:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f185cd71740 1 -- 172.21.15.33:0/159971632 wait complete. 2024-04-23T18:49:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f185cd71740 20 client.17832 trim_cache size 0 max 16384 2024-04-23T18:49:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:49:14.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:49:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:49:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:49:14.215 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:06 smithi122 ceph-mon[32546]: pgmap v725: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-23T18:49:14.286 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:08 smithi122 ceph-mon[32546]: pgmap v726: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-23T18:49:14.351 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:09 smithi122 ceph-mon[32546]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:49:14.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:10 smithi122 ceph-mon[32546]: pgmap v728: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:49:14.576 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:12 smithi122 ceph-mon[32546]: pgmap v729: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:14.672 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:06 smithi106 ceph-mon[32716]: pgmap v725: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-23T18:49:14.710 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:08 smithi106 ceph-mon[32716]: pgmap v726: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-23T18:49:14.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:09 smithi106 ceph-mon[32716]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:49:14.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:10 smithi106 ceph-mon[32716]: pgmap v728: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:49:14.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:12 smithi106 ceph-mon[32716]: pgmap v729: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 msgr2=0x7f183c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 0x7f183c0a6280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f184c0098d0 tx=0x7f184c08fc90 comp rx=0 tx=0).stop 2024-04-23T18:49:14.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 msgr2=0x7f18541a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f184c005cc0 tx=0x7f184c02f380 comp rx=0 tx=0).stop 2024-04-23T18:49:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 shutdown_connections 2024-04-23T18:49:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f183c0a3dc0 0x7f183c0a6280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:14.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1854199fb0 0x7f18541a4560 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1854106170 0x7f1854199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 --2- 172.21.15.33:0/3645451397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18541010f0 0x7f18541994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 >> 172.21.15.33:0/3645451397 conn(0x7f1854077fd0 msgr2=0x7f18540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 shutdown_connections 2024-04-23T18:49:15.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:22.003+0000 7f1859f7d640 1 -- 172.21.15.33:0/3645451397 wait complete. 2024-04-23T18:49:15.383 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:14 smithi122 ceph-mon[32546]: pgmap v730: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:15.439 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:14 smithi106 ceph-mon[32716]: pgmap v730: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:15.448 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:21 smithi033 ceph-mon[29274]: pgmap v667: 97 pgs: 97 active+clean; 15 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:49:15.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:23 smithi033 ceph-mon[29274]: pgmap v668: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:49:15.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:25 smithi033 ceph-mon[29274]: pgmap v669: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:49:15.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:27 smithi033 ceph-mon[29274]: pgmap v670: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 293 B/s wr, 10 op/s 2024-04-23T18:49:15.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:29 smithi033 ceph-mon[29274]: pgmap v671: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:15.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:31 smithi033 ceph-mon[29274]: pgmap v672: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:15.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:33 smithi033 ceph-mon[29274]: pgmap v673: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:15.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/3429265058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28102b70 msgr2=0x7fdc28106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/3429265058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28102b70 0x7fdc28106bb0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009900 tx=0x7fdc1002a410 comp rx=0 tx=0).stop 2024-04-23T18:49:15.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/3429265058 shutdown_connections 2024-04-23T18:49:16.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/3429265058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc28107290 0x7fdc2810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:16.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/3429265058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28102b70 0x7fdc28106bb0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/3429265058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc281021c0 0x7fdc281025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:16.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.104+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/3429265058 >> 172.21.15.33:0/3429265058 conn(0x7fdc280fda20 msgr2=0x7fdc280ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:16.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.105+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/3429265058 shutdown_connections 2024-04-23T18:49:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.105+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/3429265058 wait complete. 2024-04-23T18:49:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.105+0000 7fdc2d8fc640 1 Processor -- start 2024-04-23T18:49:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.105+0000 7fdc2d8fc640 1 -- start start 2024-04-23T18:49:16.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc281021c0 0x7fdc28074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc28107290 0x7fdc28074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28112220 con 0x7fdc28074cd0 2024-04-23T18:49:16.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281120a0 con 0x7fdc28107290 2024-04-23T18:49:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc2d8fc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281123a0 con 0x7fdc281021c0 2024-04-23T18:49:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40146/0 (socket says 172.21.15.33:40146) 2024-04-23T18:49:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 -- 172.21.15.33:0/2820147891 learned_addr learned my addr 172.21.15.33:0/2820147891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc28107290 0x7fdc28074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc281021c0 0x7fdc28074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 -- 172.21.15.33:0/2820147891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc281021c0 msgr2=0x7fdc28074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:16.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc281021c0 0x7fdc28074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 -- 172.21.15.33:0/2820147891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc28107290 msgr2=0x7fdc28074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc28107290 0x7fdc28074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 -- 172.21.15.33:0/2820147891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc281a6fb0 con 0x7fdc28074cd0 2024-04-23T18:49:17.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.106+0000 7fdc277fe640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800eaa0 tx=0x7fdc1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.107+0000 7fdc2c8fa640 1 -- 172.21.15.33:0/2820147891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800cdb0 con 0x7fdc28074cd0 2024-04-23T18:49:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.107+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc281a72a0 con 0x7fdc28074cd0 2024-04-23T18:49:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.107+0000 7fdc2c8fa640 1 -- 172.21.15.33:0/2820147891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc18005020 con 0x7fdc28074cd0 2024-04-23T18:49:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.107+0000 7fdc2c8fa640 1 -- 172.21.15.33:0/2820147891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180116d0 con 0x7fdc28074cd0 2024-04-23T18:49:17.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.107+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281a7810 con 0x7fdc28074cd0 2024-04-23T18:49:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc2c8fa640 1 -- 172.21.15.33:0/2820147891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc180054a0 con 0x7fdc28074cd0 2024-04-23T18:49:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 Processor -- start 2024-04-23T18:49:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 -- start start 2024-04-23T18:49:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b2970bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d9b28c8970 con 0x55d9b2963f40 2024-04-23T18:49:17.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d9b28654a0 con 0x55d9b296e800 2024-04-23T18:49:17.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.109+0000 7fdc3060c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d9b28c4930 con 0x55d9b2963620 2024-04-23T18:49:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc2c8fa640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:17.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:17.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc267fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:17.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40154/0 (socket says 172.21.15.33:40154) 2024-04-23T18:49:17.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 -- 172.21.15.33:0/959661146 learned_addr learned my addr 172.21.15.33:0/959661146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:17.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc267fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:37756/0 (socket says 172.21.15.33:37756) 2024-04-23T18:49:17.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc277fe640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc267fc640 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b2970bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc2c8fa640 1 -- 172.21.15.33:0/2820147891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fdc180d08d0 con 0x7fdc28074cd0 2024-04-23T18:49:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 -- 172.21.15.33:0/959661146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 msgr2=0x55d9b2963a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 -- 172.21.15.33:0/959661146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 msgr2=0x55d9b2970bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:18.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b2970bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:18.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 -- 172.21.15.33:0/959661146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b25db210 con 0x55d9b2963f40 2024-04-23T18:49:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc267fc640 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:49:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bcd0 tx=0x7fdc200107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca5fa8d773e27b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc277fe640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 0x7fdc140a62b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdc180214c0 tx=0x7fdc1806b260 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:18.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc1dffb640 1 -- 172.21.15.33:0/959661146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20005220 con 0x55d9b2963f40 2024-04-23T18:49:18.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.110+0000 7fdc1dffb640 1 -- 172.21.15.33:0/959661146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc20004c50 con 0x55d9b2963f40 2024-04-23T18:49:18.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc1dffb640 1 -- 172.21.15.33:0/959661146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc200116a0 con 0x55d9b2963f40 2024-04-23T18:49:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 -- 172.21.15.33:0/959661146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 msgr2=0x55d9b296e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000bcd0 tx=0x7fdc200107a0 comp rx=0 tx=0).stop 2024-04-23T18:49:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 -- 172.21.15.33:0/959661146 shutdown_connections 2024-04-23T18:49:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b2970bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296e2c0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 --2- 172.21.15.33:0/959661146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2963a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 -- 172.21.15.33:0/959661146 >> 172.21.15.33:0/959661146 conn(0x55d9b295fb30 msgr2=0x55d9b295ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:18.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 -- 172.21.15.33:0/959661146 shutdown_connections 2024-04-23T18:49:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.111+0000 7fdc3060c740 1 -- 172.21.15.33:0/959661146 wait complete. 2024-04-23T18:49:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 -- start start 2024-04-23T18:49:18.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 Processor -- start 2024-04-23T18:49:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b296e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:18.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b296b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:18.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d9b2971430 con 0x55d9b2963f40 2024-04-23T18:49:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d9b28630b0 con 0x55d9b296e800 2024-04-23T18:49:18.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d9b29715b0 con 0x55d9b2963620 2024-04-23T18:49:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc3060c740 10 client.0 _ll_register_callbacks cb 0x55d9b2850d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40158/0 (socket says 172.21.15.33:40158) 2024-04-23T18:49:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc26ffd640 1 -- 172.21.15.33:0/879637144 learned_addr learned my addr 172.21.15.33:0/879637144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.112+0000 7fdc267fc640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b296e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:19.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc267fc640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b296b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 -- 172.21.15.33:0/879637144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 msgr2=0x55d9b296e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b296e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:19.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 -- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 msgr2=0x55d9b296b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b296b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:19.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 -- 172.21.15.33:0/879637144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b26a7620 con 0x55d9b2963f40 2024-04-23T18:49:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc267fc640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b296b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:19.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005690 tx=0x7fdc200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea4f96667524b76 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20005010 con 0x55d9b2963f40 2024-04-23T18:49:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fdc200051b0 con 0x55d9b2963f40 2024-04-23T18:49:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc20004c50 con 0x55d9b2963f40 2024-04-23T18:49:19.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdc200053b0 con 0x55d9b2963f40 2024-04-23T18:49:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc3060c740 10 client.17844 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d9b297d360 con 0x55d9b2963f40 2024-04-23T18:49:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.113+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc20004c50 con 0x55d9b2963f40 2024-04-23T18:49:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fdc20011440 con 0x55d9b2963f40 2024-04-23T18:49:19.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 10 client.17844 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 10 client.17844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 20 client.17844 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b297e760 0x55d9b2980b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 4 client.17844 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:49:19.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9b28d6390 con 0x55d9b297e760 2024-04-23T18:49:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc267fc640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b297e760 0x55d9b2980b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:19.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc267fc640 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b297e760 0x55d9b2980b50 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12befa00a522da86 server_cookie=e6a8c7a48af6a1c9 in_seq=0 out_seq=0 2024-04-23T18:49:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.114+0000 7fdc06ffd640 10 client.17844 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:20.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc06ffd640 1 -- 172.21.15.33:0/879637144 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9b28d6390 con 0x55d9b297e760 2024-04-23T18:49:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc06ffd640 10 client.17844 handle_command_reply: tid=0 2024-04-23T18:49:20.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc3060c740 1 client.17844 shutdown 2024-04-23T18:49:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 msgr2=0x55d9b296b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:20.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005690 tx=0x7fdc200048f0 comp rx=0 tx=0).stop 2024-04-23T18:49:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 shutdown_connections 2024-04-23T18:49:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.115+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b297e760 0x55d9b2980b50 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b296b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b296b460 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc06ffd640 0 client.17844 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 --2- 172.21.15.33:0/879637144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b296e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 >> 172.21.15.33:0/879637144 conn(0x55d9b295fb30 msgr2=0x55d9b2965860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 shutdown_connections 2024-04-23T18:49:20.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 1 -- 172.21.15.33:0/879637144 wait complete. 2024-04-23T18:49:20.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.116+0000 7fdc3060c740 20 client.17844 trim_cache size 0 max 16384 2024-04-23T18:49:20.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 Processor -- start 2024-04-23T18:49:20.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 -- start start 2024-04-23T18:49:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b2963620 0x55d9b297a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963f40 0x55d9b295eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:20.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b1950 con 0x55d9b296e800 2024-04-23T18:49:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281af710 con 0x55d9b2963620 2024-04-23T18:49:20.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc3060c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b1280 con 0x55d9b2963f40 2024-04-23T18:49:20.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40168/0 (socket says 172.21.15.33:40168) 2024-04-23T18:49:20.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 -- 172.21.15.33:0/147051124 learned_addr learned my addr 172.21.15.33:0/147051124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:20.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b2963620 0x55d9b297a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963f40 0x55d9b295eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:21.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 -- 172.21.15.33:0/147051124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963f40 msgr2=0x55d9b295eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963f40 0x55d9b295eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:21.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 -- 172.21.15.33:0/147051124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b2963620 msgr2=0x55d9b297a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:21.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b2963620 0x55d9b297a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.131+0000 7fdc267fc640 1 -- 172.21.15.33:0/147051124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9b296a600 con 0x55d9b296e800 2024-04-23T18:49:21.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc267fc640 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdc10006010 tx=0x7fdc10040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1b7374d8060451 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc1d7fa640 1 -- 172.21.15.33:0/147051124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10035040 con 0x55d9b296e800 2024-04-23T18:49:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc1d7fa640 1 -- 172.21.15.33:0/147051124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10003aa0 con 0x55d9b296e800 2024-04-23T18:49:21.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc1d7fa640 1 -- 172.21.15.33:0/147051124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc100319d0 con 0x55d9b296e800 2024-04-23T18:49:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 -- 172.21.15.33:0/147051124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 msgr2=0x55d9b2986270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:21.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fdc10006010 tx=0x7fdc10040040 comp rx=0 tx=0).stop 2024-04-23T18:49:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 -- 172.21.15.33:0/147051124 shutdown_connections 2024-04-23T18:49:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b296e800 0x55d9b2986270 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963f40 0x55d9b295eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 --2- 172.21.15.33:0/147051124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b2963620 0x55d9b297a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 -- 172.21.15.33:0/147051124 >> 172.21.15.33:0/147051124 conn(0x55d9b295fb30 msgr2=0x55d9b295e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:21.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 -- 172.21.15.33:0/147051124 shutdown_connections 2024-04-23T18:49:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.132+0000 7fdc3060c740 1 -- 172.21.15.33:0/147051124 wait complete. 2024-04-23T18:49:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 -- start start 2024-04-23T18:49:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 Processor -- start 2024-04-23T18:49:22.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2976bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:22.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b29744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b04e0 con 0x55d9b2963f40 2024-04-23T18:49:22.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281afde0 con 0x55d9b296e800 2024-04-23T18:49:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b0bb0 con 0x55d9b2963620 2024-04-23T18:49:22.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc3060c740 10 client.0 _ll_register_callbacks cb 0x7fdc2000bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:22.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:22.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc26ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40182/0 (socket says 172.21.15.33:40182) 2024-04-23T18:49:22.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc26ffd640 1 -- 172.21.15.33:0/2900578845 learned_addr learned my addr 172.21.15.33:0/2900578845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:22.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2976bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.133+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b29744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 -- 172.21.15.33:0/2900578845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 msgr2=0x55d9b2976bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:22.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2976bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:22.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 -- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 msgr2=0x55d9b29744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:22.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b29744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:22.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 -- 172.21.15.33:0/2900578845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9b2974c80 con 0x55d9b2963f40 2024-04-23T18:49:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b29744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc26ffd640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x55d9b296acc0 tx=0x7fdc200055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1203fd003f9eb7c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:22.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20032030 con 0x55d9b2963f40 2024-04-23T18:49:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fdc20022040 con 0x55d9b2963f40 2024-04-23T18:49:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc10003aa0 con 0x55d9b2963f40 2024-04-23T18:49:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdc2000bb20 con 0x55d9b2963f40 2024-04-23T18:49:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 10 client.17850 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d9b29e4b10 con 0x55d9b2963f40 2024-04-23T18:49:22.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc10003aa0 con 0x55d9b2963f40 2024-04-23T18:49:22.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fdc20011b50 con 0x55d9b2963f40 2024-04-23T18:49:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 10 client.17850 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 20 client.17850 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:23.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b29e5170 0x55d9b29e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 4 client.17850 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:49:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.134+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9b28d0200 con 0x55d9b29e5170 2024-04-23T18:49:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.135+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b29e5170 0x55d9b29e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:23.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.135+0000 7fdc06ffd640 10 client.17850 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.135+0000 7fdc267fc640 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b29e5170 0x55d9b29e7590 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60163447a32d5df9 server_cookie=140709a163f6c361 in_seq=0 out_seq=0 2024-04-23T18:49:23.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc06ffd640 1 -- 172.21.15.33:0/2900578845 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9b28d0200 con 0x55d9b29e5170 2024-04-23T18:49:23.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc06ffd640 10 client.17850 handle_command_reply: tid=0 2024-04-23T18:49:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 client.17850 shutdown 2024-04-23T18:49:23.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 msgr2=0x55d9b2973f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:23.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x55d9b296acc0 tx=0x7fdc200055d0 comp rx=0 tx=0).stop 2024-04-23T18:49:23.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 shutdown_connections 2024-04-23T18:49:23.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55d9b29e5170 0x55d9b29e7590 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9b296e800 0x55d9b29744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d9b2963f40 0x55d9b2973f60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc06ffd640 0 client.17850 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:23.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 --2- 172.21.15.33:0/2900578845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9b2963620 0x55d9b2976bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:23.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 >> 172.21.15.33:0/2900578845 conn(0x55d9b295fb30 msgr2=0x55d9b295ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 shutdown_connections 2024-04-23T18:49:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.139+0000 7fdc3060c740 1 -- 172.21.15.33:0/2900578845 wait complete. 2024-04-23T18:49:23.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc3060c740 20 client.17850 trim_cache size 0 max 16384 2024-04-23T18:49:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:49:24.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:49:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:49:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:49:24.346 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:16 smithi122 ceph-mon[32546]: pgmap v731: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:24.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:16 smithi122 ceph-mon[32546]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:49:24.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:18 smithi122 ceph-mon[32546]: pgmap v733: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 919 B/s wr, 12 op/s 2024-04-23T18:49:24.521 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:20 smithi122 ceph-mon[32546]: pgmap v734: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:49:24.574 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:22 smithi122 ceph-mon[32546]: pgmap v735: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:24.631 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:16 smithi106 ceph-mon[32716]: pgmap v731: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:49:24.693 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:16 smithi106 ceph-mon[32716]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:49:24.700 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:18 smithi106 ceph-mon[32716]: pgmap v733: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 919 B/s wr, 12 op/s 2024-04-23T18:49:24.708 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:20 smithi106 ceph-mon[32716]: pgmap v734: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:49:24.761 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:22 smithi106 ceph-mon[32716]: pgmap v735: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:24.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 msgr2=0x7fdc140a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:24.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 0x7fdc140a62b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdc180214c0 tx=0x7fdc1806b260 comp rx=0 tx=0).stop 2024-04-23T18:49:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 msgr2=0x7fdc281a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:25.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800eaa0 tx=0x7fdc1800ef60 comp rx=0 tx=0).stop 2024-04-23T18:49:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 shutdown_connections 2024-04-23T18:49:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc28074cd0 0x7fdc281a68b0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:25.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdc28107290 0x7fdc28074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:25.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 --2- 172.21.15.33:0/2820147891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc281021c0 0x7fdc28074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:25.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 >> 172.21.15.33:0/2820147891 conn(0x7fdc280fda20 msgr2=0x7fdc280fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 shutdown_connections 2024-04-23T18:49:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:35.140+0000 7fdc2d8fc640 1 -- 172.21.15.33:0/2820147891 wait complete. 2024-04-23T18:49:25.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:35 smithi033 ceph-mon[29274]: pgmap v674: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:25.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:37 smithi033 ceph-mon[29274]: pgmap v675: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:49:25.697 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:39 smithi033 ceph-mon[29274]: pgmap v676: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:25.705 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:41 smithi033 ceph-mon[29274]: pgmap v677: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:25.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:43 smithi033 ceph-mon[29274]: pgmap v678: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:25.723 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:45 smithi033 ceph-mon[29274]: pgmap v679: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:25.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:24 smithi122 ceph-mon[32546]: pgmap v736: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:25.988 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:24 smithi106 ceph-mon[32716]: pgmap v736: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 -- 172.21.15.33:0/3266864718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 msgr2=0x7fd48c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 --2- 172.21.15.33:0/3266864718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c100aa0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009900 tx=0x7fd47c02a410 comp rx=0 tx=0).stop 2024-04-23T18:49:26.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 -- 172.21.15.33:0/3266864718 shutdown_connections 2024-04-23T18:49:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 --2- 172.21.15.33:0/3266864718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c107280 0x7fd48c10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 --2- 172.21.15.33:0/3266864718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c101070 0x7fd48c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 --2- 172.21.15.33:0/3266864718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c100aa0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:26.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 -- 172.21.15.33:0/3266864718 >> 172.21.15.33:0/3266864718 conn(0x7fd48c077390 msgr2=0x7fd48c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:26.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 -- 172.21.15.33:0/3266864718 shutdown_connections 2024-04-23T18:49:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 -- 172.21.15.33:0/3266864718 wait complete. 2024-04-23T18:49:26.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.240+0000 7fd49238a640 1 Processor -- start 2024-04-23T18:49:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 -- start start 2024-04-23T18:49:26.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:26.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 0x7fd48c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:26.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c19a730 0x7fd48c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c112210 con 0x7fd48c1006c0 2024-04-23T18:49:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c112090 con 0x7fd48c19a730 2024-04-23T18:49:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd49238a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c112390 con 0x7fd48c107280 2024-04-23T18:49:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40412/0 (socket says 172.21.15.33:40412) 2024-04-23T18:49:27.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48bfff640 1 -- 172.21.15.33:0/609605021 learned_addr learned my addr 172.21.15.33:0/609605021 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd490927640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c19a730 0x7fd48c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 0x7fd48c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48bfff640 1 -- 172.21.15.33:0/609605021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 msgr2=0x7fd48c1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.241+0000 7fd48bfff640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 0x7fd48c1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd48bfff640 1 -- 172.21.15.33:0/609605021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c19a730 msgr2=0x7fd48c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd48bfff640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c19a730 0x7fd48c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd48bfff640 1 -- 172.21.15.33:0/609605021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19b330 con 0x7fd48c1006c0 2024-04-23T18:49:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 0x7fd48c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:27.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd48bfff640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005ab0 tx=0x7fd47c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd4897fa640 1 -- 172.21.15.33:0/609605021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c03f070 con 0x7fd48c1006c0 2024-04-23T18:49:27.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd4897fa640 1 -- 172.21.15.33:0/609605021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47c031d60 con 0x7fd48c1006c0 2024-04-23T18:49:27.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19b5c0 con 0x7fd48c1006c0 2024-04-23T18:49:27.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd4897fa640 1 -- 172.21.15.33:0/609605021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47c035030 con 0x7fd48c1006c0 2024-04-23T18:49:27.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.242+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a7750 con 0x7fd48c1006c0 2024-04-23T18:49:27.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd4897fa640 1 -- 172.21.15.33:0/609605021 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd47c042070 con 0x7fd48c1006c0 2024-04-23T18:49:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 Processor -- start 2024-04-23T18:49:27.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 -- start start 2024-04-23T18:49:27.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a4f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:28.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd4897fa640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 0x7fd4740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:28.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a5002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56535a45a970 con 0x56535a500800 2024-04-23T18:49:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56535a3f74a0 con 0x56535a4f5620 2024-04-23T18:49:28.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd495176740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56535a456930 con 0x56535a4f5f40 2024-04-23T18:49:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:28.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd48bfff640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 0x7fd4740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:28.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40424/0 (socket says 172.21.15.33:40424) 2024-04-23T18:49:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.245+0000 7fd490927640 1 -- 172.21.15.33:0/3000423031 learned_addr learned my addr 172.21.15.33:0/3000423031 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd4897fa640 1 -- 172.21.15.33:0/609605021 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fd47c0f10a0 con 0x7fd48c1006c0 2024-04-23T18:49:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a5002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a4f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 -- 172.21.15.33:0/3000423031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 msgr2=0x56535a5002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a5002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 -- 172.21.15.33:0/3000423031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 msgr2=0x56535a4f5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a4f5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 -- 172.21.15.33:0/3000423031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535a16d210 con 0x56535a500800 2024-04-23T18:49:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a5002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd48bfff640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 0x7fd4740a6300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd47c000c00 tx=0x7fd47c0a7b00 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd490927640 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd484013820 tx=0x7fd4840098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa21b55e0d9159e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd4727fc640 1 -- 172.21.15.33:0/3000423031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484019070 con 0x56535a500800 2024-04-23T18:49:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd4727fc640 1 -- 172.21.15.33:0/3000423031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484011d90 con 0x56535a500800 2024-04-23T18:49:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.246+0000 7fd4727fc640 1 -- 172.21.15.33:0/3000423031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840043b0 con 0x56535a500800 2024-04-23T18:49:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- 172.21.15.33:0/3000423031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 msgr2=0x56535a502bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd484013820 tx=0x7fd4840098d0 comp rx=0 tx=0).stop 2024-04-23T18:49:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- 172.21.15.33:0/3000423031 shutdown_connections 2024-04-23T18:49:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a502bf0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:29.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a5002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 --2- 172.21.15.33:0/3000423031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a4f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- 172.21.15.33:0/3000423031 >> 172.21.15.33:0/3000423031 conn(0x56535a4f1b30 msgr2=0x56535a4f1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:29.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- 172.21.15.33:0/3000423031 shutdown_connections 2024-04-23T18:49:29.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- 172.21.15.33:0/3000423031 wait complete. 2024-04-23T18:49:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.247+0000 7fd495176740 1 -- start start 2024-04-23T18:49:29.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 Processor -- start 2024-04-23T18:49:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a5001a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:29.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 0x56535a4fda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56535a503430 con 0x56535a4f5f40 2024-04-23T18:49:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56535a3f50b0 con 0x56535a500800 2024-04-23T18:49:29.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56535a5035b0 con 0x56535a4f5620 2024-04-23T18:49:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd495176740 10 client.0 _ll_register_callbacks cb 0x56535a3e2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:30.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40432/0 (socket says 172.21.15.33:40432) 2024-04-23T18:49:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2132213555 learned_addr learned my addr 172.21.15.33:0/2132213555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd490927640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 0x56535a4fda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:30.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd490927640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a5001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:30.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2132213555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 msgr2=0x56535a5001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a5001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 msgr2=0x56535a4fda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 0x56535a4fda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2132213555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535a239620 con 0x56535a4f5f40 2024-04-23T18:49:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd490927640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 0x56535a4fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd4800124f0 tx=0x7fd4800129b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd38339074e2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.248+0000 7fd46b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:30.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480004220 con 0x56535a4f5f40 2024-04-23T18:49:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fd4800043c0 con 0x56535a4f5f40 2024-04-23T18:49:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd484011d90 con 0x56535a4f5f40 2024-04-23T18:49:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd480010780 con 0x56535a4f5f40 2024-04-23T18:49:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 10 client.17862 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56535a50f400 con 0x56535a4f5f40 2024-04-23T18:49:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484011d90 con 0x56535a4f5f40 2024-04-23T18:49:31.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fd48000f040 con 0x56535a4f5f40 2024-04-23T18:49:31.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 10 client.17862 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:31.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 20 client.17862 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a510ad0 0x56535a512f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:31.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 4 client.17862 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:49:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.249+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56535a468390 con 0x56535a510ad0 2024-04-23T18:49:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.250+0000 7fd490927640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a510ad0 0x56535a512f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.250+0000 7fd46b7fe640 10 client.17862 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.250+0000 7fd490927640 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a510ad0 0x56535a512f10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6392515727ecb687 server_cookie=a66006a1bb294708 in_seq=0 out_seq=0 2024-04-23T18:49:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd46b7fe640 1 -- 172.21.15.33:0/2132213555 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56535a468390 con 0x56535a510ad0 2024-04-23T18:49:31.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd46b7fe640 10 client.17862 handle_command_reply: tid=0 2024-04-23T18:49:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 client.17862 shutdown 2024-04-23T18:49:31.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 msgr2=0x56535a4fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:31.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd4800124f0 tx=0x7fd4800129b0 comp rx=0 tx=0).stop 2024-04-23T18:49:31.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 shutdown_connections 2024-04-23T18:49:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a510ad0 0x56535a512f10 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:31.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a500800 0x56535a4fda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a4f5f40 0x56535a4fd490 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd46b7fe640 0 client.17862 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:31.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 --2- 172.21.15.33:0/2132213555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a5001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 >> 172.21.15.33:0/2132213555 conn(0x56535a4f1b30 msgr2=0x56535a4f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 shutdown_connections 2024-04-23T18:49:32.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 1 -- 172.21.15.33:0/2132213555 wait complete. 2024-04-23T18:49:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.251+0000 7fd495176740 20 client.17862 trim_cache size 0 max 16384 2024-04-23T18:49:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.266+0000 7fd495176740 1 Processor -- start 2024-04-23T18:49:32.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.266+0000 7fd495176740 1 -- start start 2024-04-23T18:49:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a50c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5f40 0x56535a4fc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:32.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1b18f0 con 0x56535a500800 2024-04-23T18:49:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1af6b0 con 0x56535a4f5f40 2024-04-23T18:49:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd495176740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1b1220 con 0x56535a4f5620 2024-04-23T18:49:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40438/0 (socket says 172.21.15.33:40438) 2024-04-23T18:49:32.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 -- 172.21.15.33:0/1890456421 learned_addr learned my addr 172.21.15.33:0/1890456421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:32.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5f40 0x56535a4fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a50c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 -- 172.21.15.33:0/1890456421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 msgr2=0x56535a50c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a50c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 -- 172.21.15.33:0/1890456421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5f40 msgr2=0x56535a4fc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5f40 0x56535a4fc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 -- 172.21.15.33:0/1890456421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56535a4ed2e0 con 0x56535a500800 2024-04-23T18:49:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd490927640 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd484003f20 tx=0x7fd484015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b0d67ccae20e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:33.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd471ffb640 1 -- 172.21.15.33:0/1890456421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484019040 con 0x56535a500800 2024-04-23T18:49:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd471ffb640 1 -- 172.21.15.33:0/1890456421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4840118e0 con 0x56535a500800 2024-04-23T18:49:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.267+0000 7fd471ffb640 1 -- 172.21.15.33:0/1890456421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48402a410 con 0x56535a500800 2024-04-23T18:49:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- 172.21.15.33:0/1890456421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 msgr2=0x56535a5187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd484003f20 tx=0x7fd484015040 comp rx=0 tx=0).stop 2024-04-23T18:49:33.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- 172.21.15.33:0/1890456421 shutdown_connections 2024-04-23T18:49:33.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a5187e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5f40 0x56535a4fc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 --2- 172.21.15.33:0/1890456421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5620 0x56535a50c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- 172.21.15.33:0/1890456421 >> 172.21.15.33:0/1890456421 conn(0x56535a4f1b30 msgr2=0x56535a4fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:33.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- 172.21.15.33:0/1890456421 shutdown_connections 2024-04-23T18:49:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- 172.21.15.33:0/1890456421 wait complete. 2024-04-23T18:49:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.268+0000 7fd495176740 1 -- start start 2024-04-23T18:49:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 Processor -- start 2024-04-23T18:49:33.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a503a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a503f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1b0480 con 0x56535a500800 2024-04-23T18:49:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1afd80 con 0x56535a4f5620 2024-04-23T18:49:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1b0b50 con 0x56535a4f5f40 2024-04-23T18:49:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd495176740 10 client.0 _ll_register_callbacks cb 0x7fd480009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:33.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40448/0 (socket says 172.21.15.33:40448) 2024-04-23T18:49:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 -- 172.21.15.33:0/1709691463 learned_addr learned my addr 172.21.15.33:0/1709691463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a503a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a503f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 -- 172.21.15.33:0/1709691463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 msgr2=0x56535a503f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a503f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 -- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 msgr2=0x56535a503a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a503a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd490927640 1 -- 172.21.15.33:0/1709691463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56535a4fdda0 con 0x56535a500800 2024-04-23T18:49:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.269+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a503f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:34.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd490927640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd484013650 tx=0x7fd4840098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b20c2949b28592 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:34.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484019040 con 0x56535a500800 2024-04-23T18:49:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7fd48402a820 con 0x56535a500800 2024-04-23T18:49:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4840118e0 con 0x56535a500800 2024-04-23T18:49:34.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fd484011a90 con 0x56535a500800 2024-04-23T18:49:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 10 client.17868 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56535a576830 con 0x56535a500800 2024-04-23T18:49:34.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4840118e0 con 0x56535a500800 2024-04-23T18:49:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7fd484038040 con 0x56535a500800 2024-04-23T18:49:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 10 client.17868 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:35.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 20 client.17868 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a576d90 0x56535a579180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 4 client.17868 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:49:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.270+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56535a462200 con 0x56535a576d90 2024-04-23T18:49:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.271+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a576d90 0x56535a579180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:35.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.271+0000 7fd46b7fe640 10 client.17868 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:35.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.271+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a576d90 0x56535a579180 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3099ff02da02f505 server_cookie=99e217362d3e02a5 in_seq=0 out_seq=0 2024-04-23T18:49:35.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd46b7fe640 1 -- 172.21.15.33:0/1709691463 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56535a462200 con 0x56535a576d90 2024-04-23T18:49:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd46b7fe640 10 client.17868 handle_command_reply: tid=0 2024-04-23T18:49:35.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 client.17868 shutdown 2024-04-23T18:49:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 msgr2=0x56535a56f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:35.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd484013650 tx=0x7fd4840098d0 comp rx=0 tx=0).stop 2024-04-23T18:49:35.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 shutdown_connections 2024-04-23T18:49:35.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56535a576d90 0x56535a579180 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:35.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56535a500800 0x56535a56f010 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:35.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56535a4f5f40 0x56535a503f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:35.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd46b7fe640 0 client.17868 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:35.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 --2- 172.21.15.33:0/1709691463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56535a4f5620 0x56535a503a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 >> 172.21.15.33:0/1709691463 conn(0x56535a4f1b30 msgr2=0x56535a4f0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 shutdown_connections 2024-04-23T18:49:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 1 -- 172.21.15.33:0/1709691463 wait complete. 2024-04-23T18:49:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.275+0000 7fd495176740 20 client.17868 trim_cache size 0 max 16384 2024-04-23T18:49:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:49:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:49:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:49:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:49:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 msgr2=0x7fd4740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 0x7fd4740a6300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd47c000c00 tx=0x7fd47c0a7b00 comp rx=0 tx=0).stop 2024-04-23T18:49:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 msgr2=0x7fd48c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:36.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005ab0 tx=0x7fd47c003c80 comp rx=0 tx=0).stop 2024-04-23T18:49:36.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 shutdown_connections 2024-04-23T18:49:36.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3e40 0x7fd4740a6300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd48c19a730 0x7fd48c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:36.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd48c107280 0x7fd48c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 --2- 172.21.15.33:0/609605021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1006c0 0x7fd48c1a0670 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 >> 172.21.15.33:0/609605021 conn(0x7fd48c077390 msgr2=0x7fd48c105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:36.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 shutdown_connections 2024-04-23T18:49:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:47:48.276+0000 7fd49238a640 1 -- 172.21.15.33:0/609605021 wait complete. 2024-04-23T18:49:36.775 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:47 smithi033 ceph-mon[29274]: pgmap v680: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:36.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:49 smithi033 ceph-mon[29274]: pgmap v681: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:36.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:51 smithi033 ceph-mon[29274]: pgmap v682: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:36.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:54 smithi033 ceph-mon[29274]: pgmap v683: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:36.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:56 smithi033 ceph-mon[29274]: pgmap v684: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:36.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:57 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:49:36.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: pgmap v685: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:37.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:37.074 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:49:37.083 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:49:37.150 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:49:37.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:59 smithi033 ceph-mon[29274]: pgmap v686: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 270 B/s wr, 9 op/s 2024-04-23T18:49:37.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:47:59 smithi033 ceph-mon[29274]: pgmap v687: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:49:37.362 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:26 smithi122 ceph-mon[32546]: pgmap v737: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:37.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:28 smithi122 ceph-mon[32546]: pgmap v738: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 9 op/s 2024-04-23T18:49:37.518 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:30 smithi122 ceph-mon[32546]: pgmap v739: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.554 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:31 smithi122 ceph-mon[32546]: pgmap v740: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:37.609 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:34 smithi122 ceph-mon[32546]: pgmap v741: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.616 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:36 smithi122 ceph-mon[32546]: pgmap v742: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:26 smithi106 ceph-mon[32716]: pgmap v737: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:49:37.719 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:28 smithi106 ceph-mon[32716]: pgmap v738: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 9 op/s 2024-04-23T18:49:37.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:30 smithi106 ceph-mon[32716]: pgmap v739: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.776 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:31 smithi106 ceph-mon[32716]: pgmap v740: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:37.860 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:34 smithi106 ceph-mon[32716]: pgmap v741: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:36 smithi106 ceph-mon[32716]: pgmap v742: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2025173056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb41010f0 msgr2=0x7f3eb41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:37.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2025173056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb41010f0 0x7f3eb41014d0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009900 tx=0x7f3ea402a410 comp rx=0 tx=0).stop 2024-04-23T18:49:38.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2025173056 shutdown_connections 2024-04-23T18:49:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2025173056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb4106170 0x7f3eb410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:38.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2025173056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb4101aa0 0x7f3eb4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2025173056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb41010f0 0x7f3eb41014d0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2025173056 >> 172.21.15.33:0/2025173056 conn(0x7f3eb4077fd0 msgr2=0x7f3eb40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2025173056 shutdown_connections 2024-04-23T18:49:38.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.376+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2025173056 wait complete. 2024-04-23T18:49:38.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 Processor -- start 2024-04-23T18:49:38.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 -- start start 2024-04-23T18:49:38.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 0x7f3eb4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:38.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb419cff0 0x7f3eb419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:38.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41110d0 con 0x7f3eb4106170 2024-04-23T18:49:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4110f50 con 0x7f3eb41010f0 2024-04-23T18:49:38.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eba4d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4111250 con 0x7f3eb419cff0 2024-04-23T18:49:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.377+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55678/0 (socket says 172.21.15.33:55678) 2024-04-23T18:49:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2508092377 learned_addr learned my addr 172.21.15.33:0/2508092377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb419cff0 0x7f3eb419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 0x7f3eb4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:39.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2508092377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb419cff0 msgr2=0x7f3eb419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb419cff0 0x7f3eb419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:39.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2508092377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 msgr2=0x7f3eb4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 0x7f3eb4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2508092377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb419dae0 con 0x7f3eb4106170 2024-04-23T18:49:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 0x7f3eb4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d740 tx=0x7f3ea800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb17fa640 1 -- 172.21.15.33:0/2508092377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8017070 con 0x7f3eb4106170 2024-04-23T18:49:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb17fa640 1 -- 172.21.15.33:0/2508092377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea800de50 con 0x7f3eb4106170 2024-04-23T18:49:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eb17fa640 1 -- 172.21.15.33:0/2508092377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004ce0 con 0x7f3eb4106170 2024-04-23T18:49:39.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb419ddd0 con 0x7f3eb4106170 2024-04-23T18:49:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.378+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a55c0 con 0x7f3eb4106170 2024-04-23T18:49:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3eb17fa640 1 -- 172.21.15.33:0/2508092377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3ea8004e80 con 0x7f3eb4106170 2024-04-23T18:49:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 Processor -- start 2024-04-23T18:49:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 -- start start 2024-04-23T18:49:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 0x5647c36e82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:39.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36e8800 0x5647c36eabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5647c3642970 con 0x5647c36dd620 2024-04-23T18:49:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5647c35df4a0 con 0x5647c36ddf40 2024-04-23T18:49:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647c363e930 con 0x5647c36e8800 2024-04-23T18:49:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.381+0000 7f3eb8a6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55680/0 (socket says 172.21.15.33:55680) 2024-04-23T18:49:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/1450628118 learned_addr learned my addr 172.21.15.33:0/1450628118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:40.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb17fa640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 0x7f3e9c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36e8800 0x5647c36eabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 0x7f3e9c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 0x5647c36e82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:40.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/1450628118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36e8800 msgr2=0x5647c36eabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:40.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36e8800 0x5647c36eabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:40.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb17fa640 1 -- 172.21.15.33:0/2508092377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f3ea80d0190 con 0x7f3eb4106170 2024-04-23T18:49:40.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/1450628118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 msgr2=0x5647c36e82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 0x5647c36e82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:40.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/1450628118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c3355210 con 0x5647c36dd620 2024-04-23T18:49:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 0x5647c36e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3eac005c40 tx=0x7f3eac013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb5ad19a219c04a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/1450628118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac022020 con 0x5647c36dd620 2024-04-23T18:49:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 0x7f3e9c0a6300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3eac023000 tx=0x7f3eac021000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:40.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/1450628118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eac0134e0 con 0x5647c36dd620 2024-04-23T18:49:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.382+0000 7f3e9a7fc640 1 -- 172.21.15.33:0/1450628118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac010aa0 con 0x5647c36dd620 2024-04-23T18:49:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/1450628118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 msgr2=0x5647c36dda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f3eac005c40 tx=0x7f3eac013f50 comp rx=0 tx=0).stop 2024-04-23T18:49:41.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/1450628118 shutdown_connections 2024-04-23T18:49:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36e8800 0x5647c36eabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36ddf40 0x5647c36e82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/1450628118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36dda00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:41.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/1450628118 >> 172.21.15.33:0/1450628118 conn(0x5647c36d9b30 msgr2=0x5647c36d9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/1450628118 shutdown_connections 2024-04-23T18:49:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.383+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/1450628118 wait complete. 2024-04-23T18:49:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 -- start start 2024-04-23T18:49:41.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 Processor -- start 2024-04-23T18:49:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 0x5647c36e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:41.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5647c36eb430 con 0x5647c36ddf40 2024-04-23T18:49:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5647c35dd0b0 con 0x5647c36e8800 2024-04-23T18:49:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647c36eb5b0 con 0x5647c36dd620 2024-04-23T18:49:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3ebd1e2740 10 client.0 _ll_register_callbacks cb 0x5647c35cad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55694/0 (socket says 172.21.15.33:55694) 2024-04-23T18:49:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2523091718 learned_addr learned my addr 172.21.15.33:0/2523091718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2523091718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 msgr2=0x5647c36e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 0x5647c36e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.384+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 0x5647c36e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 msgr2=0x5647c36e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb37fe640 1 -- 172.21.15.33:0/2523091718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c3421620 con 0x5647c36ddf40 2024-04-23T18:49:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 0x5647c36e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3ea809bdc0 tx=0x7f3ea809bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72042fb799a26ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8020040 con 0x5647c36ddf40 2024-04-23T18:49:42.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f3ea8098730 con 0x5647c36ddf40 2024-04-23T18:49:42.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647c3650390 con 0x5647c36ddf40 2024-04-23T18:49:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ea80a0470 con 0x5647c36ddf40 2024-04-23T18:49:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3ebd1e2740 10 client.17880 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5647c36f7430 con 0x5647c36ddf40 2024-04-23T18:49:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5647c3650390 con 0x5647c36ddf40 2024-04-23T18:49:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f3ea809b510 con 0x5647c36ddf40 2024-04-23T18:49:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3ebd1e2740 10 client.17880 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.385+0000 7f3ebd1e2740 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3ebd1e2740 20 client.17880 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c36f88f0 0x5647c36face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:42.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3ebd1e2740 4 client.17880 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:49:42.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647c364a200 con 0x5647c36f88f0 2024-04-23T18:49:42.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c36f88f0 0x5647c36face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:42.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c36f88f0 0x5647c36face0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50cf7d52891ccad6 server_cookie=ef8022ee4cedec3f in_seq=0 out_seq=0 2024-04-23T18:49:42.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.386+0000 7f3e937fe640 10 client.17880 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:42.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3e937fe640 1 -- 172.21.15.33:0/2523091718 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647c364a200 con 0x5647c36f88f0 2024-04-23T18:49:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3e937fe640 10 client.17880 handle_command_reply: tid=0 2024-04-23T18:49:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 client.17880 shutdown 2024-04-23T18:49:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 msgr2=0x5647c36e54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:43.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f3ea809bdc0 tx=0x7f3ea809bb10 comp rx=0 tx=0).stop 2024-04-23T18:49:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 shutdown_connections 2024-04-23T18:49:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c36f88f0 0x5647c36face0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36ddf40 0x5647c36e54e0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:43.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2523091718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36dd620 0x5647c36e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:43.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3e937fe640 0 client.17880 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 >> 172.21.15.33:0/2523091718 conn(0x5647c36d9b30 msgr2=0x5647c36df8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:43.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 shutdown_connections 2024-04-23T18:49:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.387+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2523091718 wait complete. 2024-04-23T18:49:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.388+0000 7f3ebd1e2740 20 client.17880 trim_cache size 0 max 16384 2024-04-23T18:49:43.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.402+0000 7f3ebd1e2740 1 Processor -- start 2024-04-23T18:49:43.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.402+0000 7f3ebd1e2740 1 -- start start 2024-04-23T18:49:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 0x5647c36f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:43.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41af600 con 0x5647c36e8800 2024-04-23T18:49:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ad3c0 con 0x5647c36dd620 2024-04-23T18:49:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41aef30 con 0x5647c36ddf40 2024-04-23T18:49:43.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:43.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55704/0 (socket says 172.21.15.33:55704) 2024-04-23T18:49:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/2090313311 learned_addr learned my addr 172.21.15.33:0/2090313311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:44.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 0x5647c36f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:44.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/2090313311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 msgr2=0x5647c36df730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:44.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36df730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/2090313311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 msgr2=0x5647c36f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 0x5647c36f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 -- 172.21.15.33:0/2090313311 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c36e5e60 con 0x5647c36e8800 2024-04-23T18:49:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 0x5647c36f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:49:44.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0097a0 tx=0x7f3eac0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5d1094d2646f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3e99ffb640 1 -- 172.21.15.33:0/2090313311 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac022040 con 0x5647c36e8800 2024-04-23T18:49:44.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3e99ffb640 1 -- 172.21.15.33:0/2090313311 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eac010d10 con 0x5647c36e8800 2024-04-23T18:49:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.403+0000 7f3e99ffb640 1 -- 172.21.15.33:0/2090313311 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac012ce0 con 0x5647c36e8800 2024-04-23T18:49:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2090313311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 msgr2=0x5647c36e7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0097a0 tx=0x7f3eac0097d0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2090313311 shutdown_connections 2024-04-23T18:49:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36e8800 0x5647c36e7d20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36df730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/2090313311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36dd620 0x5647c36f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:44.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2090313311 >> 172.21.15.33:0/2090313311 conn(0x5647c36d9b30 msgr2=0x5647c36df020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:44.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2090313311 shutdown_connections 2024-04-23T18:49:44.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/2090313311 wait complete. 2024-04-23T18:49:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.404+0000 7f3ebd1e2740 1 -- start start 2024-04-23T18:49:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 Processor -- start 2024-04-23T18:49:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36e1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:45.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ae190 con 0x5647c36dd620 2024-04-23T18:49:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ada90 con 0x5647c36e8800 2024-04-23T18:49:45.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41ae860 con 0x5647c36ddf40 2024-04-23T18:49:45.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3ebd1e2740 10 client.0 _ll_register_callbacks cb 0x7f3ea809bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:45.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55716/0 (socket says 172.21.15.33:55716) 2024-04-23T18:49:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 -- 172.21.15.33:0/3650039606 learned_addr learned my addr 172.21.15.33:0/3650039606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36e1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb8a6d640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 -- 172.21.15.33:0/3650039606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 msgr2=0x5647c36e1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36e1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 -- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 msgr2=0x5647c36e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:45.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb3fff640 1 -- 172.21.15.33:0/3650039606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c36eaf60 con 0x5647c36dd620 2024-04-23T18:49:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.405+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4031440 tx=0x7f3ea40040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6988777c5698b256 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4035040 con 0x5647c36dd620 2024-04-23T18:49:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eac010d10 con 0x5647c36dd620 2024-04-23T18:49:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 8542+0+0 (secure 0 0 0) 0x7f3ea40043d0 con 0x5647c36dd620 2024-04-23T18:49:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3ea4050050 con 0x5647c36dd620 2024-04-23T18:49:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 10 client.17886 fetch_fsmap learned FSMap version 41 2024-04-23T18:49:46.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5647c375ec70 con 0x5647c36dd620 2024-04-23T18:49:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eac010d10 con 0x5647c36dd620 2024-04-23T18:49:46.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) ==== 7753+0+0 (secure 0 0 0) 0x7f3ea40422e0 con 0x5647c36dd620 2024-04-23T18:49:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 10 client.17886 fetch_fsmap finished waiting for FSMap version 41 2024-04-23T18:49:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 20 client.17886 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c375f420 0x5647c3761840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 4 client.17886 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:49:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.406+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647c35c65f0 con 0x5647c375f420 2024-04-23T18:49:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.407+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c375f420 0x5647c3761840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.407+0000 7f3eb37fe640 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c375f420 0x5647c3761840 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9ba2c64548abc31 server_cookie=4c7d57a640169d0a in_seq=0 out_seq=0 2024-04-23T18:49:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:01.407+0000 7f3e937fe640 10 client.17886 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:46.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:01 smithi033 ceph-mon[29274]: pgmap v688: 97 pgs: 97 active+clean; 15 MiB data, 491 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 327 B/s wr, 11 op/s 2024-04-23T18:49:46.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3e937fe640 1 -- 172.21.15.33:0/3650039606 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647c35c65f0 con 0x5647c375f420 2024-04-23T18:49:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3e937fe640 10 client.17886 handle_command_reply: tid=0 2024-04-23T18:49:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 client.17886 shutdown 2024-04-23T18:49:47.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 msgr2=0x5647c36f0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4031440 tx=0x7f3ea40040d0 comp rx=0 tx=0).stop 2024-04-23T18:49:47.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 shutdown_connections 2024-04-23T18:49:47.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5647c375f420 0x5647c3761840 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:47.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.191+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647c36e8800 0x5647c36e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647c36ddf40 0x5647c36e1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 1 --2- 172.21.15.33:0/3650039606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5647c36dd620 0x5647c36f0c90 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:47.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3e937fe640 0 client.17886 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 >> 172.21.15.33:0/3650039606 conn(0x5647c36d9b30 msgr2=0x5647c36e39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 shutdown_connections 2024-04-23T18:49:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 1 -- 172.21.15.33:0/3650039606 wait complete. 2024-04-23T18:49:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.192+0000 7f3ebd1e2740 20 client.17886 trim_cache size 0 max 16384 2024-04-23T18:49:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:49:47.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:49:47.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:49:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:49:47.851 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:38 smithi122 ceph-mon[32546]: pgmap v743: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:49:47.907 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:40 smithi122 ceph-mon[32546]: pgmap v744: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:47.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:42 smithi122 ceph-mon[32546]: pgmap v745: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:47.957 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:44 smithi122 ceph-mon[32546]: pgmap v746: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:47.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:46 smithi122 ceph-mon[32546]: pgmap v747: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:48.050 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:38 smithi106 ceph-mon[32716]: pgmap v743: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:49:48.059 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:40 smithi106 ceph-mon[32716]: pgmap v744: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:48.235 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:42 smithi106 ceph-mon[32716]: pgmap v745: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:48.302 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:44 smithi106 ceph-mon[32716]: pgmap v746: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:48.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:46 smithi106 ceph-mon[32716]: pgmap v747: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:49:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 msgr2=0x7f3e9c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:48.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 0x7f3e9c0a6300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3eac023000 tx=0x7f3eac021000 comp rx=0 tx=0).stop 2024-04-23T18:49:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 msgr2=0x7f3eb4198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d740 tx=0x7f3ea800dc00 comp rx=0 tx=0).stop 2024-04-23T18:49:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 shutdown_connections 2024-04-23T18:49:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e9c0a3e40 0x7f3e9c0a6300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3eb419cff0 0x7f3eb419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106170 0x7f3eb4198cc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 --2- 172.21.15.33:0/2508092377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3eb41010f0 0x7f3eb4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 >> 172.21.15.33:0/2508092377 conn(0x7f3eb4077fd0 msgr2=0x7f3eb4069a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:48.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 shutdown_connections 2024-04-23T18:49:48.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:03.193+0000 7f3eba4d0640 1 -- 172.21.15.33:0/2508092377 wait complete. 2024-04-23T18:49:48.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:03 smithi033 ceph-mon[29274]: pgmap v689: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:48.798 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:49:49.009 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: in thread 7ff020e08700 thread_name:md_log_replay 2024-04-23T18:49:49.106 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:49:49.181 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0321c9d20] 2024-04-23T18:49:49.249 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f17dce038f] 2024-04-23T18:49:49.329 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f17dce3b23] 2024-04-23T18:49:49.397 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 4: (MDLog::_replay_thread()+0x617) [0x55f17dd3e647] 2024-04-23T18:49:49.497 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f17dd47d31] 2024-04-23T18:49:49.572 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff0321bf1ca] 2024-04-23T18:49:49.625 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 7: clone() 2024-04-23T18:49:49.699 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:48:04.150+0000 7ff020e08700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:49:49.761 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: in thread 7ff020e08700 thread_name:md_log_replay 2024-04-23T18:49:49.770 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:49.779 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:49:49.788 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0321c9d20] 2024-04-23T18:49:49.839 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f17dce038f] 2024-04-23T18:49:49.904 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f17dce3b23] 2024-04-23T18:49:49.997 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 4: (MDLog::_replay_thread()+0x617) [0x55f17dd3e647] 2024-04-23T18:49:50.155 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f17dd47d31] 2024-04-23T18:49:50.268 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff0321bf1ca] 2024-04-23T18:49:50.320 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 7: clone() 2024-04-23T18:49:50.367 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:49:50.421 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:50.430 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 0> 2024-04-23T18:48:04.150+0000 7ff020e08700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:49:50.461 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: in thread 7ff020e08700 thread_name:md_log_replay 2024-04-23T18:49:50.493 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:50.501 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:49:50.543 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0321c9d20] 2024-04-23T18:49:50.573 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f17dce038f] 2024-04-23T18:49:50.581 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f17dce3b23] 2024-04-23T18:49:50.590 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 4: (MDLog::_replay_thread()+0x617) [0x55f17dd3e647] 2024-04-23T18:49:50.599 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f17dd47d31] 2024-04-23T18:49:50.607 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff0321bf1ca] 2024-04-23T18:49:50.663 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 7: clone() 2024-04-23T18:49:50.672 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:49:50.888 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:50.989 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 0> 2024-04-23T18:48:04.150+0000 7ff020e08700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:49:51.023 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: in thread 7ff020e08700 thread_name:md_log_replay 2024-04-23T18:49:51.130 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:51.176 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:49:51.185 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff0321c9d20] 2024-04-23T18:49:51.193 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f17dce038f] 2024-04-23T18:49:51.201 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f17dce3b23] 2024-04-23T18:49:51.271 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 4: (MDLog::_replay_thread()+0x617) [0x55f17dd3e647] 2024-04-23T18:49:51.279 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f17dd47d31] 2024-04-23T18:49:51.397 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7ff0321bf1ca] 2024-04-23T18:49:51.444 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 7: clone() 2024-04-23T18:49:51.454 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:49:51.464 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[94324]: 2024-04-23T18:49:51.474 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 podman[95575]: 2024-04-23 18:48:04.21314495 +0000 UTC m=+0.023214176 container died 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T18:49:51.560 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:48 smithi122 ceph-mon[32546]: pgmap v748: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:51.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:48 smithi106 ceph-mon[32716]: pgmap v748: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:51.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:05 smithi033 ceph-mon[29274]: pgmap v690: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:49:51.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:05 smithi033 ceph-mon[29274]: osdmap e127: 12 total, 12 up, 12 in 2024-04-23T18:49:51.779 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 podman[95575]: 2024-04-23 18:48:04.924122728 +0000 UTC m=+0.734191956 container remove 6abaf8447ea9c5922e982ed19c13acf94f01a64e6c27f8c4362c53060656c0fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T18:49:51.864 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:49:51.964 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:05 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:49:52.080 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:07 smithi033 ceph-mon[29274]: pgmap v692: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:49:52.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:09 smithi033 ceph-mon[29274]: pgmap v693: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:49:52.153 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:11 smithi033 ceph-mon[29274]: pgmap v694: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:49:52.162 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:11 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:49:52.218 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:11 smithi033 ceph-mon[29274]: osdmap e128: 12 total, 12 up, 12 in 2024-04-23T18:49:52.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:13 smithi033 ceph-mon[29274]: pgmap v696: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-04-23T18:49:52.286 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:15 smithi033 ceph-mon[29274]: pgmap v697: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 957 B/s wr, 10 op/s 2024-04-23T18:49:52.366 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 4. 2024-04-23T18:49:52.479 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:49:52.488 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:49:52.497 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 podman[95670]: 2024-04-23 18:48:15.388788903 +0000 UTC m=+0.019578211 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:49:52.596 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 podman[95670]: 2024-04-23 18:48:15.533371537 +0000 UTC m=+0.164160826 container create d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T18:49:52.661 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 podman[95670]: 2024-04-23 18:48:15.777986972 +0000 UTC m=+0.408776263 container init d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:49:52.766 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 podman[95670]: 2024-04-23 18:48:15.78071163 +0000 UTC m=+0.411500918 container start d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T18:49:52.869 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 ceph-mds[95690]: -- 172.21.15.33:0/3778020739 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555fcd295a40 con 0x555fcd268c00 2024-04-23T18:49:52.953 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: starting mds.d at 2024-04-23T18:49:53.001 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 bash[95670]: d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 2024-04-23T18:49:53.010 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:48:15 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:49:53.021 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:50 smithi122 ceph-mon[32546]: pgmap v749: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:53.030 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:52 smithi122 ceph-mon[32546]: pgmap v750: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:53.038 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:50 smithi106 ceph-mon[32716]: pgmap v749: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:49:53.047 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:52 smithi106 ceph-mon[32716]: pgmap v750: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:49:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.288+0000 7f73b15a2640 1 -- 172.21.15.33:0/2706668283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1021d0 msgr2=0x7f73ac1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.288+0000 7f73b15a2640 1 --2- 172.21.15.33:0/2706668283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1021d0 0x7f73ac1025b0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f739c009900 tx=0x7f739c02a410 comp rx=0 tx=0).stop 2024-04-23T18:49:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.288+0000 7f73b15a2640 1 -- 172.21.15.33:0/2706668283 shutdown_connections 2024-04-23T18:49:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.288+0000 7f73b15a2640 1 --2- 172.21.15.33:0/2706668283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1072a0 0x7f73ac10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 --2- 172.21.15.33:0/2706668283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac102b80 0x7f73ac106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 --2- 172.21.15.33:0/2706668283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1021d0 0x7f73ac1025b0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- 172.21.15.33:0/2706668283 >> 172.21.15.33:0/2706668283 conn(0x7f73ac0793e0 msgr2=0x7f73ac0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- 172.21.15.33:0/2706668283 shutdown_connections 2024-04-23T18:49:53.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- 172.21.15.33:0/2706668283 wait complete. 2024-04-23T18:49:53.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 Processor -- start 2024-04-23T18:49:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- start start 2024-04-23T18:49:53.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1021d0 0x7f73ac198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:53.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac19cfe0 0x7f73ac19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:53.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1122b0 con 0x7f73ac1072a0 2024-04-23T18:49:53.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac112130 con 0x7f73ac1021d0 2024-04-23T18:49:53.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.289+0000 7f73b15a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac112430 con 0x7f73ac19cfe0 2024-04-23T18:49:54.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32912/0 (socket says 172.21.15.33:32912) 2024-04-23T18:49:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 -- 172.21.15.33:0/3259150989 learned_addr learned my addr 172.21.15.33:0/3259150989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73aaffd640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1021d0 0x7f73ac198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac19cfe0 0x7f73ac19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 -- 172.21.15.33:0/3259150989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac19cfe0 msgr2=0x7f73ac19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:54.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac19cfe0 0x7f73ac19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:54.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 -- 172.21.15.33:0/3259150989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1021d0 msgr2=0x7f73ac198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1021d0 0x7f73ac198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 -- 172.21.15.33:0/3259150989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ac19dad0 con 0x7f73ac1072a0 2024-04-23T18:49:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f739400b450 tx=0x7f739400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:54.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a8ff9640 1 -- 172.21.15.33:0/3259150989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394015020 con 0x7f73ac1072a0 2024-04-23T18:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a8ff9640 1 -- 172.21.15.33:0/3259150989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7394011d40 con 0x7f73ac1072a0 2024-04-23T18:49:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ac19ddc0 con 0x7f73ac1072a0 2024-04-23T18:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73a8ff9640 1 -- 172.21.15.33:0/3259150989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739400f340 con 0x7f73ac1072a0 2024-04-23T18:49:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.290+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ac1a55b0 con 0x7f73ac1072a0 2024-04-23T18:49:54.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 Processor -- start 2024-04-23T18:49:54.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73a8ff9640 1 -- 172.21.15.33:0/3259150989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7394018020 con 0x7f73ac1072a0 2024-04-23T18:49:54.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 -- start start 2024-04-23T18:49:54.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:54.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55794c905970 con 0x55794c9ab800 2024-04-23T18:49:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55794c8a24a0 con 0x55794c9a0620 2024-04-23T18:49:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73b4397740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55794c901930 con 0x55794c9a0f40 2024-04-23T18:49:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32928/0 (socket says 172.21.15.33:32928) 2024-04-23T18:49:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73a3fff640 1 -- 172.21.15.33:0/1460671540 learned_addr learned my addr 172.21.15.33:0/1460671540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73aaffd640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:55.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.293+0000 7f73a8ff9640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 0x7f73980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 0x7f73980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:55.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a8ff9640 1 -- 172.21.15.33:0/3259150989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f73940d00f0 con 0x7f73ac1072a0 2024-04-23T18:49:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 -- 172.21.15.33:0/1460671540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 msgr2=0x55794c9ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:55.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 -- 172.21.15.33:0/1460671540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 msgr2=0x55794c9a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:55.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 -- 172.21.15.33:0/1460671540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c618210 con 0x55794c9ab800 2024-04-23T18:49:55.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:49:55.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f73940cd9a0 tx=0x7f73940047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f072381e73bf9d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:55.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 0x7f73980a6280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a4004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a17fa640 1 -- 172.21.15.33:0/1460671540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739409c250 con 0x55794c9ab800 2024-04-23T18:49:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a17fa640 1 -- 172.21.15.33:0/1460671540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73940a0600 con 0x55794c9ab800 2024-04-23T18:49:55.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.294+0000 7f73a17fa640 1 -- 172.21.15.33:0/1460671540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394010500 con 0x55794c9ab800 2024-04-23T18:49:55.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- 172.21.15.33:0/1460671540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 msgr2=0x55794c9adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:55.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f73940cd9a0 tx=0x7f73940047d0 comp rx=0 tx=0).stop 2024-04-23T18:49:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- 172.21.15.33:0/1460671540 shutdown_connections 2024-04-23T18:49:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9ab800 0x55794c9adbf0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- 172.21.15.33:0/1460671540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- 172.21.15.33:0/1460671540 >> 172.21.15.33:0/1460671540 conn(0x55794c99cb30 msgr2=0x55794c99cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- 172.21.15.33:0/1460671540 shutdown_connections 2024-04-23T18:49:56.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- 172.21.15.33:0/1460671540 wait complete. 2024-04-23T18:49:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- start start 2024-04-23T18:49:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 Processor -- start 2024-04-23T18:49:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9a8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9ab800 0x55794c9a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55794c9ae370 con 0x55794c9a0620 2024-04-23T18:49:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55794c8a00b0 con 0x55794c9ab800 2024-04-23T18:49:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.295+0000 7f73b4397740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55794c9ae4f0 con 0x55794c9a0f40 2024-04-23T18:49:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73b4397740 10 client.0 _ll_register_callbacks cb 0x55794c88dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:49:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32944/0 (socket says 172.21.15.33:32944) 2024-04-23T18:49:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 -- 172.21.15.33:0/2247904598 learned_addr learned my addr 172.21.15.33:0/2247904598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73a3fff640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73ab7fe640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9ab800 0x55794c9a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 -- 172.21.15.33:0/2247904598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 msgr2=0x55794c9a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9a8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 -- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9ab800 msgr2=0x55794c9a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9ab800 0x55794c9a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:57.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 -- 172.21.15.33:0/2247904598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c6e4620 con 0x55794c9a0620 2024-04-23T18:49:57.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73aaffd640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b766e4baaae3acc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73927fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:49:57.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739c035040 con 0x55794c9a0620 2024-04-23T18:49:57.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f739c004770 con 0x55794c9a0620 2024-04-23T18:49:57.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.296+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73940a0600 con 0x55794c9a0620 2024-04-23T18:49:57.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f739c004970 con 0x55794c9a0620 2024-04-23T18:49:57.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 10 client.17904 fetch_fsmap learned FSMap version 43 2024-04-23T18:49:57.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55794c9ba2a0 con 0x55794c9a0620 2024-04-23T18:49:57.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73940a0600 con 0x55794c9a0620 2024-04-23T18:49:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f739c0314a0 con 0x55794c9a0620 2024-04-23T18:49:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 10 client.17904 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:49:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 10 client.17904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:49:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 20 client.17904 populate_metadata read hostname 'smithi033' 2024-04-23T18:49:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794c9bb710 0x55794c9bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:57.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 4 client.17904 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:49:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55794c913390 con 0x55794c9bb710 2024-04-23T18:49:57.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.297+0000 7f73a3fff640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794c9bb710 0x55794c9bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:57.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73927fc640 10 client.17904 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:57.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73a3fff640 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794c9bb710 0x55794c9bdb20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=528860baa4ecc0aa server_cookie=4137e783f5a28001 in_seq=0 out_seq=0 2024-04-23T18:49:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73927fc640 1 -- 172.21.15.33:0/2247904598 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55794c913390 con 0x55794c9bb710 2024-04-23T18:49:58.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73927fc640 10 client.17904 handle_command_reply: tid=0 2024-04-23T18:49:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 client.17904 shutdown 2024-04-23T18:49:58.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 msgr2=0x55794c9ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f739c009230 tx=0x7f739c005040 comp rx=0 tx=0).stop 2024-04-23T18:49:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 shutdown_connections 2024-04-23T18:49:58.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794c9bb710 0x55794c9bdb20 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:58.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9ab800 0x55794c9a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:58.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73927fc640 0 client.17904 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:49:58.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9a0f40 0x55794c9a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:58.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 --2- 172.21.15.33:0/2247904598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0620 0x55794c9ab150 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:58.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.298+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 >> 172.21.15.33:0/2247904598 conn(0x55794c99cb30 msgr2=0x55794c9a2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:49:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.299+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 shutdown_connections 2024-04-23T18:49:58.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.299+0000 7f73b4397740 1 -- 172.21.15.33:0/2247904598 wait complete. 2024-04-23T18:49:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.299+0000 7f73b4397740 20 client.17904 trim_cache size 0 max 16384 2024-04-23T18:49:58.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.313+0000 7f73b4397740 1 Processor -- start 2024-04-23T18:49:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.313+0000 7f73b4397740 1 -- start start 2024-04-23T18:49:58.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:58.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:49:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1af5a0 con 0x55794c9a0f40 2024-04-23T18:49:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1ad390 con 0x55794c9a0620 2024-04-23T18:49:59.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73b4397740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1aeed0 con 0x55794c9ab800 2024-04-23T18:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32946/0 (socket says 172.21.15.33:32946) 2024-04-23T18:49:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 -- 172.21.15.33:0/1506259663 learned_addr learned my addr 172.21.15.33:0/1506259663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:49:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73aaffd640 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:49:59.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 -- 172.21.15.33:0/1506259663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 msgr2=0x55794c9c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:59.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 -- 172.21.15.33:0/1506259663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 msgr2=0x55794c9b7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:49:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 -- 172.21.15.33:0/1506259663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c9a73f0 con 0x55794c9a0f40 2024-04-23T18:49:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73aaffd640 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:49:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a3fff640 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f73940986b0 tx=0x7f739409bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16958dc70efec88 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:49:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a0ff9640 1 -- 172.21.15.33:0/1506259663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940a0600 con 0x55794c9a0f40 2024-04-23T18:49:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a0ff9640 1 -- 172.21.15.33:0/1506259663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73940cdb70 con 0x55794c9a0f40 2024-04-23T18:49:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.314+0000 7f73a0ff9640 1 -- 172.21.15.33:0/1506259663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394081de0 con 0x55794c9a0f40 2024-04-23T18:49:59.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- 172.21.15.33:0/1506259663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 msgr2=0x55794c99bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:49:59.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f73940986b0 tx=0x7f739409bcb0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- 172.21.15.33:0/1506259663 shutdown_connections 2024-04-23T18:49:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c99bb30 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 --2- 172.21.15.33:0/1506259663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:49:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- 172.21.15.33:0/1506259663 >> 172.21.15.33:0/1506259663 conn(0x55794c99cb30 msgr2=0x55794c99b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- 172.21.15.33:0/1506259663 shutdown_connections 2024-04-23T18:50:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- 172.21.15.33:0/1506259663 wait complete. 2024-04-23T18:50:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.315+0000 7f73b4397740 1 -- start start 2024-04-23T18:50:00.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 Processor -- start 2024-04-23T18:50:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:00.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:00.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:00.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1ae130 con 0x55794c9a0f40 2024-04-23T18:50:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1ada60 con 0x55794c9a0620 2024-04-23T18:50:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73ac1ae800 con 0x55794c9ab800 2024-04-23T18:50:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 10 client.0 _ll_register_callbacks cb 0x7f739c0042c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:00.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32948/0 (socket says 172.21.15.33:32948) 2024-04-23T18:50:00.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 -- 172.21.15.33:0/3085644611 learned_addr learned my addr 172.21.15.33:0/3085644611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 -- 172.21.15.33:0/3085644611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 msgr2=0x55794c9b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:00.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73aaffd640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 -- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 msgr2=0x55794c9b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 -- 172.21.15.33:0/3085644611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c9a0d70 con 0x55794c9a0f40 2024-04-23T18:50:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73a3fff640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f739409c1e0 tx=0x7f7394010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb871bc0f8452a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73927fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940988c0 con 0x55794c9a0f40 2024-04-23T18:50:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f7394081b80 con 0x55794c9a0f40 2024-04-23T18:50:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.316+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73940cdb70 con 0x55794c9a0f40 2024-04-23T18:50:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7394081d80 con 0x55794c9a0f40 2024-04-23T18:50:01.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 10 client.17910 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55794ca21c90 con 0x55794c9a0f40 2024-04-23T18:50:01.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73940cdb70 con 0x55794c9a0f40 2024-04-23T18:50:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f739409b730 con 0x55794c9a0f40 2024-04-23T18:50:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 10 client.17910 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 20 client.17910 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794ca21ec0 0x55794ca242b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 4 client.17910 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55794c90d200 con 0x55794ca21ec0 2024-04-23T18:50:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.317+0000 7f73aaffd640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794ca21ec0 0x55794ca242b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.318+0000 7f73aaffd640 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794ca21ec0 0x55794ca242b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9474ef43403f17d6 server_cookie=be30882b4fb86f5a in_seq=0 out_seq=0 2024-04-23T18:50:01.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.318+0000 7f73927fc640 10 client.17910 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73927fc640 1 -- 172.21.15.33:0/3085644611 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55794c90d200 con 0x55794ca21ec0 2024-04-23T18:50:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73927fc640 10 client.17910 handle_command_reply: tid=0 2024-04-23T18:50:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 client.17910 shutdown 2024-04-23T18:50:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 msgr2=0x55794c9b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f739409c1e0 tx=0x7f7394010590 comp rx=0 tx=0).stop 2024-04-23T18:50:01.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 shutdown_connections 2024-04-23T18:50:01.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55794ca21ec0 0x55794ca242b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55794c9ab800 0x55794c9b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:01.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55794c9a0f40 0x55794c9b1040 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:01.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73927fc640 0 client.17910 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 --2- 172.21.15.33:0/3085644611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55794c9a0620 0x55794c9b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:01.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 >> 172.21.15.33:0/3085644611 conn(0x55794c99cb30 msgr2=0x55794c99bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:02.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 shutdown_connections 2024-04-23T18:50:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 1 -- 172.21.15.33:0/3085644611 wait complete. 2024-04-23T18:50:02.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.325+0000 7f73b4397740 20 client.17910 trim_cache size 0 max 16384 2024-04-23T18:50:02.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:02.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:02.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 msgr2=0x7f73980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:02.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 0x7f73980a6280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009fd0 tx=0x7f73a4004ab0 comp rx=0 tx=0).stop 2024-04-23T18:50:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 msgr2=0x7f73ac198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:02.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f739400b450 tx=0x7f739400b910 comp rx=0 tx=0).stop 2024-04-23T18:50:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 shutdown_connections 2024-04-23T18:50:02.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f73980a3dc0 0x7f73980a6280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f73ac19cfe0 0x7f73ac19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac1072a0 0x7f73ac198cb0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 --2- 172.21.15.33:0/3259150989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73ac1021d0 0x7f73ac198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:02.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 >> 172.21.15.33:0/3259150989 conn(0x7f73ac0793e0 msgr2=0x7f73ac0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 shutdown_connections 2024-04-23T18:50:02.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:16.326+0000 7f73b15a2640 1 -- 172.21.15.33:0/3259150989 wait complete. 2024-04-23T18:50:02.970 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:16 smithi033 ceph-mon[29274]: daemon mds.d restarted 2024-04-23T18:50:03.004 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:16 smithi033 ceph-mon[29274]: osdmap e129: 12 total, 12 up, 12 in 2024-04-23T18:50:03.013 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/1517776929,v1:172.21.15.33:6837/1517776929] up:boot 2024-04-23T18:50:03.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:50:03.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:50:03.122 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:17 smithi033 ceph-mon[29274]: pgmap v698: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:50:03.199 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:19 smithi033 ceph-mon[29274]: pgmap v700: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:50:03.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:21 smithi033 ceph-mon[29274]: pgmap v701: 97 pgs: 97 active+clean; 15 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-23T18:50:03.346 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:23 smithi033 ceph-mon[29274]: pgmap v702: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:03.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:25 smithi033 ceph-mon[29274]: pgmap v703: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:03.415 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:27 smithi033 ceph-mon[29274]: pgmap v704: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 409 B/s wr, 11 op/s 2024-04-23T18:50:03.424 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:54 smithi122 ceph-mon[32546]: pgmap v751: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:03.433 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:56 smithi122 ceph-mon[32546]: pgmap v752: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:03.442 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:49:58 smithi122 ceph-mon[32546]: pgmap v753: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:03.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: pgmap v754: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:03.582 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:50:03.601 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-23T18:50:03.610 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-23T18:50:03.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: daemon mds.f on smithi122 is in error state 2024-04-23T18:50:03.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:50:03.694 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:50:03.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:00 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:03.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:01 smithi122 ceph-mon[32546]: pgmap v755: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 284 B/s wr, 9 op/s 2024-04-23T18:50:03.798 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:01 smithi122 ceph-mon[32546]: pgmap v756: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T18:50:03.901 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:54 smithi106 ceph-mon[32716]: pgmap v751: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:03.982 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:56 smithi106 ceph-mon[32716]: pgmap v752: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:04.032 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:49:58 smithi106 ceph-mon[32716]: pgmap v753: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:04.106 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: pgmap v754: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:04.115 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:50:04.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-23T18:50:04.180 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-23T18:50:04.188 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: daemon mds.f on smithi122 is in error state 2024-04-23T18:50:04.223 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:50:04.230 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:50:04.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:00 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:04.248 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:01 smithi106 ceph-mon[32716]: pgmap v755: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 284 B/s wr, 9 op/s 2024-04-23T18:50:04.256 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:01 smithi106 ceph-mon[32716]: pgmap v756: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T18:50:04.266 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:03 smithi122 ceph-mon[32546]: pgmap v757: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:50:04.317 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:03 smithi106 ceph-mon[32716]: pgmap v757: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:50:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.424+0000 7f6a57922640 1 -- 172.21.15.33:0/3288994177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 msgr2=0x7f6a501025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:04.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.424+0000 7f6a57922640 1 --2- 172.21.15.33:0/3288994177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501025b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009900 tx=0x7f6a4c02a410 comp rx=0 tx=0).stop 2024-04-23T18:50:04.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 -- 172.21.15.33:0/3288994177 shutdown_connections 2024-04-23T18:50:04.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 --2- 172.21.15.33:0/3288994177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 0x7f6a5010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 --2- 172.21.15.33:0/3288994177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a50102b80 0x7f6a50106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:04.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 --2- 172.21.15.33:0/3288994177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501025b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 -- 172.21.15.33:0/3288994177 >> 172.21.15.33:0/3288994177 conn(0x7f6a500793e0 msgr2=0x7f6a500797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 -- 172.21.15.33:0/3288994177 shutdown_connections 2024-04-23T18:50:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 -- 172.21.15.33:0/3288994177 wait complete. 2024-04-23T18:50:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 Processor -- start 2024-04-23T18:50:04.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.425+0000 7f6a57922640 1 -- start start 2024-04-23T18:50:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 0x7f6a50198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:04.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5019cf60 0x7f6a5019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501121e0 con 0x7f6a501021d0 2024-04-23T18:50:04.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112060 con 0x7f6a5019cf60 2024-04-23T18:50:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a50112360 con 0x7f6a501072a0 2024-04-23T18:50:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:05.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32774/0 (socket says 172.21.15.33:32774) 2024-04-23T18:50:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 -- 172.21.15.33:0/2895662006 learned_addr learned my addr 172.21.15.33:0/2895662006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5019cf60 0x7f6a5019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:05.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 0x7f6a50198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 -- 172.21.15.33:0/2895662006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 msgr2=0x7f6a50198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:05.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 0x7f6a50198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:05.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 -- 172.21.15.33:0/2895662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5019cf60 msgr2=0x7f6a5019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:05.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5019cf60 0x7f6a5019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:05.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 -- 172.21.15.33:0/2895662006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5019da50 con 0x7f6a501021d0 2024-04-23T18:50:05.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a556be640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005940 tx=0x7f6a4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:05.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a467fc640 1 -- 172.21.15.33:0/2895662006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c03f030 con 0x7f6a501021d0 2024-04-23T18:50:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a467fc640 1 -- 172.21.15.33:0/2895662006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a4c003aa0 con 0x7f6a501021d0 2024-04-23T18:50:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5019dce0 con 0x7f6a501021d0 2024-04-23T18:50:05.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.426+0000 7f6a467fc640 1 -- 172.21.15.33:0/2895662006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c035070 con 0x7f6a501021d0 2024-04-23T18:50:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.427+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a501a5500 con 0x7f6a501021d0 2024-04-23T18:50:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.429+0000 7f6a467fc640 1 -- 172.21.15.33:0/2895662006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a4c042030 con 0x7f6a501021d0 2024-04-23T18:50:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.429+0000 7f6a5a63e740 1 Processor -- start 2024-04-23T18:50:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 -- start start 2024-04-23T18:50:05.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:05.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55953300a970 con 0x5595330a5620 2024-04-23T18:50:05.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559532fa74a0 con 0x5595330a5f40 2024-04-23T18:50:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559533006930 con 0x5595330b0800 2024-04-23T18:50:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32790/0 (socket says 172.21.15.33:32790) 2024-04-23T18:50:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a467fc640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 0x7f6a3c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 -- 172.21.15.33:0/1552826007 learned_addr learned my addr 172.21.15.33:0/1552826007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a556be640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 0x7f6a3c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a467fc640 1 -- 172.21.15.33:0/2895662006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f6a4c0f0890 con 0x7f6a501021d0 2024-04-23T18:50:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 -- 172.21.15.33:0/1552826007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 msgr2=0x5595330b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 -- 172.21.15.33:0/1552826007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 msgr2=0x5595330b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:06.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:06.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a55ebf640 1 -- 172.21.15.33:0/1552826007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559532d1d210 con 0x5595330a5620 2024-04-23T18:50:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.430+0000 7f6a556be640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:50:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.431+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6a48005f40 tx=0x7f6a48014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6dbd135582d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:06.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.431+0000 7f6a3b7fe640 1 -- 172.21.15.33:0/1552826007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801ee50 con 0x5595330a5620 2024-04-23T18:50:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.431+0000 7f6a3b7fe640 1 -- 172.21.15.33:0/1552826007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48023070 con 0x5595330a5620 2024-04-23T18:50:06.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.431+0000 7f6a3b7fe640 1 -- 172.21.15.33:0/1552826007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801dc90 con 0x5595330a5620 2024-04-23T18:50:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.431+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 0x7f6a3c0a6300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a48022000 tx=0x7f6a48020000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:06.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1552826007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 msgr2=0x5595330a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:06.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6a48005f40 tx=0x7f6a48014f10 comp rx=0 tx=0).stop 2024-04-23T18:50:06.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1552826007 shutdown_connections 2024-04-23T18:50:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:06.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1552826007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5620 0x5595330a5a00 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:06.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1552826007 >> 172.21.15.33:0/1552826007 conn(0x5595330a1b30 msgr2=0x5595330a1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:06.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1552826007 shutdown_connections 2024-04-23T18:50:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1552826007 wait complete. 2024-04-23T18:50:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 -- start start 2024-04-23T18:50:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.432+0000 7f6a5a63e740 1 Processor -- start 2024-04-23T18:50:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330b0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:07.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5595330b3430 con 0x5595330a5f40 2024-04-23T18:50:07.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559532fa50b0 con 0x5595330a5620 2024-04-23T18:50:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5595330b35b0 con 0x5595330b0800 2024-04-23T18:50:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 10 client.0 _ll_register_callbacks cb 0x559532f92d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32804/0 (socket says 172.21.15.33:32804) 2024-04-23T18:50:07.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 -- 172.21.15.33:0/1618316647 learned_addr learned my addr 172.21.15.33:0/1618316647 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a556be640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 -- 172.21.15.33:0/1618316647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 msgr2=0x5595330ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330ad930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 -- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 msgr2=0x5595330b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 -- 172.21.15.33:0/1618316647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559532de9620 con 0x5595330a5f40 2024-04-23T18:50:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a556be640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:50:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6a40012700 tx=0x7f6a40012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155b4629e1855a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:07.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000f920 con 0x5595330a5f40 2024-04-23T18:50:08.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f6a4000fac0 con 0x5595330a5f40 2024-04-23T18:50:08.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.433+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a48023070 con 0x5595330a5f40 2024-04-23T18:50:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a4000fcc0 con 0x5595330a5f40 2024-04-23T18:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 10 client.17922 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5595330bf3a0 con 0x5595330a5f40 2024-04-23T18:50:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48023070 con 0x5595330a5f40 2024-04-23T18:50:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f6a4001c420 con 0x5595330a5f40 2024-04-23T18:50:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 10 client.17922 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 10 client.17922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:08.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 20 client.17922 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5595330c0920 0x5595330c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 4 client.17922 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:50:08.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559533018390 con 0x5595330c0920 2024-04-23T18:50:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.434+0000 7f6a556be640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5595330c0920 0x5595330c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:08.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.435+0000 7f6a38ff9640 10 client.17922 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:08.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.435+0000 7f6a556be640 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5595330c0920 0x5595330c2d70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0822aba81807b3e server_cookie=9a6a443303a039ad in_seq=0 out_seq=0 2024-04-23T18:50:08.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.435+0000 7f6a38ff9640 1 -- 172.21.15.33:0/1618316647 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559533018390 con 0x5595330c0920 2024-04-23T18:50:08.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.435+0000 7f6a38ff9640 10 client.17922 handle_command_reply: tid=0 2024-04-23T18:50:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.435+0000 7f6a5a63e740 1 client.17922 shutdown 2024-04-23T18:50:09.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 msgr2=0x5595330ad3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:09.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6a40012700 tx=0x7f6a40012bc0 comp rx=0 tx=0).stop 2024-04-23T18:50:09.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 shutdown_connections 2024-04-23T18:50:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5595330c0920 0x5595330c2d70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330b0800 0x5595330ad930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a38ff9640 0 client.17922 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:09.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330a5f40 0x5595330ad3f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:09.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/1618316647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:09.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 >> 172.21.15.33:0/1618316647 conn(0x5595330a1b30 msgr2=0x5595330a78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 shutdown_connections 2024-04-23T18:50:09.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 1 -- 172.21.15.33:0/1618316647 wait complete. 2024-04-23T18:50:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.436+0000 7f6a5a63e740 20 client.17922 trim_cache size 0 max 16384 2024-04-23T18:50:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 Processor -- start 2024-04-23T18:50:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 -- start start 2024-04-23T18:50:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 0x5595330a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501af5e0 con 0x5595330b0800 2024-04-23T18:50:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ad3d0 con 0x5595330a5620 2024-04-23T18:50:09.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501aef10 con 0x5595330a5f40 2024-04-23T18:50:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32814/0 (socket says 172.21.15.33:32814) 2024-04-23T18:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 -- 172.21.15.33:0/4158363542 learned_addr learned my addr 172.21.15.33:0/4158363542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:10.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a556be640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:10.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 0x5595330a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 -- 172.21.15.33:0/4158363542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 msgr2=0x5595330a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:10.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 0x5595330a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 -- 172.21.15.33:0/4158363542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 msgr2=0x5595330bc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:10.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330bc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a55ebf640 1 -- 172.21.15.33:0/4158363542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595330a4e20 con 0x5595330b0800 2024-04-23T18:50:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a556be640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.451+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 0x5595330a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009590 tx=0x7f6a480146a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade6ec5befb6262a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a3affd640 1 -- 172.21.15.33:0/4158363542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48036020 con 0x5595330b0800 2024-04-23T18:50:10.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a3affd640 1 -- 172.21.15.33:0/4158363542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48023070 con 0x5595330b0800 2024-04-23T18:50:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a3affd640 1 -- 172.21.15.33:0/4158363542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801d590 con 0x5595330b0800 2024-04-23T18:50:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 -- 172.21.15.33:0/4158363542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 msgr2=0x5595330c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:10.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009590 tx=0x7f6a480146a0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 -- 172.21.15.33:0/4158363542 shutdown_connections 2024-04-23T18:50:10.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330c8680 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5f40 0x5595330a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/4158363542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5620 0x5595330bc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:10.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 -- 172.21.15.33:0/4158363542 >> 172.21.15.33:0/4158363542 conn(0x5595330a1b30 msgr2=0x5595330a0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.452+0000 7f6a5a63e740 1 -- 172.21.15.33:0/4158363542 shutdown_connections 2024-04-23T18:50:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 -- 172.21.15.33:0/4158363542 wait complete. 2024-04-23T18:50:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 -- start start 2024-04-23T18:50:11.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 Processor -- start 2024-04-23T18:50:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5620 0x5595330b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:11.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:11.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:11.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ae170 con 0x5595330b0800 2024-04-23T18:50:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501adaa0 con 0x5595330a5f40 2024-04-23T18:50:11.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a501ae840 con 0x5595330a5620 2024-04-23T18:50:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a5a63e740 10 client.0 _ll_register_callbacks cb 0x7f6a40009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:11.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:11.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a55ebf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32822/0 (socket says 172.21.15.33:32822) 2024-04-23T18:50:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.453+0000 7f6a55ebf640 1 -- 172.21.15.33:0/635126816 learned_addr learned my addr 172.21.15.33:0/635126816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a556be640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5620 0x5595330b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a54ebd640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:11.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 -- 172.21.15.33:0/635126816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5620 msgr2=0x5595330b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:11.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5620 0x5595330b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 -- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 msgr2=0x5595330b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:11.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 -- 172.21.15.33:0/635126816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595330b6680 con 0x5595330b0800 2024-04-23T18:50:11.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a55ebf640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012d40 tx=0x7f6a48009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e6d76b34f808c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:11.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:11.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4803f070 con 0x5595330b0800 2024-04-23T18:50:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f6a48014220 con 0x5595330b0800 2024-04-23T18:50:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a48023070 con 0x5595330b0800 2024-04-23T18:50:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6a48036030 con 0x5595330b0800 2024-04-23T18:50:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a5a63e740 10 client.17928 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:11.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.454+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559533126b30 con 0x5595330b0800 2024-04-23T18:50:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a48023070 con 0x5595330b0800 2024-04-23T18:50:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f6a48031070 con 0x5595330b0800 2024-04-23T18:50:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 10 client.17928 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 10 client.17928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:12.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 20 client.17928 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:12.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559533127130 0x559533129520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 4 client.17928 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595330adcc0 con 0x559533127130 2024-04-23T18:50:12.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a556be640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559533127130 0x559533129520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:12.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a556be640 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559533127130 0x559533129520 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b3af990ccfdd71 server_cookie=fb6c8771b6b250fd in_seq=0 out_seq=0 2024-04-23T18:50:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.455+0000 7f6a38ff9640 10 client.17928 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.459+0000 7f6a38ff9640 1 -- 172.21.15.33:0/635126816 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595330adcc0 con 0x559533127130 2024-04-23T18:50:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.459+0000 7f6a38ff9640 10 client.17928 handle_command_reply: tid=0 2024-04-23T18:50:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.459+0000 7f6a5a63e740 1 client.17928 shutdown 2024-04-23T18:50:12.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 msgr2=0x5595330b6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012d40 tx=0x7f6a48009650 comp rx=0 tx=0).stop 2024-04-23T18:50:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 shutdown_connections 2024-04-23T18:50:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559533127130 0x559533129520 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5595330b0800 0x5595330b6140 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5595330a5f40 0x5595330b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a38ff9640 0 client.17928 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 --2- 172.21.15.33:0/635126816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595330a5620 0x5595330b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 >> 172.21.15.33:0/635126816 conn(0x5595330a1b30 msgr2=0x5595330a0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 shutdown_connections 2024-04-23T18:50:12.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 1 -- 172.21.15.33:0/635126816 wait complete. 2024-04-23T18:50:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a5a63e740 20 client.17928 trim_cache size 0 max 16384 2024-04-23T18:50:12.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:12.881 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:04 smithi122 ceph-mon[32546]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:50:12.912 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:05 smithi122 ceph-mon[32546]: pgmap v759: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:50:12.918 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:07 smithi122 ceph-mon[32546]: pgmap v760: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:50:12.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:09 smithi122 ceph-mon[32546]: pgmap v761: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:50:12.990 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:11 smithi122 ceph-mon[32546]: pgmap v762: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:50:12.997 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:11 smithi122 ceph-mon[32546]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:50:13.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:04 smithi106 ceph-mon[32716]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:50:13.013 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:05 smithi106 ceph-mon[32716]: pgmap v759: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:50:13.206 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:07 smithi106 ceph-mon[32716]: pgmap v760: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:50:13.319 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:09 smithi106 ceph-mon[32716]: pgmap v761: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:50:13.326 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:11 smithi106 ceph-mon[32716]: pgmap v762: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:50:13.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:11 smithi106 ceph-mon[32716]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:50:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 msgr2=0x7f6a3c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:13.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 0x7f6a3c0a6300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a48022000 tx=0x7f6a48020000 comp rx=0 tx=0).stop 2024-04-23T18:50:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 msgr2=0x7f6a501986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c005940 tx=0x7f6a4c004270 comp rx=0 tx=0).stop 2024-04-23T18:50:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 shutdown_connections 2024-04-23T18:50:13.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a3c0a3e40 0x7f6a3c0a6300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5019cf60 0x7f6a5019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a501072a0 0x7f6a50198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 --2- 172.21.15.33:0/2895662006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a501021d0 0x7f6a501986a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.460+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 >> 172.21.15.33:0/2895662006 conn(0x7f6a500793e0 msgr2=0x7f6a500ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:13.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.461+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 shutdown_connections 2024-04-23T18:50:13.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:29.461+0000 7f6a57922640 1 -- 172.21.15.33:0/2895662006 wait complete. 2024-04-23T18:50:13.781 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:29 smithi033 ceph-mon[29274]: pgmap v705: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 345 B/s wr, 9 op/s 2024-04-23T18:50:13.990 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:31 smithi033 ceph-mon[29274]: pgmap v706: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:50:14.074 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:33 smithi033 ceph-mon[29274]: pgmap v707: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:50:14.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:35 smithi033 ceph-mon[29274]: pgmap v708: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:14.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:37 smithi033 ceph-mon[29274]: pgmap v709: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:14.185 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:39 smithi033 ceph-mon[29274]: pgmap v710: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:14.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:41 smithi033 ceph-mon[29274]: pgmap v711: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:14.203 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:13 smithi122 ceph-mon[32546]: pgmap v764: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-04-23T18:50:14.212 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:13 smithi106 ceph-mon[32716]: pgmap v764: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-04-23T18:50:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 -- 172.21.15.33:0/1728931086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac104d70 msgr2=0x7f75ac108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:14.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 --2- 172.21.15.33:0/1728931086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac104d70 0x7f75ac108db0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009900 tx=0x7f75a002a410 comp rx=0 tx=0).stop 2024-04-23T18:50:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 -- 172.21.15.33:0/1728931086 shutdown_connections 2024-04-23T18:50:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 --2- 172.21.15.33:0/1728931086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac109490 0x7f75ac111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 --2- 172.21.15.33:0/1728931086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac104d70 0x7f75ac108db0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 --2- 172.21.15.33:0/1728931086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac1043c0 0x7f75ac1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 -- 172.21.15.33:0/1728931086 >> 172.21.15.33:0/1728931086 conn(0x7f75ac0ffc80 msgr2=0x7f75ac1020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:14.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 -- 172.21.15.33:0/1728931086 shutdown_connections 2024-04-23T18:50:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.560+0000 7f75b0f19640 1 -- 172.21.15.33:0/1728931086 wait complete. 2024-04-23T18:50:14.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 Processor -- start 2024-04-23T18:50:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 -- start start 2024-04-23T18:50:14.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:14.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 0x7f75ac1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac19a730 0x7f75ac19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1143b0 con 0x7f75ac1043c0 2024-04-23T18:50:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac114230 con 0x7f75ac109490 2024-04-23T18:50:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.561+0000 7f75b0f19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac114530 con 0x7f75ac19a730 2024-04-23T18:50:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35662/0 (socket says 172.21.15.33:35662) 2024-04-23T18:50:15.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 -- 172.21.15.33:0/3510858525 learned_addr learned my addr 172.21.15.33:0/3510858525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aad9d640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac19a730 0x7f75ac19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:15.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 0x7f75ac1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 -- 172.21.15.33:0/3510858525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac19a730 msgr2=0x7f75ac19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac19a730 0x7f75ac19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:15.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 -- 172.21.15.33:0/3510858525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 msgr2=0x7f75ac1a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 0x7f75ac1a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 -- 172.21.15.33:0/3510858525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac19b330 con 0x7f75ac1043c0 2024-04-23T18:50:15.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 0x7f75ac1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75aa59c640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:15.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f759b7fe640 1 -- 172.21.15.33:0/3510858525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c00cd80 con 0x7f75ac1043c0 2024-04-23T18:50:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f759b7fe640 1 -- 172.21.15.33:0/3510858525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c0108e0 con 0x7f75ac1043c0 2024-04-23T18:50:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac1a7410 con 0x7f75ac1043c0 2024-04-23T18:50:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f759b7fe640 1 -- 172.21.15.33:0/3510858525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c018980 con 0x7f75ac1043c0 2024-04-23T18:50:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.562+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1a7810 con 0x7f75ac1043c0 2024-04-23T18:50:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.565+0000 7f759b7fe640 1 -- 172.21.15.33:0/3510858525 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f759c010430 con 0x7f75ac1043c0 2024-04-23T18:50:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 Processor -- start 2024-04-23T18:50:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 -- start start 2024-04-23T18:50:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c8ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c98bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eef4bf0970 con 0x55eef4c8bf40 2024-04-23T18:50:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eef4b8d4a0 con 0x55eef4c8b620 2024-04-23T18:50:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eef4bec930 con 0x55eef4c96800 2024-04-23T18:50:16.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:16.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75aa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35674/0 (socket says 172.21.15.33:35674) 2024-04-23T18:50:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75aa59c640 1 -- 172.21.15.33:0/1165223736 learned_addr learned my addr 172.21.15.33:0/1165223736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75aad9d640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c8ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f759b7fe640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 0x7f75940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.566+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c98bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:16.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aad9d640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 0x7f75940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:16.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f759b7fe640 1 -- 172.21.15.33:0/3510858525 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f759c0d0bc0 con 0x7f75ac1043c0 2024-04-23T18:50:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 -- 172.21.15.33:0/1165223736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 msgr2=0x55eef4c98bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c98bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:16.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 -- 172.21.15.33:0/1165223736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 msgr2=0x55eef4c8ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c8ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:16.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 -- 172.21.15.33:0/1165223736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef4903210 con 0x55eef4c8bf40 2024-04-23T18:50:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aad9d640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c8ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:50:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c98bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:50:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f759c0ce290 tx=0x7f759c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c65070107ef693 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f7598ff9640 1 -- 172.21.15.33:0/1165223736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c09d270 con 0x55eef4c8bf40 2024-04-23T18:50:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f75aad9d640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 0x7f75940a6300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f75a40094d0 tx=0x7f75a4004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:16.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f7598ff9640 1 -- 172.21.15.33:0/1165223736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c09cb70 con 0x55eef4c8bf40 2024-04-23T18:50:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.567+0000 7f7598ff9640 1 -- 172.21.15.33:0/1165223736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c0ce5a0 con 0x55eef4c8bf40 2024-04-23T18:50:16.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1165223736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 msgr2=0x55eef4c962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f759c0ce290 tx=0x7f759c00cf20 comp rx=0 tx=0).stop 2024-04-23T18:50:17.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1165223736 shutdown_connections 2024-04-23T18:50:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c98bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c962c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:17.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1165223736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c8ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:17.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1165223736 >> 172.21.15.33:0/1165223736 conn(0x55eef4c87b30 msgr2=0x55eef4c87f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1165223736 shutdown_connections 2024-04-23T18:50:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.568+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1165223736 wait complete. 2024-04-23T18:50:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 -- start start 2024-04-23T18:50:17.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 Processor -- start 2024-04-23T18:50:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8b620 0x55eef4c960f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:17.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:17.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c96800 0x55eef4c93920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:17.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eef4c993d0 con 0x55eef4c8bf40 2024-04-23T18:50:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eef4b8b0b0 con 0x55eef4c96800 2024-04-23T18:50:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eef4c99550 con 0x55eef4c8b620 2024-04-23T18:50:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75b3c2e740 10 client.0 _ll_register_callbacks cb 0x55eef4b78d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:17.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:17.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35688/0 (socket says 172.21.15.33:35688) 2024-04-23T18:50:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 -- 172.21.15.33:0/1388801623 learned_addr learned my addr 172.21.15.33:0/1388801623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75aad9d640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c96800 0x55eef4c93920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:17.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8b620 0x55eef4c960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:17.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 -- 172.21.15.33:0/1388801623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8b620 msgr2=0x55eef4c960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8b620 0x55eef4c960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:17.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 -- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c96800 msgr2=0x55eef4c93920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:17.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c96800 0x55eef4c93920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.569+0000 7f75a9d9b640 1 -- 172.21.15.33:0/1388801623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef49cf620 con 0x55eef4c8bf40 2024-04-23T18:50:17.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009900 tx=0x7f75a00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b4d80efad77076 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:18.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:18.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0035040 con 0x55eef4c8bf40 2024-04-23T18:50:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f75a0004240 con 0x55eef4c8bf40 2024-04-23T18:50:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f759c09cb70 con 0x55eef4c8bf40 2024-04-23T18:50:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75a0004480 con 0x55eef4c8bf40 2024-04-23T18:50:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 10 client.17940 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eef4ca52a0 con 0x55eef4c8bf40 2024-04-23T18:50:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c09cb70 con 0x55eef4c8bf40 2024-04-23T18:50:18.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f75a0004b70 con 0x55eef4c8bf40 2024-04-23T18:50:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 10 client.17940 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:18.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 10 client.17940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:18.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 20 client.17940 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:18.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4ca6760 0x55eef4ca8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:18.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 4 client.17940 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:50:18.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.570+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef4bfe390 con 0x55eef4ca6760 2024-04-23T18:50:18.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.571+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4ca6760 0x55eef4ca8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:18.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.571+0000 7f7591ffb640 10 client.17940 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:18.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.571+0000 7f75aa59c640 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4ca6760 0x55eef4ca8b90 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a5635971a8eae1 server_cookie=ae69cc8f0e450e6e in_seq=0 out_seq=0 2024-04-23T18:50:18.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f7591ffb640 1 -- 172.21.15.33:0/1388801623 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eef4bfe390 con 0x55eef4ca6760 2024-04-23T18:50:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f7591ffb640 10 client.17940 handle_command_reply: tid=0 2024-04-23T18:50:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 client.17940 shutdown 2024-04-23T18:50:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 msgr2=0x55eef4c933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f75a0009900 tx=0x7f75a00317e0 comp rx=0 tx=0).stop 2024-04-23T18:50:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 shutdown_connections 2024-04-23T18:50:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4ca6760 0x55eef4ca8b90 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c96800 0x55eef4c93920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f7591ffb640 0 client.17940 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c933e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/1388801623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8b620 0x55eef4c960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 >> 172.21.15.33:0/1388801623 conn(0x55eef4c87b30 msgr2=0x55eef4c8d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:19.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 shutdown_connections 2024-04-23T18:50:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 1 -- 172.21.15.33:0/1388801623 wait complete. 2024-04-23T18:50:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.572+0000 7f75b3c2e740 20 client.17940 trim_cache size 0 max 16384 2024-04-23T18:50:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 Processor -- start 2024-04-23T18:50:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 -- start start 2024-04-23T18:50:19.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4ca2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8bf40 0x55eef4cae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1b1870 con 0x55eef4c96800 2024-04-23T18:50:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1af660 con 0x55eef4c8b620 2024-04-23T18:50:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.587+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1b11a0 con 0x55eef4c8bf40 2024-04-23T18:50:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:19.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35700/0 (socket says 172.21.15.33:35700) 2024-04-23T18:50:19.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 -- 172.21.15.33:0/2745059919 learned_addr learned my addr 172.21.15.33:0/2745059919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8bf40 0x55eef4cae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:19.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aa59c640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4ca2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 -- 172.21.15.33:0/2745059919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8bf40 msgr2=0x55eef4cae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:20.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8bf40 0x55eef4cae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 -- 172.21.15.33:0/2745059919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 msgr2=0x55eef4ca2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4ca2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:20.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 -- 172.21.15.33:0/2745059919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef4c91930 con 0x55eef4c96800 2024-04-23T18:50:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aa59c640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4ca2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f75aad9d640 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f75a4012620 tx=0x7f75a4009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31be42723d0082ec server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f7593fff640 1 -- 172.21.15.33:0/2745059919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4010920 con 0x55eef4c96800 2024-04-23T18:50:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f7593fff640 1 -- 172.21.15.33:0/2745059919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a4025070 con 0x55eef4c96800 2024-04-23T18:50:20.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.588+0000 7f7593fff640 1 -- 172.21.15.33:0/2745059919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4021e50 con 0x55eef4c96800 2024-04-23T18:50:20.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2745059919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 msgr2=0x55eef4cae9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:20.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f75a4012620 tx=0x7f75a4009a70 comp rx=0 tx=0).stop 2024-04-23T18:50:20.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2745059919 shutdown_connections 2024-04-23T18:50:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c96800 0x55eef4cae9e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c8bf40 0x55eef4cae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2745059919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4ca2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:20.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2745059919 >> 172.21.15.33:0/2745059919 conn(0x55eef4c87b30 msgr2=0x55eef4c86420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2745059919 shutdown_connections 2024-04-23T18:50:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2745059919 wait complete. 2024-04-23T18:50:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- start start 2024-04-23T18:50:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 Processor -- start 2024-04-23T18:50:21.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c9ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1b0400 con 0x55eef4c8bf40 2024-04-23T18:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1afd30 con 0x55eef4c8b620 2024-04-23T18:50:21.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.589+0000 7f75b3c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac1b0ad0 con 0x55eef4c96800 2024-04-23T18:50:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75b3c2e740 10 client.0 _ll_register_callbacks cb 0x7f75a0032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:21.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35708/0 (socket says 172.21.15.33:35708) 2024-04-23T18:50:21.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 -- 172.21.15.33:0/2210740298 learned_addr learned my addr 172.21.15.33:0/2210740298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:21.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75aad9d640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:21.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75aa59c640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c9ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:21.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 -- 172.21.15.33:0/2210740298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 msgr2=0x55eef4c9c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c9c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 -- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 msgr2=0x55eef4c9ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c9ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:21.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 -- 172.21.15.33:0/2210740298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef4c9ca60 con 0x55eef4c8bf40 2024-04-23T18:50:21.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75a9d9b640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f75a002a450 tx=0x7f75a0009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03c0014256503be server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:21.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f7591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:21.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0052040 con 0x55eef4c8bf40 2024-04-23T18:50:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f75a0004a00 con 0x55eef4c8bf40 2024-04-23T18:50:22.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75a4025070 con 0x55eef4c8bf40 2024-04-23T18:50:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f75a0035040 con 0x55eef4c8bf40 2024-04-23T18:50:22.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75b3c2e740 10 client.17946 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.590+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eef4d0cba0 con 0x55eef4c8bf40 2024-04-23T18:50:22.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a4025070 con 0x55eef4c8bf40 2024-04-23T18:50:22.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f75a0003db0 con 0x55eef4c8bf40 2024-04-23T18:50:22.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 10 client.17946 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 10 client.17946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 20 client.17946 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4d0d290 0x55eef4d0f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:22.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 4 client.17946 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eef4c93cb0 con 0x55eef4d0d290 2024-04-23T18:50:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.591+0000 7f75aa59c640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4d0d290 0x55eef4d0f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.592+0000 7f7591ffb640 10 client.17946 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.592+0000 7f75aa59c640 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4d0d290 0x55eef4d0f6b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c446ab6ee633f04 server_cookie=4405eadf30e74dd5 in_seq=0 out_seq=0 2024-04-23T18:50:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f7591ffb640 1 -- 172.21.15.33:0/2210740298 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eef4c93cb0 con 0x55eef4d0d290 2024-04-23T18:50:23.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f7591ffb640 10 client.17946 handle_command_reply: tid=0 2024-04-23T18:50:23.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 client.17946 shutdown 2024-04-23T18:50:23.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 msgr2=0x55eef4c9be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f75a002a450 tx=0x7f75a0009520 comp rx=0 tx=0).stop 2024-04-23T18:50:23.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 shutdown_connections 2024-04-23T18:50:23.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eef4d0d290 0x55eef4d0f6b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:23.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eef4c96800 0x55eef4c9c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:23.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eef4c8bf40 0x55eef4c9be30 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 --2- 172.21.15.33:0/2210740298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eef4c8b620 0x55eef4c9ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:23.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f7591ffb640 0 client.17946 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 >> 172.21.15.33:0/2210740298 conn(0x55eef4c87b30 msgr2=0x55eef4c86cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 shutdown_connections 2024-04-23T18:50:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 1 -- 172.21.15.33:0/2210740298 wait complete. 2024-04-23T18:50:23.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.596+0000 7f75b3c2e740 20 client.17946 trim_cache size 0 max 16384 2024-04-23T18:50:23.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 msgr2=0x7f75940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:24.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 0x7f75940a6300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f75a40094d0 tx=0x7f75a4004ab0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 msgr2=0x7f75ac1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:24.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 shutdown_connections 2024-04-23T18:50:24.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f75940a3e40 0x7f75940a6300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75ac19a730 0x7f75ac19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75ac109490 0x7f75ac1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 --2- 172.21.15.33:0/3510858525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75ac1043c0 0x7f75ac1a0670 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 >> 172.21.15.33:0/3510858525 conn(0x7f75ac0ffc80 msgr2=0x7f75ac100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 shutdown_connections 2024-04-23T18:50:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:42.597+0000 7f75b0f19640 1 -- 172.21.15.33:0/3510858525 wait complete. 2024-04-23T18:50:24.670 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:15 smithi122 ceph-mon[32546]: pgmap v765: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:50:24.678 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:16 smithi122 ceph-mon[32546]: daemon mds.d restarted 2024-04-23T18:50:24.687 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:16 smithi122 ceph-mon[32546]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:50:24.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/2383185438,v1:172.21.15.33:6837/2383185438] up:boot 2024-04-23T18:50:24.791 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:50:24.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:50:24.934 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:17 smithi122 ceph-mon[32546]: pgmap v767: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:50:24.971 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:19 smithi122 ceph-mon[32546]: pgmap v768: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:50:25.013 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:21 smithi122 ceph-mon[32546]: pgmap v769: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T18:50:25.024 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:23 smithi122 ceph-mon[32546]: pgmap v770: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:25.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:15 smithi106 ceph-mon[32716]: pgmap v765: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:50:25.112 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:16 smithi106 ceph-mon[32716]: daemon mds.d restarted 2024-04-23T18:50:25.124 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:16 smithi106 ceph-mon[32716]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:50:25.237 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/2383185438,v1:172.21.15.33:6837/2383185438] up:boot 2024-04-23T18:50:25.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:50:25.327 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:50:25.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:17 smithi106 ceph-mon[32716]: pgmap v767: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:50:25.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:19 smithi106 ceph-mon[32716]: pgmap v768: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:50:25.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:21 smithi106 ceph-mon[32716]: pgmap v769: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T18:50:25.471 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:23 smithi106 ceph-mon[32716]: pgmap v770: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:25.484 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:43 smithi033 ceph-mon[29274]: pgmap v712: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:25.689 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:45 smithi033 ceph-mon[29274]: pgmap v713: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:50:25.752 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:47 smithi033 ceph-mon[29274]: pgmap v714: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:25.760 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:49 smithi033 ceph-mon[29274]: pgmap v715: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:25.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:51 smithi033 ceph-mon[29274]: pgmap v716: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:25.867 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:53 smithi033 ceph-mon[29274]: pgmap v717: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:25.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:25 smithi122 ceph-mon[32546]: pgmap v771: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:25.971 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:25 smithi106 ceph-mon[32716]: pgmap v771: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:50:25.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/1059941499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 msgr2=0x7f8c1810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/1059941499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c1810c000 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009900 tx=0x7f8c0802a480 comp rx=0 tx=0).stop 2024-04-23T18:50:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/1059941499 shutdown_connections 2024-04-23T18:50:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/1059941499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c1810c5d0 0x7f8c1810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/1059941499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c1810c000 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/1059941499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c18103200 0x7f8c181035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/1059941499 >> 172.21.15.33:0/1059941499 conn(0x7f8c180771d0 msgr2=0x7f8c180775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/1059941499 shutdown_connections 2024-04-23T18:50:26.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.697+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/1059941499 wait complete. 2024-04-23T18:50:26.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.698+0000 7f8c1e0a0640 1 Processor -- start 2024-04-23T18:50:26.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.698+0000 7f8c1e0a0640 1 -- start start 2024-04-23T18:50:26.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c18103200 0x7f8c181a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:26.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:26.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1810c5d0 0x7f8c1819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18112210 con 0x7f8c18103b20 2024-04-23T18:50:26.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18112090 con 0x7f8c1810c5d0 2024-04-23T18:50:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c18112390 con 0x7f8c18103200 2024-04-23T18:50:26.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39922/0 (socket says 172.21.15.33:39922) 2024-04-23T18:50:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 -- 172.21.15.33:0/2371200270 learned_addr learned my addr 172.21.15.33:0/2371200270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c177fe640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c18103200 0x7f8c181a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c17fff640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1810c5d0 0x7f8c1819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 -- 172.21.15.33:0/2371200270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c18103200 msgr2=0x7f8c181a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:27.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c18103200 0x7f8c181a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:27.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 -- 172.21.15.33:0/2371200270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1810c5d0 msgr2=0x7f8c1819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1810c5d0 0x7f8c1819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 -- 172.21.15.33:0/2371200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1819af70 con 0x7f8c18103b20 2024-04-23T18:50:27.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8c08005940 tx=0x7f8c080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:27.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c14ff9640 1 -- 172.21.15.33:0/2371200270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0803f030 con 0x7f8c18103b20 2024-04-23T18:50:27.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c14ff9640 1 -- 172.21.15.33:0/2371200270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c08003cb0 con 0x7f8c18103b20 2024-04-23T18:50:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.699+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c1819b200 con 0x7f8c18103b20 2024-04-23T18:50:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.700+0000 7f8c14ff9640 1 -- 172.21.15.33:0/2371200270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08035070 con 0x7f8c18103b20 2024-04-23T18:50:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.700+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c181a7390 con 0x7f8c18103b20 2024-04-23T18:50:27.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c14ff9640 1 -- 172.21.15.33:0/2371200270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8c08042030 con 0x7f8c18103b20 2024-04-23T18:50:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 Processor -- start 2024-04-23T18:50:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 -- start start 2024-04-23T18:50:27.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56392390a970 con 0x5639239b0800 2024-04-23T18:50:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5639238a74a0 con 0x5639239a5f40 2024-04-23T18:50:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.702+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563923906930 con 0x5639239a5620 2024-04-23T18:50:27.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:27.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:27.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39928/0 (socket says 172.21.15.33:39928) 2024-04-23T18:50:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 -- 172.21.15.33:0/1973831038 learned_addr learned my addr 172.21.15.33:0/1973831038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c14ff9640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 0x7f8c040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:28.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c177fe640 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c17fff640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 0x7f8c040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c14ff9640 1 -- 172.21.15.33:0/2371200270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f8c080f0da0 con 0x7f8c18103b20 2024-04-23T18:50:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 -- 172.21.15.33:0/1973831038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 msgr2=0x5639239a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:28.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 -- 172.21.15.33:0/1973831038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 msgr2=0x5639239b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:28.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 -- 172.21.15.33:0/1973831038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56392361d210 con 0x5639239b0800 2024-04-23T18:50:28.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c177fe640 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8c080bc9f0 tx=0x7f8c0802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8c0037f895192c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c0dffb640 1 -- 172.21.15.33:0/1973831038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08049070 con 0x5639239b0800 2024-04-23T18:50:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c17fff640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 0x7f8c040a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806b860 tx=0x7f8c10004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:28.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c0dffb640 1 -- 172.21.15.33:0/1973831038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c080bd3f0 con 0x5639239b0800 2024-04-23T18:50:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.703+0000 7f8c0dffb640 1 -- 172.21.15.33:0/1973831038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c080b93c0 con 0x5639239b0800 2024-04-23T18:50:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1973831038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 msgr2=0x5639239b2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8c080bc9f0 tx=0x7f8c0802a480 comp rx=0 tx=0).stop 2024-04-23T18:50:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1973831038 shutdown_connections 2024-04-23T18:50:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239b0800 0x5639239b2bf0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1973831038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1973831038 >> 172.21.15.33:0/1973831038 conn(0x5639239a1b30 msgr2=0x5639239a1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:29.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1973831038 shutdown_connections 2024-04-23T18:50:29.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1973831038 wait complete. 2024-04-23T18:50:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.704+0000 7f8c20dbb740 1 -- start start 2024-04-23T18:50:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 Processor -- start 2024-04-23T18:50:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:29.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239ad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239b0800 0x5639239ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:29.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5639239b31f0 con 0x5639239a5620 2024-04-23T18:50:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5639238a50b0 con 0x5639239a5f40 2024-04-23T18:50:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639239b3370 con 0x5639239b0800 2024-04-23T18:50:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c20dbb740 10 client.0 _ll_register_callbacks cb 0x563923892d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:29.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:29.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39934/0 (socket says 172.21.15.33:39934) 2024-04-23T18:50:29.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c177fe640 1 -- 172.21.15.33:0/1821221540 learned_addr learned my addr 172.21.15.33:0/1821221540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:29.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c177fe640 1 -- 172.21.15.33:0/1821221540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239b0800 msgr2=0x5639239ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:29.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.705+0000 7f8c17fff640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239b0800 0x5639239ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c177fe640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239b0800 0x5639239ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c177fe640 1 -- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 msgr2=0x5639239ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:29.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c177fe640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c177fe640 1 -- 172.21.15.33:0/1821221540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639236e9620 con 0x5639239a5620 2024-04-23T18:50:29.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:29.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c177fe640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000df20 tx=0x7f8c00011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faa38f9225e41e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:29.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:30.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00005220 con 0x5639239a5620 2024-04-23T18:50:30.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f8c000053c0 con 0x5639239a5620 2024-04-23T18:50:30.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c080bd3f0 con 0x5639239a5620 2024-04-23T18:50:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8c000055c0 con 0x5639239a5620 2024-04-23T18:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c20dbb740 10 client.17958 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5639239bf0b0 con 0x5639239a5620 2024-04-23T18:50:30.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c080bd3f0 con 0x5639239a5620 2024-04-23T18:50:30.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f8c00012a00 con 0x5639239a5620 2024-04-23T18:50:30.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c20dbb740 10 client.17958 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.706+0000 7f8c20dbb740 10 client.17958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:30.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c20dbb740 20 client.17958 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5639239c0520 0x5639239c2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:30.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c20dbb740 4 client.17958 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:50:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563923918390 con 0x5639239c0520 2024-04-23T18:50:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5639239c0520 0x5639239c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5639239c0520 0x5639239c2950 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74573daec5ca212b server_cookie=5d1051304044af94 in_seq=0 out_seq=0 2024-04-23T18:50:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.707+0000 7f8bf6ffd640 10 client.17958 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/1821221540 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563923918390 con 0x5639239c0520 2024-04-23T18:50:31.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8bf6ffd640 10 client.17958 handle_command_reply: tid=0 2024-04-23T18:50:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 client.17958 shutdown 2024-04-23T18:50:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 msgr2=0x5639239b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000df20 tx=0x7f8c00011870 comp rx=0 tx=0).stop 2024-04-23T18:50:31.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 shutdown_connections 2024-04-23T18:50:31.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5639239c0520 0x5639239c2950 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:31.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239b0800 0x5639239ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:31.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239a5f40 0x5639239ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8bf6ffd640 0 client.17958 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/1821221540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239b0150 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 >> 172.21.15.33:0/1821221540 conn(0x5639239a1b30 msgr2=0x5639239a7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 shutdown_connections 2024-04-23T18:50:31.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 1 -- 172.21.15.33:0/1821221540 wait complete. 2024-04-23T18:50:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.708+0000 7f8c20dbb740 20 client.17958 trim_cache size 0 max 16384 2024-04-23T18:50:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 Processor -- start 2024-04-23T18:50:31.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 -- start start 2024-04-23T18:50:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:31.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5f40 0x5639239a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:31.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239c8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b17b0 con 0x5639239a5620 2024-04-23T18:50:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181af5a0 con 0x5639239b0800 2024-04-23T18:50:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.723+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b10e0 con 0x5639239a5f40 2024-04-23T18:50:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39950/0 (socket says 172.21.15.33:39950) 2024-04-23T18:50:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 -- 172.21.15.33:0/2692223096 learned_addr learned my addr 172.21.15.33:0/2692223096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:32.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5f40 0x5639239a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:32.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c17fff640 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 -- 172.21.15.33:0/2692223096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5f40 msgr2=0x5639239a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:32.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5f40 0x5639239a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 -- 172.21.15.33:0/2692223096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 msgr2=0x5639239c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 -- 172.21.15.33:0/2692223096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639239ac450 con 0x5639239a5620 2024-04-23T18:50:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c177fe640 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000d370 tx=0x7f8c0000d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e3783e504bd65e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:32.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c0d7fa640 1 -- 172.21.15.33:0/2692223096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0002a040 con 0x5639239a5620 2024-04-23T18:50:32.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c0d7fa640 1 -- 172.21.15.33:0/2692223096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0001b8a0 con 0x5639239a5620 2024-04-23T18:50:32.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.724+0000 7f8c0d7fa640 1 -- 172.21.15.33:0/2692223096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00012930 con 0x5639239a5620 2024-04-23T18:50:32.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- 172.21.15.33:0/2692223096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 msgr2=0x5639239bbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000d370 tx=0x7f8c0000d3a0 comp rx=0 tx=0).stop 2024-04-23T18:50:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- 172.21.15.33:0/2692223096 shutdown_connections 2024-04-23T18:50:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5f40 0x5639239a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/2692223096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5620 0x5639239bbff0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- 172.21.15.33:0/2692223096 >> 172.21.15.33:0/2692223096 conn(0x5639239a1b30 msgr2=0x5639239a0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- 172.21.15.33:0/2692223096 shutdown_connections 2024-04-23T18:50:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- 172.21.15.33:0/2692223096 wait complete. 2024-04-23T18:50:33.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 -- start start 2024-04-23T18:50:33.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.725+0000 7f8c20dbb740 1 Processor -- start 2024-04-23T18:50:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:33.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b0340 con 0x5639239a5f40 2024-04-23T18:50:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181afc70 con 0x5639239b0800 2024-04-23T18:50:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c181b0a10 con 0x5639239a5620 2024-04-23T18:50:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c20dbb740 10 client.0 _ll_register_callbacks cb 0x7f8c0000dab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:33.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39954/0 (socket says 172.21.15.33:39954) 2024-04-23T18:50:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 -- 172.21.15.33:0/54728362 learned_addr learned my addr 172.21.15.33:0/54728362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c17fff640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:33.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c177fe640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 -- 172.21.15.33:0/54728362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 msgr2=0x5639239b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 -- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 msgr2=0x5639239b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 -- 172.21.15.33:0/54728362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639239b2f60 con 0x5639239a5f40 2024-04-23T18:50:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8c16ffd640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8c08004650 tx=0x7f8c080bc930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a453aa17503ba565 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:34.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.726+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c08049040 con 0x5639239a5f40 2024-04-23T18:50:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 8498+0+0 (secure 0 0 0) 0x7f8c080b9360 con 0x5639239a5f40 2024-04-23T18:50:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c0001b8a0 con 0x5639239a5f40 2024-04-23T18:50:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8c080bd3f0 con 0x5639239a5f40 2024-04-23T18:50:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 10 client.17964 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563923a26a00 con 0x5639239a5f40 2024-04-23T18:50:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0001b8a0 con 0x5639239a5f40 2024-04-23T18:50:34.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f8c080ee720 con 0x5639239a5f40 2024-04-23T18:50:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 10 client.17964 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 10 client.17964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 20 client.17964 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563923a26fa0 0x563923a293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:34.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 4 client.17964 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:34.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563923912200 con 0x563923a26fa0 2024-04-23T18:50:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.727+0000 7f8c177fe640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563923a26fa0 0x563923a293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.728+0000 7f8c177fe640 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563923a26fa0 0x563923a293c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10334842eccd7ea3 server_cookie=9ace09afae864736 in_seq=0 out_seq=0 2024-04-23T18:50:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.728+0000 7f8bf6ffd640 10 client.17964 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:34.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8bf6ffd640 1 -- 172.21.15.33:0/54728362 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563923912200 con 0x563923a26fa0 2024-04-23T18:50:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8bf6ffd640 10 client.17964 handle_command_reply: tid=0 2024-04-23T18:50:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 client.17964 shutdown 2024-04-23T18:50:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 msgr2=0x5639239b5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:35.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8c08004650 tx=0x7f8c080bc930 comp rx=0 tx=0).stop 2024-04-23T18:50:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 shutdown_connections 2024-04-23T18:50:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563923a26fa0 0x563923a293c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:35.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5639239b0800 0x5639239b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:35.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5639239a5f40 0x5639239b5d30 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:35.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8bf6ffd640 0 client.17964 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:35.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 --2- 172.21.15.33:0/54728362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639239a5620 0x5639239b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 >> 172.21.15.33:0/54728362 conn(0x5639239a1b30 msgr2=0x5639239a0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:35.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 shutdown_connections 2024-04-23T18:50:35.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 1 -- 172.21.15.33:0/54728362 wait complete. 2024-04-23T18:50:35.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.732+0000 7f8c20dbb740 20 client.17964 trim_cache size 0 max 16384 2024-04-23T18:50:35.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:35.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:35.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:35.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:35.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 msgr2=0x7f8c040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:35.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 0x7f8c040a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806b860 tx=0x7f8c10004ab0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 msgr2=0x7f8c181a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f8c08005940 tx=0x7f8c080042f0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 shutdown_connections 2024-04-23T18:50:36.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f8c040a3dc0 0x7f8c040a6280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1810c5d0 0x7f8c1819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c18103b20 0x7f8c181a0b30 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 --2- 172.21.15.33:0/2371200270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c18103200 0x7f8c181a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 >> 172.21.15.33:0/2371200270 conn(0x7f8c180771d0 msgr2=0x7f8c180fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:36.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 shutdown_connections 2024-04-23T18:50:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:48:55.733+0000 7f8c1e0a0640 1 -- 172.21.15.33:0/2371200270 wait complete. 2024-04-23T18:50:36.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:55 smithi033 ceph-mon[29274]: pgmap v718: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:36.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:57 smithi033 ceph-mon[29274]: pgmap v719: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:36.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:58 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:50:36.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: pgmap v720: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:50:36.654 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:36.661 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:36.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:36.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:36.801 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:50:36.877 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:50:36.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:48:59 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:50:36.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:00 smithi033 ceph-mon[29274]: pgmap v721: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-23T18:50:36.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:00 smithi033 ceph-mon[29274]: pgmap v722: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-04-23T18:50:36.992 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:02 smithi033 ceph-mon[29274]: pgmap v723: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:50:37.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:04 smithi033 ceph-mon[29274]: pgmap v724: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:50:37.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:06 smithi033 ceph-mon[29274]: pgmap v725: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-23T18:50:37.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:08 smithi033 ceph-mon[29274]: pgmap v726: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 57 KiB/s wr, 33 op/s 2024-04-23T18:50:37.132 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:27 smithi122 ceph-mon[32546]: pgmap v772: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 408 B/s wr, 11 op/s 2024-04-23T18:50:37.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:29 smithi122 ceph-mon[32546]: pgmap v773: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:50:37.194 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:31 smithi122 ceph-mon[32546]: pgmap v774: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:50:37.201 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:33 smithi122 ceph-mon[32546]: pgmap v775: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:50:37.207 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:34 smithi122 ceph-mon[32546]: pgmap v776: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:37.255 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:27 smithi106 ceph-mon[32716]: pgmap v772: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 408 B/s wr, 11 op/s 2024-04-23T18:50:37.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:29 smithi106 ceph-mon[32716]: pgmap v773: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:50:37.467 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:31 smithi106 ceph-mon[32716]: pgmap v774: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:50:37.562 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:33 smithi106 ceph-mon[32716]: pgmap v775: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:50:37.656 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:34 smithi106 ceph-mon[32716]: pgmap v776: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:37.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:37 smithi122 ceph-mon[32546]: pgmap v777: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:37.801 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:37 smithi106 ceph-mon[32716]: pgmap v777: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 -- 172.21.15.33:0/1247076407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21901010f0 msgr2=0x7f21901014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 --2- 172.21.15.33:0/1247076407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21901010f0 0x7f21901014d0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f2178009900 tx=0x7f217802e820 comp rx=0 tx=0).stop 2024-04-23T18:50:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 -- 172.21.15.33:0/1247076407 shutdown_connections 2024-04-23T18:50:38.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 --2- 172.21.15.33:0/1247076407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2190106170 0x7f219010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:38.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 --2- 172.21.15.33:0/1247076407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2190101aa0 0x7f2190105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 --2- 172.21.15.33:0/1247076407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21901010f0 0x7f21901014d0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 -- 172.21.15.33:0/1247076407 >> 172.21.15.33:0/1247076407 conn(0x7f2190077fd0 msgr2=0x7f21900783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:38.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 -- 172.21.15.33:0/1247076407 shutdown_connections 2024-04-23T18:50:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.832+0000 7f2197049640 1 -- 172.21.15.33:0/1247076407 wait complete. 2024-04-23T18:50:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 Processor -- start 2024-04-23T18:50:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 -- start start 2024-04-23T18:50:38.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 0x7f2190196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:38.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:38.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f219019ada0 0x7f219019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2190110010 con 0x7f2190106170 2024-04-23T18:50:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f219010fe90 con 0x7f21901010f0 2024-04-23T18:50:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f2197049640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2190110190 con 0x7f219019ada0 2024-04-23T18:50:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f218ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f218ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35462/0 (socket says 172.21.15.33:35462) 2024-04-23T18:50:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f218ffff640 1 -- 172.21.15.33:0/1625658369 learned_addr learned my addr 172.21.15.33:0/1625658369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.833+0000 7f21955e6640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f219019ada0 0x7f219019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:39.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f2194de5640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 0x7f2190196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:39.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 -- 172.21.15.33:0/1625658369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f219019ada0 msgr2=0x7f219019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f219019ada0 0x7f219019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 -- 172.21.15.33:0/1625658369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 msgr2=0x7f2190196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:39.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 0x7f2190196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 -- 172.21.15.33:0/1625658369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219019b890 con 0x7f2190106170 2024-04-23T18:50:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f2194de5640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 0x7f2190196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218ffff640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f2180009990 tx=0x7f2180009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218dffb640 1 -- 172.21.15.33:0/1625658369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000c9e0 con 0x7f2190106170 2024-04-23T18:50:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218dffb640 1 -- 172.21.15.33:0/1625658369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21800054d0 con 0x7f2190106170 2024-04-23T18:50:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219019bb80 con 0x7f2190106170 2024-04-23T18:50:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f218dffb640 1 -- 172.21.15.33:0/1625658369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180004c40 con 0x7f2190106170 2024-04-23T18:50:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.834+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21901a3370 con 0x7f2190106170 2024-04-23T18:50:39.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.836+0000 7f218dffb640 1 -- 172.21.15.33:0/1625658369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2180004e60 con 0x7f2190106170 2024-04-23T18:50:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 Processor -- start 2024-04-23T18:50:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 -- start start 2024-04-23T18:50:39.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f218dffb640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 0x7f217c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:40.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792f40 0x55e1cb79d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb79d800 0x55e1cb79fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb6f7970 con 0x55e1cb792620 2024-04-23T18:50:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb6944a0 con 0x55e1cb792f40 2024-04-23T18:50:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2199e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb6f3930 con 0x55e1cb79d800 2024-04-23T18:50:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f21955e6640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 0x7f217c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f218dffb640 1 -- 172.21.15.33:0/1625658369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8586+0+0 (secure 0 0 0) 0x7f21800cffb0 con 0x7f2190106170 2024-04-23T18:50:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2194de5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:40.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2194de5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35464/0 (socket says 172.21.15.33:35464) 2024-04-23T18:50:40.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.837+0000 7f2194de5640 1 -- 172.21.15.33:0/3901459759 learned_addr learned my addr 172.21.15.33:0/3901459759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:40.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f218ffff640 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792f40 0x55e1cb79d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f21955e6640 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb79d800 0x55e1cb79fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 -- 172.21.15.33:0/3901459759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb79d800 msgr2=0x55e1cb79fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb79d800 0x55e1cb79fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 -- 172.21.15.33:0/3901459759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792f40 msgr2=0x55e1cb79d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792f40 0x55e1cb79d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:40.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 -- 172.21.15.33:0/3901459759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cb40a210 con 0x55e1cb792620 2024-04-23T18:50:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f21955e6640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 0x7f217c0a6300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2188009fd0 tx=0x7f2188013b00 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2194de5640 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178036db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e7e9dc99b625ba server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2186ffd640 1 -- 172.21.15.33:0/3901459759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178046020 con 0x55e1cb792620 2024-04-23T18:50:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2186ffd640 1 -- 172.21.15.33:0/3901459759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178036410 con 0x55e1cb792620 2024-04-23T18:50:40.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.838+0000 7f2186ffd640 1 -- 172.21.15.33:0/3901459759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178005470 con 0x55e1cb792620 2024-04-23T18:50:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- 172.21.15.33:0/3901459759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 msgr2=0x55e1cb792a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2178000c00 tx=0x7f2178036db0 comp rx=0 tx=0).stop 2024-04-23T18:50:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- 172.21.15.33:0/3901459759 shutdown_connections 2024-04-23T18:50:41.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb79d800 0x55e1cb79fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:41.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792f40 0x55e1cb79d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 --2- 172.21.15.33:0/3901459759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb792a00 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:41.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- 172.21.15.33:0/3901459759 >> 172.21.15.33:0/3901459759 conn(0x55e1cb78eb30 msgr2=0x55e1cb78ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- 172.21.15.33:0/3901459759 shutdown_connections 2024-04-23T18:50:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- 172.21.15.33:0/3901459759 wait complete. 2024-04-23T18:50:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 -- start start 2024-04-23T18:50:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.839+0000 7f2199e34740 1 Processor -- start 2024-04-23T18:50:41.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 0x55e1cb79d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb79a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb7a03d0 con 0x55e1cb79d800 2024-04-23T18:50:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb6920b0 con 0x55e1cb792620 2024-04-23T18:50:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e1cb7a0550 con 0x55e1cb792f40 2024-04-23T18:50:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2199e34740 10 client.0 _ll_register_callbacks cb 0x55e1cb67fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35470/0 (socket says 172.21.15.33:35470) 2024-04-23T18:50:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 -- 172.21.15.33:0/4119932663 learned_addr learned my addr 172.21.15.33:0/4119932663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2194de5640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 0x55e1cb79d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f218ffff640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb79a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 -- 172.21.15.33:0/4119932663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 msgr2=0x55e1cb79a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb79a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 -- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 msgr2=0x55e1cb79d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 0x55e1cb79d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:42.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f21955e6640 1 -- 172.21.15.33:0/4119932663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cb4d6620 con 0x55e1cb79d800 2024-04-23T18:50:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f218ffff640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb79a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.840+0000 7f2194de5640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 0x55e1cb79d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f21955e6640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f218800fb90 tx=0x7f2188004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f38e87ae583c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188024020 con 0x55e1cb79d800 2024-04-23T18:50:42.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 8586+0+0 (secure 0 0 0) 0x7f2188010d40 con 0x55e1cb79d800 2024-04-23T18:50:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2178036410 con 0x55e1cb79d800 2024-04-23T18:50:42.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f218802f050 con 0x55e1cb79d800 2024-04-23T18:50:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 10 client.17976 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1cb7ac370 con 0x55e1cb79d800 2024-04-23T18:50:42.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178036410 con 0x55e1cb79d800 2024-04-23T18:50:42.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f218801f070 con 0x55e1cb79d800 2024-04-23T18:50:42.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 10 client.17976 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 10 client.17976 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 20 client.17976 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb7ad880 0x55e1cb7afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 4 client.17976 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:50:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.841+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1cb705390 con 0x55e1cb7ad880 2024-04-23T18:50:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.842+0000 7f2194de5640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb7ad880 0x55e1cb7afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:43.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.842+0000 7f216ffff640 10 client.17976 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.842+0000 7f2194de5640 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb7ad880 0x55e1cb7afcb0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81c21b1855fb6d94 server_cookie=5b73fe094a87afa7 in_seq=0 out_seq=0 2024-04-23T18:50:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f216ffff640 1 -- 172.21.15.33:0/4119932663 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1cb705390 con 0x55e1cb7ad880 2024-04-23T18:50:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f216ffff640 10 client.17976 handle_command_reply: tid=0 2024-04-23T18:50:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 client.17976 shutdown 2024-04-23T18:50:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 msgr2=0x55e1cb79a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:43.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f218800fb90 tx=0x7f2188004ab0 comp rx=0 tx=0).stop 2024-04-23T18:50:43.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 shutdown_connections 2024-04-23T18:50:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb7ad880 0x55e1cb7afcb0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:43.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb79d800 0x55e1cb79a9f0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb79a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 --2- 172.21.15.33:0/4119932663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb792620 0x55e1cb79d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f216ffff640 0 client.17976 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 >> 172.21.15.33:0/4119932663 conn(0x55e1cb78eb30 msgr2=0x55e1cb7948b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 shutdown_connections 2024-04-23T18:50:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.843+0000 7f2199e34740 1 -- 172.21.15.33:0/4119932663 wait complete. 2024-04-23T18:50:43.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.844+0000 7f2199e34740 20 client.17976 trim_cache size 0 max 16384 2024-04-23T18:50:43.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 Processor -- start 2024-04-23T18:50:43.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 -- start start 2024-04-23T18:50:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb7993b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:44.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ad4f0 con 0x55e1cb792620 2024-04-23T18:50:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ab2e0 con 0x55e1cb79d800 2024-04-23T18:50:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2199e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ace20 con 0x55e1cb792f40 2024-04-23T18:50:44.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35472/0 (socket says 172.21.15.33:35472) 2024-04-23T18:50:44.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 -- 172.21.15.33:0/1962534661 learned_addr learned my addr 172.21.15.33:0/1962534661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:44.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f21955e6640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 -- 172.21.15.33:0/1962534661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 msgr2=0x55e1cb7993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f218ffff640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb7993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb7993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:44.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 -- 172.21.15.33:0/1962534661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 msgr2=0x55e1cb7b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:44.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 -- 172.21.15.33:0/1962534661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cb7938d0 con 0x55e1cb792620 2024-04-23T18:50:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f218ffff640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb7993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f21955e6640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:44.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.859+0000 7f2194de5640 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f217802ed40 tx=0x7f2178005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc9014583c4b295 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f21867fc640 1 -- 172.21.15.33:0/1962534661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178053070 con 0x55e1cb792620 2024-04-23T18:50:44.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f21867fc640 1 -- 172.21.15.33:0/1962534661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178036410 con 0x55e1cb792620 2024-04-23T18:50:44.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f21867fc640 1 -- 172.21.15.33:0/1962534661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178046040 con 0x55e1cb792620 2024-04-23T18:50:44.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 -- 172.21.15.33:0/1962534661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 msgr2=0x55e1cb7a9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f217802ed40 tx=0x7f2178005a20 comp rx=0 tx=0).stop 2024-04-23T18:50:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 -- 172.21.15.33:0/1962534661 shutdown_connections 2024-04-23T18:50:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792f40 0x55e1cb7993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:45.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 --2- 172.21.15.33:0/1962534661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792620 0x55e1cb7a9290 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:45.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 -- 172.21.15.33:0/1962534661 >> 172.21.15.33:0/1962534661 conn(0x55e1cb78eb30 msgr2=0x55e1cb78dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:45.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 -- 172.21.15.33:0/1962534661 shutdown_connections 2024-04-23T18:50:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.860+0000 7f2199e34740 1 -- 172.21.15.33:0/1962534661 wait complete. 2024-04-23T18:50:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 -- start start 2024-04-23T18:50:45.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 Processor -- start 2024-04-23T18:50:45.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792620 0x55e1cb7a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:45.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ac080 con 0x55e1cb792f40 2024-04-23T18:50:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ab9b0 con 0x55e1cb79d800 2024-04-23T18:50:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21901ac750 con 0x55e1cb792620 2024-04-23T18:50:45.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f2199e34740 10 client.0 _ll_register_callbacks cb 0x7f218800f810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:45.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f218ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f218ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35488/0 (socket says 172.21.15.33:35488) 2024-04-23T18:50:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.861+0000 7f218ffff640 1 -- 172.21.15.33:0/1804492715 learned_addr learned my addr 172.21.15.33:0/1804492715 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:45.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2194de5640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792620 0x55e1cb7a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f21955e6640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 -- 172.21.15.33:0/1804492715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792620 msgr2=0x55e1cb7a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792620 0x55e1cb7a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:45.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 -- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 msgr2=0x55e1cb7a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 -- 172.21.15.33:0/1804492715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cb7a3a30 con 0x55e1cb792f40 2024-04-23T18:50:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f21955e6640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f218ffff640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f21800cd6c0 tx=0x7f218009b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319bf9ba27e113f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218009beb0 con 0x55e1cb792f40 2024-04-23T18:50:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 8586+0+0 (secure 0 0 0) 0x7f21800b52f0 con 0x55e1cb792f40 2024-04-23T18:50:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2178036410 con 0x55e1cb792f40 2024-04-23T18:50:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f21800a0260 con 0x55e1cb792f40 2024-04-23T18:50:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 10 client.17982 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:46.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1cb813cc0 con 0x55e1cb792f40 2024-04-23T18:50:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178036410 con 0x55e1cb792f40 2024-04-23T18:50:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f218007bce0 con 0x55e1cb792f40 2024-04-23T18:50:46.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 10 client.17982 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:46.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:46.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.862+0000 7f2199e34740 20 client.17982 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb814440 0x55e1cb816860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f2199e34740 4 client.17982 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1cb6ff200 con 0x55e1cb814440 2024-04-23T18:50:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f2194de5640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb814440 0x55e1cb816860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f2194de5640 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb814440 0x55e1cb816860 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a906954d3a5049d3 server_cookie=2354cb9b9ab414fd in_seq=0 out_seq=0 2024-04-23T18:50:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.863+0000 7f216ffff640 10 client.17982 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f216ffff640 1 -- 172.21.15.33:0/1804492715 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1cb6ff200 con 0x55e1cb814440 2024-04-23T18:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f216ffff640 10 client.17982 handle_command_reply: tid=0 2024-04-23T18:50:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 client.17982 shutdown 2024-04-23T18:50:47.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 msgr2=0x55e1cb7a2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f21800cd6c0 tx=0x7f218009b900 comp rx=0 tx=0).stop 2024-04-23T18:50:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 shutdown_connections 2024-04-23T18:50:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e1cb814440 0x55e1cb816860 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e1cb79d800 0x55e1cb7a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:47.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e1cb792f40 0x55e1cb7a2e00 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f216ffff640 0 client.17982 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 --2- 172.21.15.33:0/1804492715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1cb792620 0x55e1cb7a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 >> 172.21.15.33:0/1804492715 conn(0x55e1cb78eb30 msgr2=0x55e1cb7a8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 shutdown_connections 2024-04-23T18:50:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.870+0000 7f2199e34740 1 -- 172.21.15.33:0/1804492715 wait complete. 2024-04-23T18:50:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2199e34740 20 client.17982 trim_cache size 0 max 16384 2024-04-23T18:50:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 msgr2=0x7f217c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:47.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 0x7f217c0a6300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2188009fd0 tx=0x7f2188013b00 comp rx=0 tx=0).stop 2024-04-23T18:50:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 msgr2=0x7f2190196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f2180009990 tx=0x7f2180009e50 comp rx=0 tx=0).stop 2024-04-23T18:50:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 shutdown_connections 2024-04-23T18:50:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f217c0a3e40 0x7f217c0a6300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f219019ada0 0x7f219019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2190106170 0x7f2190196a70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 --2- 172.21.15.33:0/1625658369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21901010f0 0x7f2190196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 >> 172.21.15.33:0/1625658369 conn(0x7f2190077fd0 msgr2=0x7f21900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 shutdown_connections 2024-04-23T18:50:48.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:08.871+0000 7f2197049640 1 -- 172.21.15.33:0/1625658369 wait complete. 2024-04-23T18:50:48.371 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:39 smithi122 ceph-mon[32546]: pgmap v778: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:48.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:41 smithi122 ceph-mon[32546]: pgmap v779: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:48.438 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:43 smithi122 ceph-mon[32546]: pgmap v780: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:48.446 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:45 smithi122 ceph-mon[32546]: pgmap v781: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:50:48.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:47 smithi122 ceph-mon[32546]: pgmap v782: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:48.617 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:39 smithi106 ceph-mon[32716]: pgmap v778: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:48.707 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:41 smithi106 ceph-mon[32716]: pgmap v779: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:48.716 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:43 smithi106 ceph-mon[32716]: pgmap v780: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:48.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:45 smithi106 ceph-mon[32716]: pgmap v781: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:50:48.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:47 smithi106 ceph-mon[32716]: pgmap v782: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:48.780 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:09 smithi033 ceph-mon[29274]: osdmap e130: 12 total, 12 up, 12 in 2024-04-23T18:50:48.789 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:10 smithi033 ceph-mon[29274]: pgmap v728: 97 pgs: 97 active+clean; 15 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:50:48.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:12 smithi033 ceph-mon[29274]: pgmap v729: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:50:48.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:14 smithi033 ceph-mon[29274]: pgmap v730: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:50:49.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:16 smithi033 ceph-mon[29274]: pgmap v731: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:50:49.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:16 smithi033 ceph-mon[29274]: osdmap e131: 12 total, 12 up, 12 in 2024-04-23T18:50:49.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:18 smithi033 ceph-mon[29274]: pgmap v733: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 919 B/s wr, 12 op/s 2024-04-23T18:50:49.229 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:20 smithi033 ceph-mon[29274]: pgmap v734: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 818 B/s wr, 10 op/s 2024-04-23T18:50:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.971+0000 7fd48d529640 1 -- 172.21.15.33:0/2961054312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd488101a70 msgr2=0x7fd488105ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.971+0000 7fd48d529640 1 --2- 172.21.15.33:0/2961054312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd488101a70 0x7fd488105ab0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd470009900 tx=0x7fd47002a410 comp rx=0 tx=0).stop 2024-04-23T18:50:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.971+0000 7fd48d529640 1 -- 172.21.15.33:0/2961054312 shutdown_connections 2024-04-23T18:50:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 --2- 172.21.15.33:0/2961054312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 0x7fd48810dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 --2- 172.21.15.33:0/2961054312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd488101a70 0x7fd488105ab0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 --2- 172.21.15.33:0/2961054312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881010c0 0x7fd4881014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:49.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 -- 172.21.15.33:0/2961054312 >> 172.21.15.33:0/2961054312 conn(0x7fd488077fd0 msgr2=0x7fd4880783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 -- 172.21.15.33:0/2961054312 shutdown_connections 2024-04-23T18:50:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 -- 172.21.15.33:0/2961054312 wait complete. 2024-04-23T18:50:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 Processor -- start 2024-04-23T18:50:49.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 -- start start 2024-04-23T18:50:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.972+0000 7fd48d529640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 0x7fd488103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 0x7fd4881a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881110d0 con 0x7fd4881010c0 2024-04-23T18:50:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd488110f50 con 0x7fd488106190 2024-04-23T18:50:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd488111250 con 0x7fd4881035d0 2024-04-23T18:50:49.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50470/0 (socket says 172.21.15.33:50470) 2024-04-23T18:50:50.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 -- 172.21.15.33:0/2456922948 learned_addr learned my addr 172.21.15.33:0/2456922948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:50.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd47ffff640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 0x7fd488103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 0x7fd4881a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:50.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 -- 172.21.15.33:0/2456922948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 msgr2=0x7fd4881a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:50.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 0x7fd4881a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:50.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 -- 172.21.15.33:0/2456922948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 msgr2=0x7fd488103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 0x7fd488103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 -- 172.21.15.33:0/2456922948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4881a6fe0 con 0x7fd4881010c0 2024-04-23T18:50:50.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 0x7fd4881a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd47800d770 tx=0x7fd47800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:50.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd484ff9640 1 -- 172.21.15.33:0/2456922948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478017070 con 0x7fd4881010c0 2024-04-23T18:50:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd484ff9640 1 -- 172.21.15.33:0/2456922948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4780054d0 con 0x7fd4881010c0 2024-04-23T18:50:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd484ff9640 1 -- 172.21.15.33:0/2456922948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478004c30 con 0x7fd4881010c0 2024-04-23T18:50:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.973+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4881a72d0 con 0x7fd4881010c0 2024-04-23T18:50:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.974+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4881a77a0 con 0x7fd4881010c0 2024-04-23T18:50:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.976+0000 7fd484ff9640 1 -- 172.21.15.33:0/2456922948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd478004dd0 con 0x7fd4881010c0 2024-04-23T18:50:50.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.976+0000 7fd490240740 1 Processor -- start 2024-04-23T18:50:50.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.976+0000 7fd490240740 1 -- start start 2024-04-23T18:50:50.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:50.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b8bbaf970 con 0x562b8bc4af40 2024-04-23T18:50:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b8bb4c4a0 con 0x562b8bc4a620 2024-04-23T18:50:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd490240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b8bbab930 con 0x562b8bc55800 2024-04-23T18:50:50.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50472/0 (socket says 172.21.15.33:50472) 2024-04-23T18:50:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 -- 172.21.15.33:0/2958050950 learned_addr learned my addr 172.21.15.33:0/2958050950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd484ff9640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 0x7fd4740a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:50.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:51.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd47ffff640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:51.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 -- 172.21.15.33:0/2958050950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 msgr2=0x562b8bc57bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc57bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 -- 172.21.15.33:0/2958050950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 msgr2=0x562b8bc4aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:51.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc4aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd486ffd640 1 -- 172.21.15.33:0/2958050950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8b8c2210 con 0x562b8bc4af40 2024-04-23T18:50:51.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc4aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:50:51.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 0x7fd4740a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.977+0000 7fd484ff9640 1 -- 172.21.15.33:0/2456922948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fd4780d0330 con 0x7fd4881010c0 2024-04-23T18:50:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd486ffd640 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd4780cd980 tx=0x7fd478022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccef15ad15372db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:51.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd47d7fa640 1 -- 172.21.15.33:0/2958050950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478005680 con 0x562b8bc4af40 2024-04-23T18:50:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd4877fe640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 0x7fd4740a62b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd47d7fa640 1 -- 172.21.15.33:0/2958050950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47809c960 con 0x562b8bc4af40 2024-04-23T18:50:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd47d7fa640 1 -- 172.21.15.33:0/2958050950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47801e120 con 0x562b8bc4af40 2024-04-23T18:50:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd490240740 1 -- 172.21.15.33:0/2958050950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 msgr2=0x562b8bc552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.978+0000 7fd490240740 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd4780cd980 tx=0x7fd478022000 comp rx=0 tx=0).stop 2024-04-23T18:50:51.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- 172.21.15.33:0/2958050950 shutdown_connections 2024-04-23T18:50:51.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc57bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc552c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:51.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- 172.21.15.33:0/2958050950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc4aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:51.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- 172.21.15.33:0/2958050950 >> 172.21.15.33:0/2958050950 conn(0x562b8bc46b30 msgr2=0x562b8bc46f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:51.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- 172.21.15.33:0/2958050950 shutdown_connections 2024-04-23T18:50:51.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- 172.21.15.33:0/2958050950 wait complete. 2024-04-23T18:50:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- start start 2024-04-23T18:50:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 Processor -- start 2024-04-23T18:50:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc4af40 0x562b8bc52480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b8bc58370 con 0x562b8bc55800 2024-04-23T18:50:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562b8bb4a0b0 con 0x562b8bc4a620 2024-04-23T18:50:52.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b8bc584f0 con 0x562b8bc4af40 2024-04-23T18:50:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd490240740 10 client.0 _ll_register_callbacks cb 0x562b8bb37d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd4877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd4877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50476/0 (socket says 172.21.15.33:50476) 2024-04-23T18:50:52.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.979+0000 7fd4877fe640 1 -- 172.21.15.33:0/1835748201 learned_addr learned my addr 172.21.15.33:0/1835748201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd47ffff640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc4af40 0x562b8bc52480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd486ffd640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 -- 172.21.15.33:0/1835748201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc4af40 msgr2=0x562b8bc52480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc4af40 0x562b8bc52480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:52.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 -- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 msgr2=0x562b8bc55190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc55190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 -- 172.21.15.33:0/1835748201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8b98e620 con 0x562b8bc55800 2024-04-23T18:50:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd4877fe640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd480009e40 tx=0x7fd480004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c588db5aa1d212c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd46e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48001ed70 con 0x562b8bc55800 2024-04-23T18:50:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fd480028470 con 0x562b8bc55800 2024-04-23T18:50:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47809c960 con 0x562b8bc55800 2024-04-23T18:50:52.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd480013e30 con 0x562b8bc55800 2024-04-23T18:50:53.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd490240740 10 client.17994 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:53.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.980+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562b8bc64370 con 0x562b8bc55800 2024-04-23T18:50:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47809c960 con 0x562b8bc55800 2024-04-23T18:50:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7fd48000f040 con 0x562b8bc55800 2024-04-23T18:50:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 10 client.17994 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 20 client.17994 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bc65830 0x562b8bc67c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:53.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 4 client.17994 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:50:53.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8bbbd390 con 0x562b8bc65830 2024-04-23T18:50:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd486ffd640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bc65830 0x562b8bc67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd486ffd640 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bc65830 0x562b8bc67c20 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=962bdf3adbec1da3 server_cookie=2c638a4044856754 in_seq=0 out_seq=0 2024-04-23T18:50:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.981+0000 7fd46e7fc640 10 client.17994 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd46e7fc640 1 -- 172.21.15.33:0/1835748201 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b8bbbd390 con 0x562b8bc65830 2024-04-23T18:50:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd46e7fc640 10 client.17994 handle_command_reply: tid=0 2024-04-23T18:50:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 client.17994 shutdown 2024-04-23T18:50:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 msgr2=0x562b8bc52a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd480009e40 tx=0x7fd480004ab0 comp rx=0 tx=0).stop 2024-04-23T18:50:53.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 shutdown_connections 2024-04-23T18:50:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bc65830 0x562b8bc67c20 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:53.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc55800 0x562b8bc52a00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc4af40 0x562b8bc52480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:53.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 --2- 172.21.15.33:0/1835748201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd46e7fc640 0 client.17994 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 >> 172.21.15.33:0/1835748201 conn(0x562b8bc46b30 msgr2=0x562b8bc4c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.982+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 shutdown_connections 2024-04-23T18:50:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.983+0000 7fd490240740 1 -- 172.21.15.33:0/1835748201 wait complete. 2024-04-23T18:50:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.983+0000 7fd490240740 20 client.17994 trim_cache size 0 max 16384 2024-04-23T18:50:54.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 Processor -- start 2024-04-23T18:50:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 -- start start 2024-04-23T18:50:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 0x562b8bc6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc6da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b18a0 con 0x562b8bc4a620 2024-04-23T18:50:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881af690 con 0x562b8bc4af40 2024-04-23T18:50:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd490240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b11d0 con 0x562b8bc55800 2024-04-23T18:50:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50480/0 (socket says 172.21.15.33:50480) 2024-04-23T18:50:54.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd486ffd640 1 -- 172.21.15.33:0/543669894 learned_addr learned my addr 172.21.15.33:0/543669894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:54.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd4877fe640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd47ffff640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 0x562b8bc6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd486ffd640 1 -- 172.21.15.33:0/543669894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 msgr2=0x562b8bc6da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.998+0000 7fd486ffd640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc6da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd486ffd640 1 -- 172.21.15.33:0/543669894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 msgr2=0x562b8bc6d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd486ffd640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 0x562b8bc6d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:54.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd486ffd640 1 -- 172.21.15.33:0/543669894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8bc422e0 con 0x562b8bc4a620 2024-04-23T18:50:54.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd47ffff640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 0x562b8bc6d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:50:54.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd486ffd640 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd47800d740 tx=0x7fd478009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ba4564c75b216b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd47cff9640 1 -- 172.21.15.33:0/543669894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd478098500 con 0x562b8bc4a620 2024-04-23T18:50:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd47cff9640 1 -- 172.21.15.33:0/543669894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47809c960 con 0x562b8bc4a620 2024-04-23T18:50:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd47cff9640 1 -- 172.21.15.33:0/543669894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4780cdc90 con 0x562b8bc4a620 2024-04-23T18:50:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd490240740 1 -- 172.21.15.33:0/543669894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 msgr2=0x562b8bc61230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:21.999+0000 7fd490240740 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd47800d740 tx=0x7fd478009f90 comp rx=0 tx=0).stop 2024-04-23T18:50:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- 172.21.15.33:0/543669894 shutdown_connections 2024-04-23T18:50:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bc6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4af40 0x562b8bc6d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- 172.21.15.33:0/543669894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4a620 0x562b8bc61230 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:55.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- 172.21.15.33:0/543669894 >> 172.21.15.33:0/543669894 conn(0x562b8bc46b30 msgr2=0x562b8bc52440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- 172.21.15.33:0/543669894 shutdown_connections 2024-04-23T18:50:55.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- 172.21.15.33:0/543669894 wait complete. 2024-04-23T18:50:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- start start 2024-04-23T18:50:55.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 Processor -- start 2024-04-23T18:50:55.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc59850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bccada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b0430 con 0x562b8bc4af40 2024-04-23T18:50:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881afd60 con 0x562b8bc4a620 2024-04-23T18:50:55.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4881b0b00 con 0x562b8bc55800 2024-04-23T18:50:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd490240740 10 client.0 _ll_register_callbacks cb 0x7fd480013590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:50:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50494/0 (socket says 172.21.15.33:50494) 2024-04-23T18:50:55.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.000+0000 7fd47ffff640 1 -- 172.21.15.33:0/3582011836 learned_addr learned my addr 172.21.15.33:0/3582011836 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:50:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd4877fe640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bccada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:55.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd486ffd640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc59850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:55.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 -- 172.21.15.33:0/3582011836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 msgr2=0x562b8bccada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bccada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:56.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 -- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 msgr2=0x562b8bc59850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:50:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc59850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 -- 172.21.15.33:0/3582011836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8bc67f90 con 0x562b8bc4af40 2024-04-23T18:50:56.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd47ffff640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd48806a650 tx=0x7fd4700316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa139ffe5498891 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:50:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd46e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:50:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470035040 con 0x562b8bc4af40 2024-04-23T18:50:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fd470004240 con 0x562b8bc4af40 2024-04-23T18:50:56.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd47809c960 con 0x562b8bc4af40 2024-04-23T18:50:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd470004440 con 0x562b8bc4af40 2024-04-23T18:50:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd490240740 10 client.18000 fetch_fsmap learned FSMap version 43 2024-04-23T18:50:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562b8bccc340 con 0x562b8bc4af40 2024-04-23T18:50:56.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.001+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd47809c960 con 0x562b8bc4af40 2024-04-23T18:50:56.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7fd470004c50 con 0x562b8bc4af40 2024-04-23T18:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 10 client.18000 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:50:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 10 client.18000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:50:56.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 20 client.18000 populate_metadata read hostname 'smithi033' 2024-04-23T18:50:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bcccaf0 0x562b8bccef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:56.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 4 client.18000 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:50:56.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b8bbb7200 con 0x562b8bcccaf0 2024-04-23T18:50:57.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd486ffd640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bcccaf0 0x562b8bccef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd46e7fc640 10 client.18000 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.002+0000 7fd486ffd640 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bcccaf0 0x562b8bccef10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ce2f91cdf3eee6 server_cookie=477cb32979bda55 in_seq=0 out_seq=0 2024-04-23T18:50:57.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.006+0000 7fd46e7fc640 1 -- 172.21.15.33:0/3582011836 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b8bbb7200 con 0x562b8bcccaf0 2024-04-23T18:50:57.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.006+0000 7fd46e7fc640 10 client.18000 handle_command_reply: tid=0 2024-04-23T18:50:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.006+0000 7fd490240740 1 client.18000 shutdown 2024-04-23T18:50:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 msgr2=0x562b8bc59d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:57.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd48806a650 tx=0x7fd4700316d0 comp rx=0 tx=0).stop 2024-04-23T18:50:57.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 shutdown_connections 2024-04-23T18:50:57.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562b8bcccaf0 0x562b8bccef10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:57.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b8bc55800 0x562b8bccada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:57.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b8bc4af40 0x562b8bc59d90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 --2- 172.21.15.33:0/3582011836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562b8bc4a620 0x562b8bc59850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:57.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd46e7fc640 0 client.18000 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:50:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 >> 172.21.15.33:0/3582011836 conn(0x562b8bc46b30 msgr2=0x562b8bc4bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 shutdown_connections 2024-04-23T18:50:57.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 1 -- 172.21.15.33:0/3582011836 wait complete. 2024-04-23T18:50:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd490240740 20 client.18000 trim_cache size 0 max 16384 2024-04-23T18:50:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:50:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:50:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:50:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:50:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 msgr2=0x7fd4740a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 0x7fd4740a62b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd480005fd0 tx=0x7fd480004a40 comp rx=0 tx=0).stop 2024-04-23T18:50:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 msgr2=0x7fd488102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:58.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fd47800d770 tx=0x7fd47800dc30 comp rx=0 tx=0).stop 2024-04-23T18:50:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 shutdown_connections 2024-04-23T18:50:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd4740a3df0 0x7fd4740a62b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4881035d0 0x7fd4881a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd488106190 0x7fd488103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:58.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 --2- 172.21.15.33:0/2456922948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4881010c0 0x7fd488102b50 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:58.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 >> 172.21.15.33:0/2456922948 conn(0x7fd488077fd0 msgr2=0x7fd48806aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 shutdown_connections 2024-04-23T18:50:58.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:22.007+0000 7fd48d529640 1 -- 172.21.15.33:0/2456922948 wait complete. 2024-04-23T18:50:58.629 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:22 smithi033 ceph-mon[29274]: pgmap v735: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:50:58.741 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:24 smithi033 ceph-mon[29274]: pgmap v736: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:50:58.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:26 smithi033 ceph-mon[29274]: pgmap v737: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T18:50:58.800 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:28 smithi033 ceph-mon[29274]: pgmap v738: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 273 B/s wr, 9 op/s 2024-04-23T18:50:58.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:30 smithi033 ceph-mon[29274]: pgmap v739: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:58.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:31 smithi033 ceph-mon[29274]: pgmap v740: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:59.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:34 smithi033 ceph-mon[29274]: pgmap v741: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.038 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:49 smithi122 ceph-mon[32546]: pgmap v783: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.045 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:51 smithi122 ceph-mon[32546]: pgmap v784: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.052 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:53 smithi122 ceph-mon[32546]: pgmap v785: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:59.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:55 smithi122 ceph-mon[32546]: pgmap v786: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.066 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:57 smithi122 ceph-mon[32546]: pgmap v787: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:59.076 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:49 smithi106 ceph-mon[32716]: pgmap v783: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.145 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:51 smithi106 ceph-mon[32716]: pgmap v784: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:53 smithi106 ceph-mon[32716]: pgmap v785: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:59.184 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:55 smithi106 ceph-mon[32716]: pgmap v786: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:50:59.237 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:57 smithi106 ceph-mon[32716]: pgmap v787: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:50:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 -- 172.21.15.33:0/4014635738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781072a0 msgr2=0x7fe67810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:50:59.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 --2- 172.21.15.33:0/4014635738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781072a0 0x7fe67810ee70 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe670009ca0 tx=0x7fe67002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:50:59.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 -- 172.21.15.33:0/4014635738 shutdown_connections 2024-04-23T18:50:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 --2- 172.21.15.33:0/4014635738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781072a0 0x7fe67810ee70 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:59.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 --2- 172.21.15.33:0/4014635738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe678102b80 0x7fe678106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 --2- 172.21.15.33:0/4014635738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe6781021d0 0x7fe6781025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:50:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 -- 172.21.15.33:0/4014635738 >> 172.21.15.33:0/4014635738 conn(0x7fe6780793e0 msgr2=0x7fe6780797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:50:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 -- 172.21.15.33:0/4014635738 shutdown_connections 2024-04-23T18:50:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.108+0000 7fe67cf01640 1 -- 172.21.15.33:0/4014635738 wait complete. 2024-04-23T18:50:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.109+0000 7fe67cf01640 1 Processor -- start 2024-04-23T18:50:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.109+0000 7fe67cf01640 1 -- start start 2024-04-23T18:50:59.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6781072a0 0x7fe6781a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe67819a660 0x7fe67819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:50:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112210 con 0x7fe6781021d0 2024-04-23T18:50:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112090 con 0x7fe67819a660 2024-04-23T18:50:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67cf01640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe678112390 con 0x7fe6781072a0 2024-04-23T18:50:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:50:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58820/0 (socket says 172.21.15.33:58820) 2024-04-23T18:51:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 -- 172.21.15.33:0/2186026145 learned_addr learned my addr 172.21.15.33:0/2186026145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6781072a0 0x7fe6781a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe676d9d640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe67819a660 0x7fe67819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 -- 172.21.15.33:0/2186026145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6781072a0 msgr2=0x7fe6781a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6781072a0 0x7fe6781a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 -- 172.21.15.33:0/2186026145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe67819a660 msgr2=0x7fe67819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe67819a660 0x7fe67819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 -- 172.21.15.33:0/2186026145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67819b260 con 0x7fe6781021d0 2024-04-23T18:51:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.110+0000 7fe67659c640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9f0 tx=0x7fe66800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.111+0000 7fe66ffff640 1 -- 172.21.15.33:0/2186026145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66800cd80 con 0x7fe6781021d0 2024-04-23T18:51:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.111+0000 7fe66ffff640 1 -- 172.21.15.33:0/2186026145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6680108e0 con 0x7fe6781021d0 2024-04-23T18:51:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.111+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6781a7340 con 0x7fe6781021d0 2024-04-23T18:51:00.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.111+0000 7fe66ffff640 1 -- 172.21.15.33:0/2186026145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668018980 con 0x7fe6781021d0 2024-04-23T18:51:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.111+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6781a7740 con 0x7fe6781021d0 2024-04-23T18:51:00.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.113+0000 7fe66ffff640 1 -- 172.21.15.33:0/2186026145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe668010430 con 0x7fe6781021d0 2024-04-23T18:51:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.113+0000 7fe67fc18740 1 Processor -- start 2024-04-23T18:51:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 -- start start 2024-04-23T18:51:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f77a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:00.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f84bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556527edc970 con 0x556527f77f40 2024-04-23T18:51:00.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556527e794a0 con 0x556527f82800 2024-04-23T18:51:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67fc18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556527ed8930 con 0x556527f77620 2024-04-23T18:51:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe66dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f84bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:01.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58830/0 (socket says 172.21.15.33:58830) 2024-04-23T18:51:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 -- 172.21.15.33:0/1494637759 learned_addr learned my addr 172.21.15.33:0/1494637759 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:01.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe66ffff640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:01.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe676d9d640 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f77a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:01.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 -- 172.21.15.33:0/1494637759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 msgr2=0x556527f77a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f77a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:01.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 -- 172.21.15.33:0/1494637759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 msgr2=0x556527f84bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f84bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe676d9d640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.114+0000 7fe67659c640 1 -- 172.21.15.33:0/1494637759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556527bef210 con 0x556527f77f40 2024-04-23T18:51:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe66ffff640 1 -- 172.21.15.33:0/2186026145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fe6680d0b30 con 0x7fe6781021d0 2024-04-23T18:51:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe67659c640 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe6680ce180 tx=0x7fe66809c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a55de4d0d950c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe676d9d640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe67002a8c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:01.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe66cd99640 1 -- 172.21.15.33:0/1494637759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6680bd0d0 con 0x556527f77f40 2024-04-23T18:51:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe66cd99640 1 -- 172.21.15.33:0/1494637759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6680a56d0 con 0x556527f77f40 2024-04-23T18:51:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.115+0000 7fe66cd99640 1 -- 172.21.15.33:0/1494637759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe668099180 con 0x556527f77f40 2024-04-23T18:51:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- 172.21.15.33:0/1494637759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 msgr2=0x556527f822c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:01.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe6680ce180 tx=0x7fe66809c5a0 comp rx=0 tx=0).stop 2024-04-23T18:51:01.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- 172.21.15.33:0/1494637759 shutdown_connections 2024-04-23T18:51:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:02.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f822c0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1494637759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f77a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- 172.21.15.33:0/1494637759 >> 172.21.15.33:0/1494637759 conn(0x556527f73b30 msgr2=0x556527f73f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- 172.21.15.33:0/1494637759 shutdown_connections 2024-04-23T18:51:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- 172.21.15.33:0/1494637759 wait complete. 2024-04-23T18:51:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 -- start start 2024-04-23T18:51:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.116+0000 7fe67fc18740 1 Processor -- start 2024-04-23T18:51:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f820a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:02.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:02.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:02.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556527f85370 con 0x556527f82800 2024-04-23T18:51:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556527e770b0 con 0x556527f77f40 2024-04-23T18:51:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556527f854f0 con 0x556527f77620 2024-04-23T18:51:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67fc18740 10 client.0 _ll_register_callbacks cb 0x556527e64d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58836/0 (socket says 172.21.15.33:58836) 2024-04-23T18:51:02.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe66dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:02.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 -- 172.21.15.33:0/1403615767 learned_addr learned my addr 172.21.15.33:0/1403615767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:02.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe67659c640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f820a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:02.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 -- 172.21.15.33:0/1403615767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 msgr2=0x556527f820a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:02.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f820a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:03.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 -- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 msgr2=0x556527f7f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:03.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 -- 172.21.15.33:0/1403615767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556527cbb620 con 0x556527f82800 2024-04-23T18:51:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:51:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe676d9d640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe67002f460 tx=0x7fe6700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430787533a6de258 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.117+0000 7fe65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670004450 con 0x556527f82800 2024-04-23T18:51:03.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fe6700045f0 con 0x556527f82800 2024-04-23T18:51:03.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6680a56d0 con 0x556527f82800 2024-04-23T18:51:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe67005c050 con 0x556527f82800 2024-04-23T18:51:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 10 client.18012 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556527f91230 con 0x556527f82800 2024-04-23T18:51:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6680a56d0 con 0x556527f82800 2024-04-23T18:51:03.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7fe67004a050 con 0x556527f82800 2024-04-23T18:51:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 10 client.18012 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:03.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 10 client.18012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:03.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 20 client.18012 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527f927c0 0x556527f94c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 4 client.18012 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:51:03.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.118+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556527eea390 con 0x556527f927c0 2024-04-23T18:51:03.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.119+0000 7fe67659c640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527f927c0 0x556527f94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:03.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.119+0000 7fe67659c640 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527f927c0 0x556527f94c00 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a01af6a41414526 server_cookie=aae6c3e8fcf798d8 in_seq=0 out_seq=0 2024-04-23T18:51:03.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.119+0000 7fe65dffb640 10 client.18012 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe65dffb640 1 -- 172.21.15.33:0/1403615767 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556527eea390 con 0x556527f927c0 2024-04-23T18:51:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe65dffb640 10 client.18012 handle_command_reply: tid=0 2024-04-23T18:51:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 client.18012 shutdown 2024-04-23T18:51:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 msgr2=0x556527f7f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe67002f460 tx=0x7fe6700095c0 comp rx=0 tx=0).stop 2024-04-23T18:51:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 shutdown_connections 2024-04-23T18:51:04.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527f927c0 0x556527f94c00 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:04.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7f8d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe65dffb640 0 client.18012 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 --2- 172.21.15.33:0/1403615767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f820a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 >> 172.21.15.33:0/1403615767 conn(0x556527f73b30 msgr2=0x556527f798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:04.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 shutdown_connections 2024-04-23T18:51:04.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 1 -- 172.21.15.33:0/1403615767 wait complete. 2024-04-23T18:51:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.120+0000 7fe67fc18740 20 client.18012 trim_cache size 0 max 16384 2024-04-23T18:51:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 Processor -- start 2024-04-23T18:51:04.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 -- start start 2024-04-23T18:51:04.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:04.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781b1900 con 0x556527f82800 2024-04-23T18:51:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781af6f0 con 0x556527f77f40 2024-04-23T18:51:04.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67fc18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781b1230 con 0x556527f77620 2024-04-23T18:51:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58840/0 (socket says 172.21.15.33:58840) 2024-04-23T18:51:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 -- 172.21.15.33:0/3425768402 learned_addr learned my addr 172.21.15.33:0/3425768402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:05.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:05.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67659c640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 -- 172.21.15.33:0/3425768402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 msgr2=0x556527f8e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 -- 172.21.15.33:0/3425768402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 msgr2=0x556527f7aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe676d9d640 1 -- 172.21.15.33:0/3425768402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556527f74d60 con 0x556527f82800 2024-04-23T18:51:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.135+0000 7fe67659c640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:05.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:05.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe676d9d640 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe670009ca0 tx=0x7fe670004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb58ee0b49ce99 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe65ffff640 1 -- 172.21.15.33:0/3425768402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe670065070 con 0x556527f82800 2024-04-23T18:51:05.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe65ffff640 1 -- 172.21.15.33:0/3425768402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe670036050 con 0x556527f82800 2024-04-23T18:51:05.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe65ffff640 1 -- 172.21.15.33:0/3425768402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67002f660 con 0x556527f82800 2024-04-23T18:51:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 -- 172.21.15.33:0/3425768402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 msgr2=0x556527f7e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe670009ca0 tx=0x7fe670004a80 comp rx=0 tx=0).stop 2024-04-23T18:51:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 -- 172.21.15.33:0/3425768402 shutdown_connections 2024-04-23T18:51:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f82800 0x556527f7e240 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f77f40 0x556527f7aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:05.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3425768402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:05.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 -- 172.21.15.33:0/3425768402 >> 172.21.15.33:0/3425768402 conn(0x556527f73b30 msgr2=0x556527f7db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:05.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.136+0000 7fe67fc18740 1 -- 172.21.15.33:0/3425768402 shutdown_connections 2024-04-23T18:51:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 -- 172.21.15.33:0/3425768402 wait complete. 2024-04-23T18:51:06.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 -- start start 2024-04-23T18:51:06.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 Processor -- start 2024-04-23T18:51:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:06.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f86240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781b0490 con 0x556527f77f40 2024-04-23T18:51:06.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781afdc0 con 0x556527f82800 2024-04-23T18:51:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6781b0b60 con 0x556527f77620 2024-04-23T18:51:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67fc18740 10 client.0 _ll_register_callbacks cb 0x7fe670031810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:06.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe66dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe66dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58856/0 (socket says 172.21.15.33:58856) 2024-04-23T18:51:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe66dd9b640 1 -- 172.21.15.33:0/3441816907 learned_addr learned my addr 172.21.15.33:0/3441816907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe676d9d640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:06.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.137+0000 7fe67659c640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:06.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 -- 172.21.15.33:0/3441816907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 msgr2=0x556527f8a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 -- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 msgr2=0x556527f86240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f86240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:07.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 -- 172.21.15.33:0/3441816907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556527f86780 con 0x556527f77f40 2024-04-23T18:51:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe67659c640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe66dd9b640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe67806b630 tx=0x7fe6600039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54d0cd775db57aa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe65dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660005020 con 0x556527f77f40 2024-04-23T18:51:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7fe6600051c0 con 0x556527f77f40 2024-04-23T18:51:07.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe670036050 con 0x556527f77f40 2024-04-23T18:51:07.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe660029050 con 0x556527f77f40 2024-04-23T18:51:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe67fc18740 10 client.18018 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556527ff8aa0 con 0x556527f77f40 2024-04-23T18:51:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.138+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe670036050 con 0x556527f77f40 2024-04-23T18:51:07.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7fe660011670 con 0x556527f77f40 2024-04-23T18:51:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 10 client.18018 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 10 client.18018 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 20 client.18018 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527ff92d0 0x556527ffb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 4 client.18018 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:51:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556527f7fc60 con 0x556527ff92d0 2024-04-23T18:51:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67659c640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527ff92d0 0x556527ffb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe65dffb640 10 client.18018 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.139+0000 7fe67659c640 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527ff92d0 0x556527ffb6f0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45aecc32972ff8e server_cookie=98cc9f86e0ca9a4d in_seq=0 out_seq=0 2024-04-23T18:51:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe65dffb640 1 -- 172.21.15.33:0/3441816907 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556527f7fc60 con 0x556527ff92d0 2024-04-23T18:51:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe65dffb640 10 client.18018 handle_command_reply: tid=0 2024-04-23T18:51:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 client.18018 shutdown 2024-04-23T18:51:08.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 msgr2=0x556527f85d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fe67806b630 tx=0x7fe6600039c0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 shutdown_connections 2024-04-23T18:51:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556527ff92d0 0x556527ffb6f0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556527f82800 0x556527f86240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556527f77f40 0x556527f85d00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe65dffb640 0 client.18018 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 --2- 172.21.15.33:0/3441816907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556527f77620 0x556527f8a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 >> 172.21.15.33:0/3441816907 conn(0x556527f73b30 msgr2=0x556527f72420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 shutdown_connections 2024-04-23T18:51:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.143+0000 7fe67fc18740 1 -- 172.21.15.33:0/3441816907 wait complete. 2024-04-23T18:51:08.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67fc18740 20 client.18018 trim_cache size 0 max 16384 2024-04-23T18:51:08.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:51:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:51:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:51:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:51:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 msgr2=0x7fe6640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:08.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe670006010 tx=0x7fe67002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 msgr2=0x7fe6781a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe66800e9f0 tx=0x7fe66800eeb0 comp rx=0 tx=0).stop 2024-04-23T18:51:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 shutdown_connections 2024-04-23T18:51:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fe6640a3dc0 0x7fe6640a6280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:09.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe67819a660 0x7fe67819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6781072a0 0x7fe6781a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 --2- 172.21.15.33:0/2186026145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6781021d0 0x7fe6781a05a0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:09.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 >> 172.21.15.33:0/2186026145 conn(0x7fe6780793e0 msgr2=0x7fe6780fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 shutdown_connections 2024-04-23T18:51:09.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:35.144+0000 7fe67cf01640 1 -- 172.21.15.33:0/2186026145 wait complete. 2024-04-23T18:51:09.375 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:36 smithi033 ceph-mon[29274]: pgmap v742: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:09.446 DEBUG:teuthology.parallel:result is None 2024-04-23T18:51:09.472 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:51:09.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:38 smithi033 ceph-mon[29274]: pgmap v743: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:51:09.489 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:40 smithi033 ceph-mon[29274]: pgmap v744: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:09.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:42 smithi033 ceph-mon[29274]: pgmap v745: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:09.571 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:44 smithi033 ceph-mon[29274]: pgmap v746: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:09.579 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:46 smithi033 ceph-mon[29274]: pgmap v747: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:09.587 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:50:59 smithi122 ceph-mon[32546]: pgmap v788: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:09.643 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:01 smithi122 ceph-mon[32546]: pgmap v789: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:09.679 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:01 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:51:09.754 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:01 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:51:09.934 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:01 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:51:09.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:01 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:51:09.991 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:02 smithi122 ceph-mon[32546]: pgmap v790: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 KiB/s wr, 33 op/s 2024-04-23T18:51:09.997 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:04 smithi122 ceph-mon[32546]: pgmap v791: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.083 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:06 smithi122 ceph-mon[32546]: pgmap v792: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.118 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:08 smithi122 ceph-mon[32546]: pgmap v793: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.177 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:50:59 smithi106 ceph-mon[32716]: pgmap v788: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:10.189 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:01 smithi106 ceph-mon[32716]: pgmap v789: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:10.197 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:01 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:51:10.304 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:01 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:51:10.368 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:01 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:51:10.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:01 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:51:10.469 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:02 smithi106 ceph-mon[32716]: pgmap v790: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 KiB/s wr, 33 op/s 2024-04-23T18:51:10.599 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:04 smithi106 ceph-mon[32716]: pgmap v791: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.675 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:06 smithi106 ceph-mon[32716]: pgmap v792: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.732 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:08 smithi106 ceph-mon[32716]: pgmap v793: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:10 smithi122 ceph-mon[32546]: pgmap v794: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:10.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:10 smithi122 ceph-mon[32546]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:51:11.105 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:10 smithi106 ceph-mon[32716]: pgmap v794: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:51:11.114 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:10 smithi106 ceph-mon[32716]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:51:11.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 -- 172.21.15.33:0/2742911653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74301010c0 msgr2=0x7f74301014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:11.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 --2- 172.21.15.33:0/2742911653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74301010c0 0x7f74301014a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c02a410 comp rx=0 tx=0).stop 2024-04-23T18:51:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 -- 172.21.15.33:0/2742911653 shutdown_connections 2024-04-23T18:51:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 --2- 172.21.15.33:0/2742911653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430106190 0x7f743010dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 --2- 172.21.15.33:0/2742911653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430101a70 0x7f7430105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 --2- 172.21.15.33:0/2742911653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74301010c0 0x7f74301014a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 -- 172.21.15.33:0/2742911653 >> 172.21.15.33:0/2742911653 conn(0x7f7430077fd0 msgr2=0x7f74300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 -- 172.21.15.33:0/2742911653 shutdown_connections 2024-04-23T18:51:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 -- 172.21.15.33:0/2742911653 wait complete. 2024-04-23T18:51:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.244+0000 7f74387e8640 1 Processor -- start 2024-04-23T18:51:11.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 -- start start 2024-04-23T18:51:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 0x7f7430199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:11.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430199ff0 0x7f74301a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74301110f0 con 0x7f7430106190 2024-04-23T18:51:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7430110f70 con 0x7f7430199ff0 2024-04-23T18:51:11.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f74387e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7430111270 con 0x7f74301010c0 2024-04-23T18:51:12.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:12.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44914/0 (socket says 172.21.15.33:44914) 2024-04-23T18:51:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 -- 172.21.15.33:0/3879140447 learned_addr learned my addr 172.21.15.33:0/3879140447 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7436d85640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430199ff0 0x7f74301a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7436584640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 0x7f7430199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 -- 172.21.15.33:0/3879140447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 msgr2=0x7f7430199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 0x7f7430199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:12.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 -- 172.21.15.33:0/3879140447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430199ff0 msgr2=0x7f74301a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430199ff0 0x7f74301a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 -- 172.21.15.33:0/3879140447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74301a4d70 con 0x7f7430106190 2024-04-23T18:51:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7436584640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 0x7f7430199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.245+0000 7f7435d83640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f742000d740 tx=0x7f742000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:12.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.246+0000 7f74277fe640 1 -- 172.21.15.33:0/3879140447 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420017070 con 0x7f7430106190 2024-04-23T18:51:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.246+0000 7f74277fe640 1 -- 172.21.15.33:0/3879140447 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742000de50 con 0x7f7430106190 2024-04-23T18:51:12.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.246+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74301a5060 con 0x7f7430106190 2024-04-23T18:51:12.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.246+0000 7f74277fe640 1 -- 172.21.15.33:0/3879140447 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420004ce0 con 0x7f7430106190 2024-04-23T18:51:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.246+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74301a55b0 con 0x7f7430106190 2024-04-23T18:51:12.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f74277fe640 1 -- 172.21.15.33:0/3879140447 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7420004e80 con 0x7f7430106190 2024-04-23T18:51:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 Processor -- start 2024-04-23T18:51:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 -- start start 2024-04-23T18:51:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb25a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb25cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:13.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557adb1b4970 con 0x557adb24f620 2024-04-23T18:51:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557adb1514a0 con 0x557adb25a800 2024-04-23T18:51:13.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.249+0000 7f743b5cf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557adb1b0930 con 0x557adb24ff40 2024-04-23T18:51:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436584640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb25a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44922/0 (socket says 172.21.15.33:44922) 2024-04-23T18:51:13.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 -- 172.21.15.33:0/3893267111 learned_addr learned my addr 172.21.15.33:0/3893267111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f74277fe640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 0x7f741c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:13.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7435d83640 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb25cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:13.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 -- 172.21.15.33:0/3893267111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 msgr2=0x557adb25a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb25a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:13.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 -- 172.21.15.33:0/3893267111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 msgr2=0x557adb25cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:13.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb25cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 -- 172.21.15.33:0/3893267111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adaec7210 con 0x557adb24f620 2024-04-23T18:51:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7435d83640 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb25cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 0x7f741c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f74277fe640 1 -- 172.21.15.33:0/3879140447 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f74200d01f0 con 0x7f7430106190 2024-04-23T18:51:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7436d85640 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7428009fc0 tx=0x7f7428010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d07411a7b6062 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:13.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.250+0000 7f7424ff9640 1 -- 172.21.15.33:0/3893267111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742801c900 con 0x557adb24f620 2024-04-23T18:51:13.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f7424ff9640 1 -- 172.21.15.33:0/3893267111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742801b3d0 con 0x557adb24f620 2024-04-23T18:51:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f7424ff9640 1 -- 172.21.15.33:0/3893267111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7428026680 con 0x557adb24f620 2024-04-23T18:51:13.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 -- 172.21.15.33:0/3893267111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 msgr2=0x557adb24fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f7436d85640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 0x7f741c0a6300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7428020000 tx=0x7f742801d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7428009fc0 tx=0x7f7428010800 comp rx=0 tx=0).stop 2024-04-23T18:51:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 -- 172.21.15.33:0/3893267111 shutdown_connections 2024-04-23T18:51:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb25cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb25a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:14.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3893267111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb24fa00 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:14.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 -- 172.21.15.33:0/3893267111 >> 172.21.15.33:0/3893267111 conn(0x557adb24bb30 msgr2=0x557adb24bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:14.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 -- 172.21.15.33:0/3893267111 shutdown_connections 2024-04-23T18:51:14.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.251+0000 7f743b5cf740 1 -- 172.21.15.33:0/3893267111 wait complete. 2024-04-23T18:51:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 -- start start 2024-04-23T18:51:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 Processor -- start 2024-04-23T18:51:14.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb257460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb2579c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557adb25d430 con 0x557adb24f620 2024-04-23T18:51:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557adb14f0b0 con 0x557adb25a800 2024-04-23T18:51:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557adb25d5b0 con 0x557adb24ff40 2024-04-23T18:51:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f743b5cf740 10 client.0 _ll_register_callbacks cb 0x557adb13cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44930/0 (socket says 172.21.15.33:44930) 2024-04-23T18:51:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 -- 172.21.15.33:0/1093593027 learned_addr learned my addr 172.21.15.33:0/1093593027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:15.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7435d83640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb257460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:15.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436d85640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb2579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 -- 172.21.15.33:0/1093593027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 msgr2=0x557adb257460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb257460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 -- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 msgr2=0x557adb2579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb2579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:15.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7436584640 1 -- 172.21.15.33:0/1093593027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adaf93620 con 0x557adb24f620 2024-04-23T18:51:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.252+0000 7f7435d83640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb257460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7436584640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c031730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9288a84a4b2e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c035040 con 0x557adb24f620 2024-04-23T18:51:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f742c004240 con 0x557adb24f620 2024-04-23T18:51:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f742801b3d0 con 0x557adb24f620 2024-04-23T18:51:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f742c004440 con 0x557adb24f620 2024-04-23T18:51:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 10 client.18030 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557adb2693b0 con 0x557adb24f620 2024-04-23T18:51:15.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742801b3d0 con 0x557adb24f620 2024-04-23T18:51:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f742c004b50 con 0x557adb24f620 2024-04-23T18:51:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 10 client.18030 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:15.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 10 client.18030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.253+0000 7f743b5cf740 20 client.18030 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:16.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb26a870 0x557adb26cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f743b5cf740 4 client.18030 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:51:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557adb1c2390 con 0x557adb26a870 2024-04-23T18:51:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f7435d83640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb26a870 0x557adb26cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f7435d83640 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb26a870 0x557adb26cc80 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3078ecd164245e14 server_cookie=701a84edf9b1f208 in_seq=0 out_seq=0 2024-04-23T18:51:16.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.254+0000 7f7419ffb640 10 client.18030 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:16.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f7419ffb640 1 -- 172.21.15.33:0/1093593027 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557adb1c2390 con 0x557adb26a870 2024-04-23T18:51:16.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f7419ffb640 10 client.18030 handle_command_reply: tid=0 2024-04-23T18:51:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 client.18030 shutdown 2024-04-23T18:51:16.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 msgr2=0x557adb25a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f742c009900 tx=0x7f742c031730 comp rx=0 tx=0).stop 2024-04-23T18:51:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 shutdown_connections 2024-04-23T18:51:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb26a870 0x557adb26cc80 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb2579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24ff40 0x557adb257460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:16.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 --2- 172.21.15.33:0/1093593027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb25a170 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:16.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f7419ffb640 0 client.18030 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:16.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 >> 172.21.15.33:0/1093593027 conn(0x557adb24bb30 msgr2=0x557adb251860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 shutdown_connections 2024-04-23T18:51:17.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.255+0000 7f743b5cf740 1 -- 172.21.15.33:0/1093593027 wait complete. 2024-04-23T18:51:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.256+0000 7f743b5cf740 20 client.18030 trim_cache size 0 max 16384 2024-04-23T18:51:17.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 Processor -- start 2024-04-23T18:51:17.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 -- start start 2024-04-23T18:51:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24f620 0x557adb2662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb262be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:17.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74301af5c0 con 0x557adb24ff40 2024-04-23T18:51:17.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ad3b0 con 0x557adb25a800 2024-04-23T18:51:17.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f743b5cf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74301aeef0 con 0x557adb24f620 2024-04-23T18:51:17.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:17.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44946/0 (socket says 172.21.15.33:44946) 2024-04-23T18:51:17.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 -- 172.21.15.33:0/2683840679 learned_addr learned my addr 172.21.15.33:0/2683840679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7436584640 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24f620 0x557adb2662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7436d85640 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb262be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:17.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 -- 172.21.15.33:0/2683840679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24f620 msgr2=0x557adb2662f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24f620 0x557adb2662f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 -- 172.21.15.33:0/2683840679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 msgr2=0x557adb262be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb262be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.271+0000 7f7435d83640 1 -- 172.21.15.33:0/2683840679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adb2505b0 con 0x557adb24ff40 2024-04-23T18:51:17.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f7435d83640 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7420098b50 tx=0x7f74200d2700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b051bf3ed06e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:17.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f741bfff640 1 -- 172.21.15.33:0/2683840679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420020070 con 0x557adb24ff40 2024-04-23T18:51:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f741bfff640 1 -- 172.21.15.33:0/2683840679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200a4df0 con 0x557adb24ff40 2024-04-23T18:51:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f741bfff640 1 -- 172.21.15.33:0/2683840679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200cdd40 con 0x557adb24ff40 2024-04-23T18:51:17.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 -- 172.21.15.33:0/2683840679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 msgr2=0x557adb257360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7420098b50 tx=0x7f74200d2700 comp rx=0 tx=0).stop 2024-04-23T18:51:18.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 -- 172.21.15.33:0/2683840679 shutdown_connections 2024-04-23T18:51:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb25a800 0x557adb262be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24ff40 0x557adb257360 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 --2- 172.21.15.33:0/2683840679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb24f620 0x557adb2662f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 -- 172.21.15.33:0/2683840679 >> 172.21.15.33:0/2683840679 conn(0x557adb24bb30 msgr2=0x557adb251860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 -- 172.21.15.33:0/2683840679 shutdown_connections 2024-04-23T18:51:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.272+0000 7f743b5cf740 1 -- 172.21.15.33:0/2683840679 wait complete. 2024-04-23T18:51:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 -- start start 2024-04-23T18:51:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 Processor -- start 2024-04-23T18:51:18.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:18.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 0x557adb2c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:18.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb25a800 0x557adb2cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:18.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ae150 con 0x557adb24f620 2024-04-23T18:51:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ada80 con 0x557adb24ff40 2024-04-23T18:51:18.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74301ae820 con 0x557adb25a800 2024-04-23T18:51:18.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f743b5cf740 10 client.0 _ll_register_callbacks cb 0x7f742c005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44954/0 (socket says 172.21.15.33:44954) 2024-04-23T18:51:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 -- 172.21.15.33:0/3670298197 learned_addr learned my addr 172.21.15.33:0/3670298197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436d85640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb25a800 0x557adb2cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7435d83640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 0x557adb2c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:19.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 -- 172.21.15.33:0/3670298197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb25a800 msgr2=0x557adb2cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:19.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb25a800 0x557adb2cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 -- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 msgr2=0x557adb2c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:19.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 0x557adb2c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7436584640 1 -- 172.21.15.33:0/3670298197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adb2ce4c0 con 0x557adb24f620 2024-04-23T18:51:19.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.273+0000 7f7435d83640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 0x557adb2c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:19.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7436584640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f742c003690 tx=0x7f742c0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122c49c8ea8f8365 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c035050 con 0x557adb24f620 2024-04-23T18:51:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f742c004fc0 con 0x557adb24f620 2024-04-23T18:51:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74200a4df0 con 0x557adb24f620 2024-04-23T18:51:19.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f742c051040 con 0x557adb24f620 2024-04-23T18:51:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 10 client.18036 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557adb2d5880 con 0x557adb24f620 2024-04-23T18:51:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74200a4df0 con 0x557adb24f620 2024-04-23T18:51:19.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f742c003f30 con 0x557adb24f620 2024-04-23T18:51:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 10 client.18036 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 20 client.18036 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb2d5f40 0x557adb2d8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 4 client.18036 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:51:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.274+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557adb1bc200 con 0x557adb2d5f40 2024-04-23T18:51:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.275+0000 7f7435d83640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb2d5f40 0x557adb2d8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.275+0000 7f7419ffb640 10 client.18036 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.275+0000 7f7435d83640 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb2d5f40 0x557adb2d8350 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd772c50003f69 server_cookie=246c86dc10623841 in_seq=0 out_seq=0 2024-04-23T18:51:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f7419ffb640 1 -- 172.21.15.33:0/3670298197 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557adb1bc200 con 0x557adb2d5f40 2024-04-23T18:51:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f7419ffb640 10 client.18036 handle_command_reply: tid=0 2024-04-23T18:51:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 client.18036 shutdown 2024-04-23T18:51:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 msgr2=0x557adb2c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:20.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f742c003690 tx=0x7f742c0048d0 comp rx=0 tx=0).stop 2024-04-23T18:51:20.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 shutdown_connections 2024-04-23T18:51:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557adb2d5f40 0x557adb2d8350 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557adb25a800 0x557adb2cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f7419ffb640 0 client.18036 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557adb24ff40 0x557adb2c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 --2- 172.21.15.33:0/3670298197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557adb24f620 0x557adb2c8fb0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:20.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 >> 172.21.15.33:0/3670298197 conn(0x557adb24bb30 msgr2=0x557adb2521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 shutdown_connections 2024-04-23T18:51:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.280+0000 7f743b5cf740 1 -- 172.21.15.33:0/3670298197 wait complete. 2024-04-23T18:51:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f743b5cf740 20 client.18036 trim_cache size 0 max 16384 2024-04-23T18:51:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:51:21.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:51:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:51:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:51:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 msgr2=0x7f741c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 0x7f741c0a6300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7428020000 tx=0x7f742801d000 comp rx=0 tx=0).stop 2024-04-23T18:51:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 msgr2=0x7f7430199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f742000d740 tx=0x7f742000dc00 comp rx=0 tx=0).stop 2024-04-23T18:51:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 shutdown_connections 2024-04-23T18:51:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f741c0a3e40 0x7f741c0a6300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7430199ff0 0x7f74301a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7430106190 0x7f7430199ab0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:21.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 --2- 172.21.15.33:0/3879140447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74301010c0 0x7f7430199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:21.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 >> 172.21.15.33:0/3879140447 conn(0x7f7430077fd0 msgr2=0x7f74300fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 shutdown_connections 2024-04-23T18:51:21.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:49:48.281+0000 7f74387e8640 1 -- 172.21.15.33:0/3879140447 wait complete. 2024-04-23T18:51:22.040 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:48 smithi033 ceph-mon[29274]: pgmap v748: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:22.049 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:50 smithi033 ceph-mon[29274]: pgmap v749: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:22.090 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:52 smithi033 ceph-mon[29274]: pgmap v750: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:22.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:54 smithi033 ceph-mon[29274]: pgmap v751: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:22.110 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:56 smithi033 ceph-mon[29274]: pgmap v752: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:22.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:49:58 smithi033 ceph-mon[29274]: pgmap v753: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:22.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: pgmap v754: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:22.283 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:51:22.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-23T18:51:22.482 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-23T18:51:22.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: daemon mds.f on smithi122 is in error state 2024-04-23T18:51:22.599 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:51:22.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:51:22.709 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:00 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:51:22.718 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:12 smithi106 ceph-mon[32716]: pgmap v796: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:22.727 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:14 smithi106 ceph-mon[32716]: pgmap v797: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:22.789 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:16 smithi106 ceph-mon[32716]: pgmap v798: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:22.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:16 smithi106 ceph-mon[32716]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:51:22.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:18 smithi106 ceph-mon[32716]: pgmap v800: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T18:51:22.850 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:20 smithi106 ceph-mon[32716]: pgmap v801: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 950 B/s wr, 11 op/s 2024-04-23T18:51:22.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:12 smithi122 ceph-mon[32546]: pgmap v796: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:22.950 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:14 smithi122 ceph-mon[32546]: pgmap v797: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:23.064 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:16 smithi122 ceph-mon[32546]: pgmap v798: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:51:23.109 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:16 smithi122 ceph-mon[32546]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:51:23.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:18 smithi122 ceph-mon[32546]: pgmap v800: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T18:51:23.254 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:20 smithi122 ceph-mon[32546]: pgmap v801: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 950 B/s wr, 11 op/s 2024-04-23T18:51:23.299 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:22 smithi106 ceph-mon[32716]: pgmap v802: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:23.398 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:22 smithi122 ceph-mon[32546]: pgmap v802: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:23.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- 172.21.15.33:0/307485479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a41006c0 msgr2=0x7f40a4100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 --2- 172.21.15.33:0/307485479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a41006c0 0x7f40a4100aa0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a002a410 comp rx=0 tx=0).stop 2024-04-23T18:51:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- 172.21.15.33:0/307485479 shutdown_connections 2024-04-23T18:51:23.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 --2- 172.21.15.33:0/307485479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a4107280 0x7f40a410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:23.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 --2- 172.21.15.33:0/307485479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a4101070 0x7f40a4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 --2- 172.21.15.33:0/307485479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a41006c0 0x7f40a4100aa0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- 172.21.15.33:0/307485479 >> 172.21.15.33:0/307485479 conn(0x7f40a4077390 msgr2=0x7f40a40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- 172.21.15.33:0/307485479 shutdown_connections 2024-04-23T18:51:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- 172.21.15.33:0/307485479 wait complete. 2024-04-23T18:51:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 Processor -- start 2024-04-23T18:51:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.382+0000 7f40aba9a640 1 -- start start 2024-04-23T18:51:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a41006c0 0x7f40a4101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:23.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 0x7f40a41a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:24.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41121e0 con 0x7f40a4107280 2024-04-23T18:51:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4112060 con 0x7f40a41006c0 2024-04-23T18:51:24.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aba9a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4112360 con 0x7f40a41020b0 2024-04-23T18:51:24.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:24.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39182/0 (socket says 172.21.15.33:39182) 2024-04-23T18:51:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 -- 172.21.15.33:0/4283336845 learned_addr learned my addr 172.21.15.33:0/4283336845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:24.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9836640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a41006c0 0x7f40a4101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aa037640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 0x7f40a41a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 -- 172.21.15.33:0/4283336845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 msgr2=0x7f40a41a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:24.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 0x7f40a41a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 -- 172.21.15.33:0/4283336845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a41006c0 msgr2=0x7f40a4101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a41006c0 0x7f40a4101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:24.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 -- 172.21.15.33:0/4283336845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41a6ff0 con 0x7f40a4107280 2024-04-23T18:51:24.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40aa037640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 0x7f40a41a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:24.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f40a9035640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f409400e9f0 tx=0x7f409400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.383+0000 7f409affd640 1 -- 172.21.15.33:0/4283336845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409400ccc0 con 0x7f40a4107280 2024-04-23T18:51:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.384+0000 7f409affd640 1 -- 172.21.15.33:0/4283336845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40940054d0 con 0x7f40a4107280 2024-04-23T18:51:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.384+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41a72e0 con 0x7f40a4107280 2024-04-23T18:51:25.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.384+0000 7f409affd640 1 -- 172.21.15.33:0/4283336845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094010430 con 0x7f40a4107280 2024-04-23T18:51:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.384+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a77e0 con 0x7f40a4107280 2024-04-23T18:51:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.386+0000 7f409affd640 1 -- 172.21.15.33:0/4283336845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40940046c0 con 0x7f40a4107280 2024-04-23T18:51:25.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 Processor -- start 2024-04-23T18:51:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 -- start start 2024-04-23T18:51:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:25.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16f40 0x56065fc212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:25.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc23bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:25.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56065fb7b970 con 0x56065fc21800 2024-04-23T18:51:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56065fb184a0 con 0x56065fc16620 2024-04-23T18:51:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56065fb77930 con 0x56065fc16f40 2024-04-23T18:51:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.387+0000 7f409affd640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 0x7f40900a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:25.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc23bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40a9836640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16f40 0x56065fc212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:38716/0 (socket says 172.21.15.33:38716) 2024-04-23T18:51:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 -- 172.21.15.33:0/1142024681 learned_addr learned my addr 172.21.15.33:0/1142024681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc23bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-04-23T18:51:25.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 0x7f40900a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f409affd640 1 -- 172.21.15.33:0/4283336845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f40940d0d00 con 0x7f40a4107280 2024-04-23T18:51:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 -- 172.21.15.33:0/1142024681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16f40 msgr2=0x56065fc212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16f40 0x56065fc212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 -- 172.21.15.33:0/1142024681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 msgr2=0x56065fc23bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:26.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc23bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:26.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 -- 172.21.15.33:0/1142024681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065f88e210 con 0x56065fc16620 2024-04-23T18:51:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 0x7f40900a62b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f409c010c40 tx=0x7f409c015000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f40aa037640 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f409c013de0 tx=0x7f409c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596f7f90e8837540 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:26.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.388+0000 7f408ffff640 1 -- 172.21.15.33:0/1142024681 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c010e50 con 0x56065fc16620 2024-04-23T18:51:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f408ffff640 1 -- 172.21.15.33:0/1142024681 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c013740 con 0x56065fc16620 2024-04-23T18:51:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f408ffff640 1 -- 172.21.15.33:0/1142024681 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c005310 con 0x56065fc16620 2024-04-23T18:51:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 -- 172.21.15.33:0/1142024681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 msgr2=0x56065fc16a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f409c013de0 tx=0x7f409c010740 comp rx=0 tx=0).stop 2024-04-23T18:51:26.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 -- 172.21.15.33:0/1142024681 shutdown_connections 2024-04-23T18:51:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc23bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16f40 0x56065fc212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/1142024681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc16a00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 -- 172.21.15.33:0/1142024681 >> 172.21.15.33:0/1142024681 conn(0x56065fc12b30 msgr2=0x56065fc12f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.389+0000 7f40ae7b1740 1 -- 172.21.15.33:0/1142024681 shutdown_connections 2024-04-23T18:51:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 -- 172.21.15.33:0/1142024681 wait complete. 2024-04-23T18:51:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 -- start start 2024-04-23T18:51:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 Processor -- start 2024-04-23T18:51:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56065fc243d0 con 0x56065fc21800 2024-04-23T18:51:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56065fb160b0 con 0x56065fc16f40 2024-04-23T18:51:27.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.390+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56065fc24550 con 0x56065fc16620 2024-04-23T18:51:27.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40ae7b1740 10 client.0 _ll_register_callbacks cb 0x56065fb03d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:27.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40aa037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40aa037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39190/0 (socket says 172.21.15.33:39190) 2024-04-23T18:51:27.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40aa037640 1 -- 172.21.15.33:0/3908670611 learned_addr learned my addr 172.21.15.33:0/3908670611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:38728/0 (socket says 172.21.15.33:38728) 2024-04-23T18:51:27.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9836640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 -- 172.21.15.33:0/3908670611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 msgr2=0x56065fc21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:27.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 -- 172.21.15.33:0/3908670611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 msgr2=0x56065fc1ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:27.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 -- 172.21.15.33:0/3908670611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065f95a620 con 0x56065fc16f40 2024-04-23T18:51:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9836640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40aa037640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:51:27.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40a9035640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40940ce3e0 tx=0x7f409400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d776a0bb7c8dd633 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:28.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f408d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:51:28.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094087040 con 0x56065fc16f40 2024-04-23T18:51:28.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f40940bd0b0 con 0x56065fc16f40 2024-04-23T18:51:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409c013740 con 0x56065fc16f40 2024-04-23T18:51:28.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.391+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f40940bd2b0 con 0x56065fc16f40 2024-04-23T18:51:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 10 client.27263 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:28.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56065fc30380 con 0x56065fc16f40 2024-04-23T18:51:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c013740 con 0x56065fc16f40 2024-04-23T18:51:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f40940a5410 con 0x56065fc16f40 2024-04-23T18:51:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 10 client.27263 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 10 client.27263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 20 client.27263 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:28.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc31840 0x56065fc33c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 4 client.27263 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:51:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56065fb89390 con 0x56065fc31840 2024-04-23T18:51:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.392+0000 7f40a9836640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc31840 0x56065fc33c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40a9836640 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc31840 0x56065fc33c30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982e58b3b4cc27ad server_cookie=ee1b76cbe1be8158 in_seq=0 out_seq=0 2024-04-23T18:51:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f408d7fa640 10 client.27263 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f408d7fa640 1 -- 172.21.15.33:0/3908670611 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56065fb89390 con 0x56065fc31840 2024-04-23T18:51:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f408d7fa640 10 client.27263 handle_command_reply: tid=0 2024-04-23T18:51:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 client.27263 shutdown 2024-04-23T18:51:29.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 msgr2=0x56065fc1e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f40940ce3e0 tx=0x7f409400ce60 comp rx=0 tx=0).stop 2024-04-23T18:51:29.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 shutdown_connections 2024-04-23T18:51:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc31840 0x56065fc33c30 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc1ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:29.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc1e480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f408d7fa640 0 client.27263 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:29.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.393+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/3908670611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:29.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.394+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 >> 172.21.15.33:0/3908670611 conn(0x56065fc12b30 msgr2=0x56065fc188b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.394+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 shutdown_connections 2024-04-23T18:51:29.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.394+0000 7f40ae7b1740 1 -- 172.21.15.33:0/3908670611 wait complete. 2024-04-23T18:51:29.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.394+0000 7f40ae7b1740 20 client.27263 trim_cache size 0 max 16384 2024-04-23T18:51:29.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 Processor -- start 2024-04-23T18:51:29.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 -- start start 2024-04-23T18:51:29.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:29.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc19970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b1880 con 0x56065fc21800 2024-04-23T18:51:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41af670 con 0x56065fc16f40 2024-04-23T18:51:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b11b0 con 0x56065fc16620 2024-04-23T18:51:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:30.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39200/0 (socket says 172.21.15.33:39200) 2024-04-23T18:51:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 -- 172.21.15.33:0/2243760831 learned_addr learned my addr 172.21.15.33:0/2243760831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40a9035640 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc19970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40a9836640 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 -- 172.21.15.33:0/2243760831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 msgr2=0x56065fc2d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:30.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc2d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:30.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 -- 172.21.15.33:0/2243760831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 msgr2=0x56065fc19970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:30.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc19970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.409+0000 7f40aa037640 1 -- 172.21.15.33:0/2243760831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56065fc178d0 con 0x56065fc21800 2024-04-23T18:51:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40aa037640 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f409c0135d0 tx=0x7f409c013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e434dd2f0707e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:30.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f408f7fe640 1 -- 172.21.15.33:0/2243760831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c02e070 con 0x56065fc21800 2024-04-23T18:51:30.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f408f7fe640 1 -- 172.21.15.33:0/2243760831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c003e20 con 0x56065fc21800 2024-04-23T18:51:30.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f408f7fe640 1 -- 172.21.15.33:0/2243760831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c013050 con 0x56065fc21800 2024-04-23T18:51:30.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2243760831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 msgr2=0x56065fc20cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:30.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f409c0135d0 tx=0x7f409c013f90 comp rx=0 tx=0).stop 2024-04-23T18:51:30.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2243760831 shutdown_connections 2024-04-23T18:51:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc21800 0x56065fc20cc0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:31.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16f40 0x56065fc19970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2243760831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc16620 0x56065fc2d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2243760831 >> 172.21.15.33:0/2243760831 conn(0x56065fc12b30 msgr2=0x56065fc19260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:31.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.410+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2243760831 shutdown_connections 2024-04-23T18:51:31.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2243760831 wait complete. 2024-04-23T18:51:31.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 -- start start 2024-04-23T18:51:31.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 Processor -- start 2024-04-23T18:51:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc29960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:31.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc21800 0x56065fc1b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b0410 con 0x56065fc16f40 2024-04-23T18:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41afd40 con 0x56065fc16620 2024-04-23T18:51:31.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40a41b0ae0 con 0x56065fc21800 2024-04-23T18:51:31.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40ae7b1740 10 client.0 _ll_register_callbacks cb 0x7f409409c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:31.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:31.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.411+0000 7f40a9035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39216/0 (socket says 172.21.15.33:39216) 2024-04-23T18:51:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 -- 172.21.15.33:0/2844754318 learned_addr learned my addr 172.21.15.33:0/2844754318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:31.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40aa037640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc21800 0x56065fc1b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:31.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9836640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc29960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:31.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 -- 172.21.15.33:0/2844754318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc21800 msgr2=0x56065fc1b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:31.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc21800 0x56065fc1b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 -- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 msgr2=0x56065fc29960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:32.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc29960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 -- 172.21.15.33:0/2844754318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56065fc1b7b0 con 0x56065fc16f40 2024-04-23T18:51:32.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40a9035640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f409400c6a0 tx=0x7f4094016fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd2028d8c03e716 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f408d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094066060 con 0x56065fc16f40 2024-04-23T18:51:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 8542+0+0 (secure 0 0 0) 0x7f4094061bb0 con 0x56065fc16f40 2024-04-23T18:51:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f409c003e20 con 0x56065fc16f40 2024-04-23T18:51:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4094087030 con 0x56065fc16f40 2024-04-23T18:51:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40ae7b1740 10 client.18054 fetch_fsmap learned FSMap version 43 2024-04-23T18:51:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.412+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56065fc97b20 con 0x56065fc16f40 2024-04-23T18:51:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409c003e20 con 0x56065fc16f40 2024-04-23T18:51:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) ==== 7749+0+0 (secure 0 0 0) 0x7f4094005000 con 0x56065fc16f40 2024-04-23T18:51:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 10 client.18054 fetch_fsmap finished waiting for FSMap version 43 2024-04-23T18:51:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:32.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 20 client.18054 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc980f0 0x56065fc9a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 4 client.18054 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:51:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56065fb83200 con 0x56065fc980f0 2024-04-23T18:51:32.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40a9836640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc980f0 0x56065fc9a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f408d7fa640 10 client.18054 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:01.413+0000 7f40a9836640 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc980f0 0x56065fc9a510 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9fa4287527e7f3 server_cookie=fcc93f788dab1969 in_seq=0 out_seq=0 2024-04-23T18:51:32.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:01 smithi033 ceph-mon[29274]: pgmap v755: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 284 B/s wr, 9 op/s 2024-04-23T18:51:32.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:01 smithi033 ceph-mon[29274]: pgmap v756: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T18:51:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f408d7fa640 1 -- 172.21.15.33:0/2844754318 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56065fb83200 con 0x56065fc980f0 2024-04-23T18:51:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f408d7fa640 10 client.18054 handle_command_reply: tid=0 2024-04-23T18:51:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 client.18054 shutdown 2024-04-23T18:51:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 msgr2=0x56065fc1acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f409400c6a0 tx=0x7f4094016fe0 comp rx=0 tx=0).stop 2024-04-23T18:51:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 shutdown_connections 2024-04-23T18:51:33.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x56065fc980f0 0x56065fc9a510 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:33.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56065fc21800 0x56065fc1b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56065fc16f40 0x56065fc1acb0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:33.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f408d7fa640 0 client.18054 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 --2- 172.21.15.33:0/2844754318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56065fc16620 0x56065fc29960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:33.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 >> 172.21.15.33:0/2844754318 conn(0x56065fc12b30 msgr2=0x56065fc188b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 shutdown_connections 2024-04-23T18:51:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 1 -- 172.21.15.33:0/2844754318 wait complete. 2024-04-23T18:51:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.194+0000 7f40ae7b1740 20 client.18054 trim_cache size 0 max 16384 2024-04-23T18:51:33.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:51:33.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:51:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:51:33.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:51:33.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 msgr2=0x7f40900a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:34.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 0x7f40900a62b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f409c010c40 tx=0x7f409c015000 comp rx=0 tx=0).stop 2024-04-23T18:51:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 msgr2=0x7f40a4101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f409400e9f0 tx=0x7f409400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:51:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 shutdown_connections 2024-04-23T18:51:34.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f40900a3df0 0x7f40900a62b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40a41020b0 0x7f40a41a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:34.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40a4107280 0x7f40a4101b70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 --2- 172.21.15.33:0/4283336845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40a41006c0 0x7f40a4101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 >> 172.21.15.33:0/4283336845 conn(0x7f40a4077390 msgr2=0x7f40a40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:34.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 shutdown_connections 2024-04-23T18:51:34.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:03.195+0000 7f40aba9a640 1 -- 172.21.15.33:0/4283336845 wait complete. 2024-04-23T18:51:34.619 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:03 smithi033 ceph-mon[29274]: pgmap v757: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:51:34.701 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-mon[29274]: osdmap e132: 12 total, 12 up, 12 in 2024-04-23T18:51:34.710 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:24 smithi106 ceph-mon[32716]: pgmap v803: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:34.781 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:26 smithi106 ceph-mon[32716]: pgmap v804: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:34.806 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:28 smithi106 ceph-mon[32716]: pgmap v805: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 378 B/s wr, 9 op/s 2024-04-23T18:51:34.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:30 smithi106 ceph-mon[32716]: pgmap v806: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:34.931 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:32 smithi106 ceph-mon[32716]: pgmap v807: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:51:34.965 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:34 smithi106 ceph-mon[32716]: pgmap v808: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:35.013 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:24 smithi122 ceph-mon[32546]: pgmap v803: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:35.021 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:26 smithi122 ceph-mon[32546]: pgmap v804: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:51:35.029 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:28 smithi122 ceph-mon[32546]: pgmap v805: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 378 B/s wr, 9 op/s 2024-04-23T18:51:35.039 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:30 smithi122 ceph-mon[32546]: pgmap v806: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:35.048 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:32 smithi122 ceph-mon[32546]: pgmap v807: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:51:35.148 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:34 smithi122 ceph-mon[32546]: pgmap v808: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:35.193 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:51:35.280 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: in thread 7fb2cb4f1700 thread_name:md_log_replay 2024-04-23T18:51:35.374 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:51:35.416 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb2dc8b2d20] 2024-04-23T18:51:35.424 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2: /usr/bin/ceph-mds(+0x549372) [0x555fcbb26372] 2024-04-23T18:51:35.432 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555fcbb29b23] 2024-04-23T18:51:35.454 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 4: (MDLog::_replay_thread()+0x617) [0x555fcbb84647] 2024-04-23T18:51:35.511 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555fcbb8dd31] 2024-04-23T18:51:35.521 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb2dc8a81ca] 2024-04-23T18:51:35.633 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 7: clone() 2024-04-23T18:51:35.644 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:50:04.171+0000 7fb2cb4f1700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:51:35.651 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: in thread 7fb2cb4f1700 thread_name:md_log_replay 2024-04-23T18:51:35.658 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:35.714 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:51:35.721 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb2dc8b2d20] 2024-04-23T18:51:35.729 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2: /usr/bin/ceph-mds(+0x549372) [0x555fcbb26372] 2024-04-23T18:51:35.791 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555fcbb29b23] 2024-04-23T18:51:35.867 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 4: (MDLog::_replay_thread()+0x617) [0x555fcbb84647] 2024-04-23T18:51:35.890 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555fcbb8dd31] 2024-04-23T18:51:35.920 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb2dc8a81ca] 2024-04-23T18:51:35.931 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 7: clone() 2024-04-23T18:51:35.940 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:51:35.948 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:35.956 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 0> 2024-04-23T18:50:04.171+0000 7fb2cb4f1700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:51:36.013 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: in thread 7fb2cb4f1700 thread_name:md_log_replay 2024-04-23T18:51:36.097 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:36.222 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:51:36.244 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb2dc8b2d20] 2024-04-23T18:51:36.284 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2: /usr/bin/ceph-mds(+0x549372) [0x555fcbb26372] 2024-04-23T18:51:36.298 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555fcbb29b23] 2024-04-23T18:51:36.307 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 4: (MDLog::_replay_thread()+0x617) [0x555fcbb84647] 2024-04-23T18:51:36.315 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555fcbb8dd31] 2024-04-23T18:51:36.423 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb2dc8a81ca] 2024-04-23T18:51:36.431 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 7: clone() 2024-04-23T18:51:36.484 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:51:36.524 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:36.532 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 0> 2024-04-23T18:50:04.171+0000 7fb2cb4f1700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:51:36.541 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: in thread 7fb2cb4f1700 thread_name:md_log_replay 2024-04-23T18:51:36.550 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:36.558 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:51:36.605 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb2dc8b2d20] 2024-04-23T18:51:36.661 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2: /usr/bin/ceph-mds(+0x549372) [0x555fcbb26372] 2024-04-23T18:51:36.669 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555fcbb29b23] 2024-04-23T18:51:36.770 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 4: (MDLog::_replay_thread()+0x617) [0x555fcbb84647] 2024-04-23T18:51:36.882 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555fcbb8dd31] 2024-04-23T18:51:36.892 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb2dc8a81ca] 2024-04-23T18:51:36.943 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 7: clone() 2024-04-23T18:51:36.953 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:51:37.027 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[95686]: 2024-04-23T18:51:37.036 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 podman[96448]: 2024-04-23 18:50:04.224260711 +0000 UTC m=+0.023596498 container died d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T18:51:37.044 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 podman[96448]: 2024-04-23 18:50:04.955841218 +0000 UTC m=+0.755177058 container remove d9d7ced9b0fa4d72abf6348560870f21bed0c725747ae1458eed4a4b525b76b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T18:51:37.097 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:51:37.117 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:05 smithi033 ceph-mon[29274]: pgmap v759: 97 pgs: 97 active+clean; 15 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:51:37.206 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:05 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:51:37.215 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:36 smithi106 ceph-mon[32716]: pgmap v809: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:37.223 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:36 smithi122 ceph-mon[32546]: pgmap v809: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:51:37.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:07 smithi033 ceph-mon[29274]: pgmap v760: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:51:37.237 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:09 smithi033 ceph-mon[29274]: pgmap v761: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 39 op/s 2024-04-23T18:51:37.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:11 smithi033 ceph-mon[29274]: pgmap v762: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:51:37.335 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:11 smithi033 ceph-mon[29274]: osdmap e133: 12 total, 12 up, 12 in 2024-04-23T18:51:37.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:13 smithi033 ceph-mon[29274]: pgmap v764: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.0 KiB/s wr, 11 op/s 2024-04-23T18:51:37.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:15 smithi033 ceph-mon[29274]: pgmap v765: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:51:37.741 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 5. 2024-04-23T18:51:37.751 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:51:37.760 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 systemd[1]: Starting Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:51:37.768 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 podman[96557]: 2024-04-23 18:50:15.378581237 +0000 UTC m=+0.020766445 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:51:37.776 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 podman[96557]: 2024-04-23 18:50:15.523225974 +0000 UTC m=+0.165411152 container create bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T18:51:37.825 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 podman[96557]: 2024-04-23 18:50:15.859728755 +0000 UTC m=+0.501913935 container init bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T18:51:37.914 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 podman[96557]: 2024-04-23 18:50:15.862678193 +0000 UTC m=+0.504863379 container start bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-23T18:51:37.988 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 ceph-mds[96577]: -- 172.21.15.33:0/3342078161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9452f7a40 con 0x55f9452cb000 2024-04-23T18:51:38.027 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: starting mds.d at 2024-04-23T18:51:38.108 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 bash[96557]: bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e 2024-04-23T18:51:38.214 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:50:15 smithi033 systemd[1]: Started Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:51:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/4045596398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 msgr2=0x7f0f341025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:38.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/4045596398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341025b0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009900 tx=0x7f0f3002a410 comp rx=0 tx=0).stop 2024-04-23T18:51:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/4045596398 shutdown_connections 2024-04-23T18:51:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/4045596398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f341072a0 0x7f0f3410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/4045596398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f34102b80 0x7f0f34106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:38.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/4045596398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341025b0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/4045596398 >> 172.21.15.33:0/4045596398 conn(0x7f0f340793e0 msgr2=0x7f0f340797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/4045596398 shutdown_connections 2024-04-23T18:51:38.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.292+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/4045596398 wait complete. 2024-04-23T18:51:38.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 Processor -- start 2024-04-23T18:51:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 -- start start 2024-04-23T18:51:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 0x7f0f34198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 0x7f0f3419d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112200 con 0x7f0f341021d0 2024-04-23T18:51:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112080 con 0x7f0f3419cfa0 2024-04-23T18:51:38.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3b7cf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f34112380 con 0x7f0f341072a0 2024-04-23T18:51:38.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3956b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3956b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60358/0 (socket says 172.21.15.33:60358) 2024-04-23T18:51:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.293+0000 7f0f3956b640 1 -- 172.21.15.33:0/1057890441 learned_addr learned my addr 172.21.15.33:0/1057890441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 0x7f0f3419d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:39.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 -- 172.21.15.33:0/1057890441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 msgr2=0x7f0f34198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 0x7f0f34198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 0x7f0f34198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 -- 172.21.15.33:0/1057890441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 msgr2=0x7f0f3419d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 0x7f0f3419d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 -- 172.21.15.33:0/1057890441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3419da90 con 0x7f0f341021d0 2024-04-23T18:51:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 0x7f0f34198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 0x7f0f3419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:39.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005940 tx=0x7f0f300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f2a7fc640 1 -- 172.21.15.33:0/1057890441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3003f030 con 0x7f0f341021d0 2024-04-23T18:51:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f2a7fc640 1 -- 172.21.15.33:0/1057890441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f30003cb0 con 0x7f0f341021d0 2024-04-23T18:51:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3419dd20 con 0x7f0f341021d0 2024-04-23T18:51:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f2a7fc640 1 -- 172.21.15.33:0/1057890441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f30035070 con 0x7f0f341021d0 2024-04-23T18:51:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.294+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f341a5540 con 0x7f0f341021d0 2024-04-23T18:51:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.296+0000 7f0f2a7fc640 1 -- 172.21.15.33:0/1057890441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f30042030 con 0x7f0f341021d0 2024-04-23T18:51:39.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 Processor -- start 2024-04-23T18:51:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 -- start start 2024-04-23T18:51:40.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:40.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c38193e970 con 0x55c3819e8c10 2024-04-23T18:51:40.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c3818db4a0 con 0x55c3819d9e20 2024-04-23T18:51:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c38193a930 con 0x55c3819d9500 2024-04-23T18:51:40.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f2a7fc640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 0x7f0f200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60362/0 (socket says 172.21.15.33:60362) 2024-04-23T18:51:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 -- 172.21.15.33:0/1148003058 learned_addr learned my addr 172.21.15.33:0/1148003058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 0x7f0f200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f2a7fc640 1 -- 172.21.15.33:0/1057890441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f0f300f1100 con 0x7f0f341021d0 2024-04-23T18:51:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 -- 172.21.15.33:0/1148003058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 msgr2=0x55c3819d98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819d98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 -- 172.21.15.33:0/1148003058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 msgr2=0x55c3819e86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:41.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f38d6a640 1 -- 172.21.15.33:0/1148003058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c381651210 con 0x55c3819e8c10 2024-04-23T18:51:41.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:41.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.297+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009c20 tx=0x7f0f2400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee3754d72c84ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 0x7f0f200a6300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c009560 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:41.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f1f7fe640 1 -- 172.21.15.33:0/1148003058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24016070 con 0x55c3819e8c10 2024-04-23T18:51:41.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f1f7fe640 1 -- 172.21.15.33:0/1148003058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24011920 con 0x55c3819e8c10 2024-04-23T18:51:41.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f1f7fe640 1 -- 172.21.15.33:0/1148003058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24010810 con 0x55c3819e8c10 2024-04-23T18:51:41.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1148003058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 msgr2=0x55c3819eb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:41.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.298+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009c20 tx=0x7f0f2400e6c0 comp rx=0 tx=0).stop 2024-04-23T18:51:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1148003058 shutdown_connections 2024-04-23T18:51:41.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819eb000 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:41.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1148003058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819d98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1148003058 >> 172.21.15.33:0/1148003058 conn(0x55c3819d5b30 msgr2=0x55c3819d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1148003058 shutdown_connections 2024-04-23T18:51:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1148003058 wait complete. 2024-04-23T18:51:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- start start 2024-04-23T18:51:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 Processor -- start 2024-04-23T18:51:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:42.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 0x55c3819e5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3819eb840 con 0x55c3819d9e20 2024-04-23T18:51:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c3818d90b0 con 0x55c3819e8c10 2024-04-23T18:51:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c3819eb9c0 con 0x55c3819d9500 2024-04-23T18:51:42.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f3e5bd740 10 client.0 _ll_register_callbacks cb 0x55c3818c6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:42.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.299+0000 7f0f38d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:42.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60376/0 (socket says 172.21.15.33:60376) 2024-04-23T18:51:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 -- 172.21.15.33:0/449500272 learned_addr learned my addr 172.21.15.33:0/449500272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f3956b640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:42.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 0x55c3819e5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:42.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 -- 172.21.15.33:0/449500272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 msgr2=0x55c3819e8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:42.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:42.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 -- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 msgr2=0x55c3819e5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 0x55c3819e5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:43.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 -- 172.21.15.33:0/449500272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c38171d620 con 0x55c3819d9e20 2024-04-23T18:51:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 0x55c3819e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f3956b640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009bf0 tx=0x7f0f240098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc17858a683c9db server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24028060 con 0x55c3819d9e20 2024-04-23T18:51:43.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f0f24005040 con 0x55c3819d9e20 2024-04-23T18:51:43.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f24011920 con 0x55c3819d9e20 2024-04-23T18:51:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0f24016070 con 0x55c3819d9e20 2024-04-23T18:51:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f3e5bd740 10 client.18072 fetch_fsmap learned FSMap version 44 2024-04-23T18:51:43.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c3819f77c0 con 0x55c3819d9e20 2024-04-23T18:51:43.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.300+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f24011920 con 0x55c3819d9e20 2024-04-23T18:51:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f0f2400f040 con 0x55c3819d9e20 2024-04-23T18:51:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 10 client.18072 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:51:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 10 client.18072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 20 client.18072 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c3819f8c20 0x55c3819fb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:43.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 4 client.18072 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:51:43.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c38194c390 con 0x55c3819f8c20 2024-04-23T18:51:43.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3956b640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c3819f8c20 0x55c3819fb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:43.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f3956b640 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c3819f8c20 0x55c3819fb010 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669bb48f56690fd4 server_cookie=cfaefce814033895 in_seq=0 out_seq=0 2024-04-23T18:51:43.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.301+0000 7f0f1cff9640 10 client.18072 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f1cff9640 1 -- 172.21.15.33:0/449500272 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c38194c390 con 0x55c3819f8c20 2024-04-23T18:51:43.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f1cff9640 10 client.18072 handle_command_reply: tid=0 2024-04-23T18:51:43.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 client.18072 shutdown 2024-04-23T18:51:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 msgr2=0x55c3819e57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:43.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009bf0 tx=0x7f0f240098b0 comp rx=0 tx=0).stop 2024-04-23T18:51:43.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 shutdown_connections 2024-04-23T18:51:44.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c3819f8c20 0x55c3819fb010 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819e8c10 0x55c3819e5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:44.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819d9e20 0x55c3819e57f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f1cff9640 0 client.18072 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:44.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/449500272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 >> 172.21.15.33:0/449500272 conn(0x55c3819d5b30 msgr2=0x55c3819db9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 shutdown_connections 2024-04-23T18:51:44.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.302+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/449500272 wait complete. 2024-04-23T18:51:44.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.303+0000 7f0f3e5bd740 20 client.18072 trim_cache size 0 max 16384 2024-04-23T18:51:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 Processor -- start 2024-04-23T18:51:44.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 -- start start 2024-04-23T18:51:44.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:44.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ad3a0 con 0x55c3819e8c10 2024-04-23T18:51:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ab190 con 0x55c3819d9e20 2024-04-23T18:51:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341accd0 con 0x55c3819d9500 2024-04-23T18:51:44.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f39d6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f39d6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60390/0 (socket says 172.21.15.33:60390) 2024-04-23T18:51:45.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.317+0000 7f0f39d6c640 1 -- 172.21.15.33:0/157385779 learned_addr learned my addr 172.21.15.33:0/157385779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:45.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f3956b640 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 -- 172.21.15.33:0/157385779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 msgr2=0x55c3819e0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:45.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 -- 172.21.15.33:0/157385779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 msgr2=0x55c3819d4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:45.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819d4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:45.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 -- 172.21.15.33:0/157385779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3819d5960 con 0x55c3819e8c10 2024-04-23T18:51:45.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c012660 tx=0x7f0f2c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a65dfdda3036808 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:45.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f1effd640 1 -- 172.21.15.33:0/157385779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c018070 con 0x55c3819e8c10 2024-04-23T18:51:45.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f1effd640 1 -- 172.21.15.33:0/157385779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2c010e00 con 0x55c3819e8c10 2024-04-23T18:51:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.318+0000 7f0f1effd640 1 -- 172.21.15.33:0/157385779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c028410 con 0x55c3819e8c10 2024-04-23T18:51:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/157385779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 msgr2=0x55c3819e8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:45.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c012660 tx=0x7f0f2c009240 comp rx=0 tx=0).stop 2024-04-23T18:51:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/157385779 shutdown_connections 2024-04-23T18:51:45.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e8030 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819d4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:45.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/157385779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819e0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:45.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/157385779 >> 172.21.15.33:0/157385779 conn(0x55c3819d5b30 msgr2=0x55c3819d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/157385779 shutdown_connections 2024-04-23T18:51:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/157385779 wait complete. 2024-04-23T18:51:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- start start 2024-04-23T18:51:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 Processor -- start 2024-04-23T18:51:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819f1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:46.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341abf30 con 0x55c3819e8c10 2024-04-23T18:51:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ab860 con 0x55c3819d9e20 2024-04-23T18:51:46.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.319+0000 7f0f3e5bd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f341ac600 con 0x55c3819d9500 2024-04-23T18:51:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f3e5bd740 10 client.0 _ll_register_callbacks cb 0x7f0f24011870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:46.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:46.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60400/0 (socket says 172.21.15.33:60400) 2024-04-23T18:51:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 -- 172.21.15.33:0/1538842467 learned_addr learned my addr 172.21.15.33:0/1538842467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819f1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f38d6a640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 -- 172.21.15.33:0/1538842467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 msgr2=0x55c3819f1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819f1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:47.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 -- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 msgr2=0x55c3819e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:47.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 -- 172.21.15.33:0/1538842467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3819e3040 con 0x55c3819e8c10 2024-04-23T18:51:47.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f39d6c640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009de0 tx=0x7f0f2c009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d0341500d62933 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:47.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c018040 con 0x55c3819e8c10 2024-04-23T18:51:47.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f0f2c01e450 con 0x55c3819e8c10 2024-04-23T18:51:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2c010e00 con 0x55c3819e8c10 2024-04-23T18:51:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0f2c01e650 con 0x55c3819e8c10 2024-04-23T18:51:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f3e5bd740 10 client.18078 fetch_fsmap learned FSMap version 44 2024-04-23T18:51:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.320+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c381a5f100 con 0x55c3819e8c10 2024-04-23T18:51:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f2c010e00 con 0x55c3819e8c10 2024-04-23T18:51:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f0f2c01b040 con 0x55c3819e8c10 2024-04-23T18:51:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 10 client.18078 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:51:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:47.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 20 client.18078 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:47.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c381a5f800 0x55c381a61c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 4 client.18078 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:51:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c381946200 con 0x55c381a5f800 2024-04-23T18:51:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c381a5f800 0x55c381a61c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f1cff9640 10 client.18078 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.321+0000 7f0f3956b640 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c381a5f800 0x55c381a61c20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a5fdbda8d617c4 server_cookie=279be337b855c078 in_seq=0 out_seq=0 2024-04-23T18:51:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f1cff9640 1 -- 172.21.15.33:0/1538842467 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c381946200 con 0x55c381a5f800 2024-04-23T18:51:48.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f1cff9640 10 client.18078 handle_command_reply: tid=0 2024-04-23T18:51:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 client.18078 shutdown 2024-04-23T18:51:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 msgr2=0x55c3819e2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:48.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009de0 tx=0x7f0f2c009690 comp rx=0 tx=0).stop 2024-04-23T18:51:48.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 shutdown_connections 2024-04-23T18:51:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c381a5f800 0x55c381a61c20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3819e8c10 0x55c3819e2810 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c3819d9e20 0x55c3819e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f1cff9640 0 client.18078 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:48.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 --2- 172.21.15.33:0/1538842467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c3819d9500 0x55c3819f1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 >> 172.21.15.33:0/1538842467 conn(0x55c3819d5b30 msgr2=0x55c3819d4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:48.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 shutdown_connections 2024-04-23T18:51:48.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.328+0000 7f0f3e5bd740 1 -- 172.21.15.33:0/1538842467 wait complete. 2024-04-23T18:51:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3e5bd740 20 client.18078 trim_cache size 0 max 16384 2024-04-23T18:51:48.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:51:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:51:48.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:51:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:51:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 msgr2=0x7f0f200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 0x7f0f200a6300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c005fd0 tx=0x7f0f2c009560 comp rx=0 tx=0).stop 2024-04-23T18:51:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 msgr2=0x7f0f341986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:49.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005940 tx=0x7f0f300042f0 comp rx=0 tx=0).stop 2024-04-23T18:51:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 shutdown_connections 2024-04-23T18:51:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f200a3e40 0x7f0f200a6300 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3419cfa0 0x7f0f3419d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f341072a0 0x7f0f34198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:49.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 --2- 172.21.15.33:0/1057890441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f341021d0 0x7f0f341986e0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:49.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 >> 172.21.15.33:0/1057890441 conn(0x7f0f340793e0 msgr2=0x7f0f340ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:49.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 shutdown_connections 2024-04-23T18:51:49.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:16.329+0000 7f0f3b7cf640 1 -- 172.21.15.33:0/1057890441 wait complete. 2024-04-23T18:51:49.762 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:16 smithi033 ceph-mon[29274]: daemon mds.d restarted 2024-04-23T18:51:49.773 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:16 smithi033 ceph-mon[29274]: osdmap e134: 12 total, 12 up, 12 in 2024-04-23T18:51:49.784 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/2383185438,v1:172.21.15.33:6837/2383185438] up:boot 2024-04-23T18:51:49.836 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:51:49.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-23T18:51:49.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:17 smithi033 ceph-mon[29274]: pgmap v767: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:51:50.026 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:19 smithi033 ceph-mon[29274]: pgmap v768: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 127 B/s wr, 10 op/s 2024-04-23T18:51:50.037 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:21 smithi033 ceph-mon[29274]: pgmap v769: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-23T18:51:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:23 smithi033 ceph-mon[29274]: pgmap v770: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:51:50.112 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:25 smithi033 ceph-mon[29274]: pgmap v771: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:51:50.122 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:27 smithi033 ceph-mon[29274]: pgmap v772: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 408 B/s wr, 11 op/s 2024-04-23T18:51:50.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:38 smithi106 ceph-mon[32716]: pgmap v810: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:51:50.203 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:40 smithi106 ceph-mon[32716]: pgmap v811: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.212 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:42 smithi106 ceph-mon[32716]: pgmap v812: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:50.450 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:44 smithi106 ceph-mon[32716]: pgmap v813: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.512 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:46 smithi106 ceph-mon[32716]: pgmap v814: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.522 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:48 smithi106 ceph-mon[32716]: pgmap v815: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:50.532 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:38 smithi122 ceph-mon[32546]: pgmap v810: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:51:50.541 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:40 smithi122 ceph-mon[32546]: pgmap v811: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.552 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:42 smithi122 ceph-mon[32546]: pgmap v812: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:50.607 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:44 smithi122 ceph-mon[32546]: pgmap v813: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.668 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:46 smithi122 ceph-mon[32546]: pgmap v814: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:51:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:48 smithi122 ceph-mon[32546]: pgmap v815: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:51:50.730 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:50 smithi106 ceph-mon[32716]: pgmap v816: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:50.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:50 smithi122 ceph-mon[32546]: pgmap v816: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:51:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 -- 172.21.15.33:0/2599552297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3501005c0 msgr2=0x7fa350100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:50.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2599552297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3501005c0 0x7fa350100a40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fa344009930 tx=0x7fa34402a420 comp rx=0 tx=0).stop 2024-04-23T18:51:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 -- 172.21.15.33:0/2599552297 shutdown_connections 2024-04-23T18:51:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2599552297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa350100f80 0x7fa35010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2599552297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3501005c0 0x7fa350100a40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2599552297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350106660 0x7fa350106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:51.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 -- 172.21.15.33:0/2599552297 >> 172.21.15.33:0/2599552297 conn(0x7fa350077390 msgr2=0x7fa3500fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 -- 172.21.15.33:0/2599552297 shutdown_connections 2024-04-23T18:51:51.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.428+0000 7fa3562a7640 1 -- 172.21.15.33:0/2599552297 wait complete. 2024-04-23T18:51:51.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 Processor -- start 2024-04-23T18:51:51.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 -- start start 2024-04-23T18:51:51.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501005c0 0x7fa350102260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:51.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 0x7fa3501027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa350110890 con 0x7fa350106660 2024-04-23T18:51:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa350110710 con 0x7fa3501005c0 2024-04-23T18:51:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.429+0000 7fa3562a7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa350110a10 con 0x7fa350100f80 2024-04-23T18:51:51.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 0x7fa3501027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36954/0 (socket says 172.21.15.33:36954) 2024-04-23T18:51:51.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 0x7fa3501027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:47178/0 (socket says 172.21.15.33:47178) 2024-04-23T18:51:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 -- 172.21.15.33:0/2966685291 learned_addr learned my addr 172.21.15.33:0/2966685291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34ffff640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501005c0 0x7fa350102260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 -- 172.21.15.33:0/2966685291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 msgr2=0x7fa3501027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 0x7fa3501027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 -- 172.21.15.33:0/2966685291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501005c0 msgr2=0x7fa350102260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:52.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501005c0 0x7fa350102260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 -- 172.21.15.33:0/2966685291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501a6be0 con 0x7fa350106660 2024-04-23T18:51:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa354844640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa34800eb10 tx=0x7fa34800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34d7fa640 1 -- 172.21.15.33:0/2966685291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348004b20 con 0x7fa350106660 2024-04-23T18:51:52.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34d7fa640 1 -- 172.21.15.33:0/2966685291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa348004cc0 con 0x7fa350106660 2024-04-23T18:51:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501a6ed0 con 0x7fa350106660 2024-04-23T18:51:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa34d7fa640 1 -- 172.21.15.33:0/2966685291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348010670 con 0x7fa350106660 2024-04-23T18:51:52.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.430+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501a73a0 con 0x7fa350106660 2024-04-23T18:51:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa34d7fa640 1 -- 172.21.15.33:0/2966685291 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa348005020 con 0x7fa350106660 2024-04-23T18:51:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 Processor -- start 2024-04-23T18:51:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 -- start start 2024-04-23T18:51:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:53.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d2112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d213bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f26d16b970 con 0x55f26d206620 2024-04-23T18:51:53.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f26d1084a0 con 0x55f26d211800 2024-04-23T18:51:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.433+0000 7fa358fc4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f26d167930 con 0x55f26d206f40 2024-04-23T18:51:53.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34d7fa640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 0x7fa3380a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:53.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36962/0 (socket says 172.21.15.33:36962) 2024-04-23T18:51:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1546218419 learned_addr learned my addr 172.21.15.33:0/1546218419 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34ffff640 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d2112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa354844640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 0x7fa3380a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34d7fa640 1 -- 172.21.15.33:0/2966685291 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fa3480d0d60 con 0x7fa350106660 2024-04-23T18:51:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d213bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1546218419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 msgr2=0x55f26d2112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d2112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1546218419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 msgr2=0x55f26d213bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d213bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1546218419 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26ce7e210 con 0x55f26d206620 2024-04-23T18:51:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c00 tx=0x7fa344005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bafa97344cdea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:53.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa354844640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 0x7fa3380a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3480ce570 tx=0x7fa34806b6c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa3367fc640 1 -- 172.21.15.33:0/1546218419 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344044030 con 0x55f26d206620 2024-04-23T18:51:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa3367fc640 1 -- 172.21.15.33:0/1546218419 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344004030 con 0x55f26d206620 2024-04-23T18:51:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.434+0000 7fa3367fc640 1 -- 172.21.15.33:0/1546218419 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344035040 con 0x55f26d206620 2024-04-23T18:51:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- 172.21.15.33:0/1546218419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 msgr2=0x55f26d206a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c00 tx=0x7fa344005e50 comp rx=0 tx=0).stop 2024-04-23T18:51:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- 172.21.15.33:0/1546218419 shutdown_connections 2024-04-23T18:51:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d213bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d2112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:54.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1546218419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d206a00 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- 172.21.15.33:0/1546218419 >> 172.21.15.33:0/1546218419 conn(0x55f26d202b30 msgr2=0x55f26d202f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- 172.21.15.33:0/1546218419 shutdown_connections 2024-04-23T18:51:54.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- 172.21.15.33:0/1546218419 wait complete. 2024-04-23T18:51:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.435+0000 7fa358fc4740 1 -- start start 2024-04-23T18:51:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 Processor -- start 2024-04-23T18:51:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d20e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:54.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d20e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:54.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f26d2143d0 con 0x55f26d206620 2024-04-23T18:51:54.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f26d1060b0 con 0x55f26d211800 2024-04-23T18:51:54.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f26d214550 con 0x55f26d206f40 2024-04-23T18:51:54.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa358fc4740 10 client.0 _ll_register_callbacks cb 0x55f26d0f3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:54.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36970/0 (socket says 172.21.15.33:36970) 2024-04-23T18:51:54.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa34f7fe640 1 -- 172.21.15.33:0/870963369 learned_addr learned my addr 172.21.15.33:0/870963369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa34ffff640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d20e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.436+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d20e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 -- 172.21.15.33:0/870963369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 msgr2=0x55f26d20e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d20e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 -- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 msgr2=0x55f26d20e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d20e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 -- 172.21.15.33:0/870963369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26cf4a620 con 0x55f26d206620 2024-04-23T18:51:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa344031d40 tx=0x7fa344005160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a4693fe164e6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:55.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344052060 con 0x55f26d206620 2024-04-23T18:51:55.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fa344031720 con 0x55f26d206620 2024-04-23T18:51:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa344004030 con 0x55f26d206620 2024-04-23T18:51:55.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa344035070 con 0x55f26d206620 2024-04-23T18:51:55.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa358fc4740 10 client.18090 fetch_fsmap learned FSMap version 44 2024-04-23T18:51:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f26d220300 con 0x55f26d206620 2024-04-23T18:51:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344004030 con 0x55f26d206620 2024-04-23T18:51:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.437+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7fa344004c20 con 0x55f26d206620 2024-04-23T18:51:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 10 client.18090 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:51:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 10 client.18090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:51:55.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 20 client.18090 populate_metadata read hostname 'smithi033' 2024-04-23T18:51:55.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d221650 0x55f26d223a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:55.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 4 client.18090 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:51:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f26d179390 con 0x55f26d221650 2024-04-23T18:51:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa34ffff640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d221650 0x55f26d223a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa34ffff640 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d221650 0x55f26d223a40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf4a75a105d159c server_cookie=8b1ed1a5fc32e63b in_seq=0 out_seq=0 2024-04-23T18:51:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.438+0000 7fa32f7fe640 10 client.18090 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.439+0000 7fa32f7fe640 1 -- 172.21.15.33:0/870963369 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f26d179390 con 0x55f26d221650 2024-04-23T18:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.439+0000 7fa32f7fe640 10 client.18090 handle_command_reply: tid=0 2024-04-23T18:51:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.439+0000 7fa358fc4740 1 client.18090 shutdown 2024-04-23T18:51:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 msgr2=0x55f26d211160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fa344031d40 tx=0x7fa344005160 comp rx=0 tx=0).stop 2024-04-23T18:51:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 shutdown_connections 2024-04-23T18:51:56.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d221650 0x55f26d223a40 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d211800 0x55f26d20e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:56.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa32f7fe640 0 client.18090 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:51:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d206f40 0x55f26d20e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 --2- 172.21.15.33:0/870963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d211160 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 >> 172.21.15.33:0/870963369 conn(0x55f26d202b30 msgr2=0x55f26d2088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 shutdown_connections 2024-04-23T18:51:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 1 -- 172.21.15.33:0/870963369 wait complete. 2024-04-23T18:51:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.440+0000 7fa358fc4740 20 client.18090 trim_cache size 0 max 16384 2024-04-23T18:51:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 Processor -- start 2024-04-23T18:51:56.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 -- start start 2024-04-23T18:51:56.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:56.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d210c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d226a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:57.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b1860 con 0x55f26d206620 2024-04-23T18:51:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501af650 con 0x55f26d206f40 2024-04-23T18:51:57.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa358fc4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b1190 con 0x55f26d211800 2024-04-23T18:51:57.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36986/0 (socket says 172.21.15.33:36986) 2024-04-23T18:51:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.455+0000 7fa34f7fe640 1 -- 172.21.15.33:0/543031436 learned_addr learned my addr 172.21.15.33:0/543031436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:57.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 -- 172.21.15.33:0/543031436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 msgr2=0x55f26d226a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:57.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d226a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34ffff640 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d210c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:57.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 -- 172.21.15.33:0/543031436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 msgr2=0x55f26d210c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:57.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d210c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 -- 172.21.15.33:0/543031436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f26d20ca00 con 0x55f26d206620 2024-04-23T18:51:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34ffff640 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d210c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:51:57.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa344031440 tx=0x7fa34402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf240f7a545b8de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa335ffb640 1 -- 172.21.15.33:0/543031436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344052040 con 0x55f26d206620 2024-04-23T18:51:58.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa335ffb640 1 -- 172.21.15.33:0/543031436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344031d00 con 0x55f26d206620 2024-04-23T18:51:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa335ffb640 1 -- 172.21.15.33:0/543031436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344044040 con 0x55f26d206620 2024-04-23T18:51:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa358fc4740 1 -- 172.21.15.33:0/543031436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 msgr2=0x55f26d201b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:51:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.456+0000 7fa358fc4740 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fa344031440 tx=0x7fa34402a420 comp rx=0 tx=0).stop 2024-04-23T18:51:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- 172.21.15.33:0/543031436 shutdown_connections 2024-04-23T18:51:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d226a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d210c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- 172.21.15.33:0/543031436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d201b30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- 172.21.15.33:0/543031436 >> 172.21.15.33:0/543031436 conn(0x55f26d202b30 msgr2=0x55f26d201420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:51:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- 172.21.15.33:0/543031436 shutdown_connections 2024-04-23T18:51:58.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- 172.21.15.33:0/543031436 wait complete. 2024-04-23T18:51:58.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- start start 2024-04-23T18:51:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 Processor -- start 2024-04-23T18:51:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:58.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d224040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:58.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d224580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:51:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b03f0 con 0x55f26d206620 2024-04-23T18:51:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501afd20 con 0x55f26d206f40 2024-04-23T18:51:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3501b0ac0 con 0x55f26d211800 2024-04-23T18:51:59.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.457+0000 7fa358fc4740 10 client.0 _ll_register_callbacks cb 0x7fa344031bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:51:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:59.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37000/0 (socket says 172.21.15.33:37000) 2024-04-23T18:51:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1346858011 learned_addr learned my addr 172.21.15.33:0/1346858011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:51:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34ffff640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d224040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:51:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1346858011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 msgr2=0x55f26d224580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d224580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 msgr2=0x55f26d224040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:51:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d224040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:51:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 -- 172.21.15.33:0/1346858011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f26d224bc0 con 0x55f26d206620 2024-04-23T18:51:59.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34ffff640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d224040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:51:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa34f7fe640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa344005100 tx=0x7fa344004100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ff3d16b60c8412 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:51:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa32f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:51:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344063060 con 0x55f26d206620 2024-04-23T18:51:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fa344005160 con 0x55f26d206620 2024-04-23T18:51:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.458+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa344031d00 con 0x55f26d206620 2024-04-23T18:51:59.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fa344044040 con 0x55f26d206620 2024-04-23T18:51:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 10 client.18096 fetch_fsmap learned FSMap version 44 2024-04-23T18:51:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f26d2875c0 con 0x55f26d206620 2024-04-23T18:51:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa344031d00 con 0x55f26d206620 2024-04-23T18:51:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7fa344005360 con 0x55f26d206620 2024-04-23T18:52:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 10 client.18096 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:00.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 10 client.18096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:00.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 20 client.18096 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d287bc0 0x55f26d289fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 4 client.18096 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:52:00.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f26d173200 con 0x55f26d287bc0 2024-04-23T18:52:00.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.459+0000 7fa34ffff640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d287bc0 0x55f26d289fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:00.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.460+0000 7fa34ffff640 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d287bc0 0x55f26d289fe0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5306435c347b2b8 server_cookie=bb46d9d4e3f5c3dc in_seq=0 out_seq=0 2024-04-23T18:52:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.460+0000 7fa32f7fe640 10 client.18096 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa32f7fe640 1 -- 172.21.15.33:0/1346858011 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f26d173200 con 0x55f26d287bc0 2024-04-23T18:52:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa32f7fe640 10 client.18096 handle_command_reply: tid=0 2024-04-23T18:52:00.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 client.18096 shutdown 2024-04-23T18:52:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 msgr2=0x55f26d226c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:00.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa344005100 tx=0x7fa344004100 comp rx=0 tx=0).stop 2024-04-23T18:52:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 shutdown_connections 2024-04-23T18:52:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55f26d287bc0 0x55f26d289fe0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f26d211800 0x55f26d224580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f26d206f40 0x55f26d224040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 --2- 172.21.15.33:0/1346858011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f26d206620 0x55f26d226c50 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa32f7fe640 0 client.18096 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 >> 172.21.15.33:0/1346858011 conn(0x55f26d202b30 msgr2=0x55f26d201cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 shutdown_connections 2024-04-23T18:52:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 1 -- 172.21.15.33:0/1346858011 wait complete. 2024-04-23T18:52:01.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.464+0000 7fa358fc4740 20 client.18096 trim_cache size 0 max 16384 2024-04-23T18:52:01.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:52:01.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:52:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:52:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:52:01.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 msgr2=0x7fa3380a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 0x7fa3380a6280 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3480ce570 tx=0x7fa34806b6c0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 msgr2=0x7fa350102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fa34800eb10 tx=0x7fa34800c2d0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 shutdown_connections 2024-04-23T18:52:01.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa3380a3dc0 0x7fa3380a6280 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa350106660 0x7fa350102ce0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa350100f80 0x7fa3501027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 --2- 172.21.15.33:0/2966685291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3501005c0 0x7fa350102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 >> 172.21.15.33:0/2966685291 conn(0x7fa350077390 msgr2=0x7fa3500fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:01.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 shutdown_connections 2024-04-23T18:52:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:29.465+0000 7fa3562a7640 1 -- 172.21.15.33:0/2966685291 wait complete. 2024-04-23T18:52:01.852 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:29 smithi033 ceph-mon[29274]: pgmap v773: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:52:01.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:31 smithi033 ceph-mon[29274]: pgmap v774: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:52:01.869 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:33 smithi033 ceph-mon[29274]: pgmap v775: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:52:01.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:34 smithi033 ceph-mon[29274]: pgmap v776: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:01.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:37 smithi033 ceph-mon[29274]: pgmap v777: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:02.009 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:39 smithi033 ceph-mon[29274]: pgmap v778: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:41 smithi033 ceph-mon[29274]: pgmap v779: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.157 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:52 smithi106 ceph-mon[32716]: pgmap v817: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:02.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:54 smithi106 ceph-mon[32716]: pgmap v818: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:56 smithi106 ceph-mon[32716]: pgmap v819: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.342 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:51:58 smithi106 ceph-mon[32716]: pgmap v820: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:02.350 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:00 smithi106 ceph-mon[32716]: pgmap v821: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.429 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:01 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:02.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:52 smithi122 ceph-mon[32546]: pgmap v817: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:02.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:54 smithi122 ceph-mon[32546]: pgmap v818: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.757 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:56 smithi122 ceph-mon[32546]: pgmap v819: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:51:58 smithi122 ceph-mon[32546]: pgmap v820: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:02.882 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:00 smithi122 ceph-mon[32546]: pgmap v821: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:02.903 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:01 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:03.005 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:02 smithi106 ceph-mon[32716]: pgmap v822: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:03.015 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:02 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:03.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:02 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:03.113 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:02 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:52:03.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:02 smithi122 ceph-mon[32546]: pgmap v822: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:52:03.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:02 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:03.205 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:02 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:03.212 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:02 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:52:03.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 -- 172.21.15.33:0/1042469869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 msgr2=0x7f3e781068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:03.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/1042469869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e781068a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009cd0 tx=0x7f3e7002a970 comp rx=0 tx=0).stop 2024-04-23T18:52:03.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 -- 172.21.15.33:0/1042469869 shutdown_connections 2024-04-23T18:52:03.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/1042469869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e781068a0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/1042469869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78101910 0x7f3e78105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:03.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/1042469869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78100ff0 0x7f3e781013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.566+0000 7f3e7f75c640 1 -- 172.21.15.33:0/1042469869 >> 172.21.15.33:0/1042469869 conn(0x7f3e780771d0 msgr2=0x7f3e780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- 172.21.15.33:0/1042469869 shutdown_connections 2024-04-23T18:52:03.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- 172.21.15.33:0/1042469869 wait complete. 2024-04-23T18:52:03.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 Processor -- start 2024-04-23T18:52:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- start start 2024-04-23T18:52:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78100ff0 0x7f3e781a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78101910 0x7f3e781a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:04.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781187d0 con 0x7f3e781063f0 2024-04-23T18:52:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78118650 con 0x7f3e78100ff0 2024-04-23T18:52:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.567+0000 7f3e7f75c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78118950 con 0x7f3e78101910 2024-04-23T18:52:04.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:04.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59544/0 (socket says 172.21.15.33:59544) 2024-04-23T18:52:04.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 -- 172.21.15.33:0/3577459887 learned_addr learned my addr 172.21.15.33:0/3577459887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78101910 0x7f3e781a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78100ff0 0x7f3e781a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:04.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 -- 172.21.15.33:0/3577459887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78101910 msgr2=0x7f3e781a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78101910 0x7f3e781a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:04.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 -- 172.21.15.33:0/3577459887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78100ff0 msgr2=0x7f3e781a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:04.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78100ff0 0x7f3e781a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 -- 172.21.15.33:0/3577459887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7819f1f0 con 0x7f3e781063f0 2024-04-23T18:52:04.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7dcf9640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009df0 tx=0x7f3e700093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:04.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e6e7fc640 1 -- 172.21.15.33:0/3577459887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70047070 con 0x7f3e781063f0 2024-04-23T18:52:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e6e7fc640 1 -- 172.21.15.33:0/3577459887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e7002f830 con 0x7f3e781063f0 2024-04-23T18:52:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7819f480 con 0x7f3e781063f0 2024-04-23T18:52:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e6e7fc640 1 -- 172.21.15.33:0/3577459887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70034070 con 0x7f3e781063f0 2024-04-23T18:52:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.568+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781ab780 con 0x7f3e781063f0 2024-04-23T18:52:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.571+0000 7f3e6e7fc640 1 -- 172.21.15.33:0/3577459887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3e7004a070 con 0x7f3e781063f0 2024-04-23T18:52:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.571+0000 7f3e82471740 1 Processor -- start 2024-04-23T18:52:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.571+0000 7f3e82471740 1 -- start start 2024-04-23T18:52:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84be2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:05.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561db8418970 con 0x561db84be800 2024-04-23T18:52:05.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561db83b54a0 con 0x561db84b3f40 2024-04-23T18:52:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e82471740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561db8414930 con 0x561db84b3620 2024-04-23T18:52:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e6e7fc640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 0x7f3e640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7dcf9640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 0x7f3e640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59550/0 (socket says 172.21.15.33:59550) 2024-04-23T18:52:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/725311362 learned_addr learned my addr 172.21.15.33:0/725311362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:05.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84be2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e6e7fc640 1 -- 172.21.15.33:0/3577459887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f3e700f0c80 con 0x7f3e781063f0 2024-04-23T18:52:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/725311362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 msgr2=0x561db84b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/725311362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 msgr2=0x561db84be2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:06.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84be2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/725311362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db812b210 con 0x561db84be800 2024-04-23T18:52:06.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:52:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7dcf9640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 0x7f3e640a6300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3e700ee490 tx=0x7f3e7008e8d0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.572+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009560 tx=0x7f3e68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a545b5b67613a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e637fe640 1 -- 172.21.15.33:0/725311362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004050 con 0x561db84be800 2024-04-23T18:52:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e637fe640 1 -- 172.21.15.33:0/725311362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e680054d0 con 0x561db84be800 2024-04-23T18:52:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e637fe640 1 -- 172.21.15.33:0/725311362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004b20 con 0x561db84be800 2024-04-23T18:52:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 -- 172.21.15.33:0/725311362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 msgr2=0x561db84c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009560 tx=0x7f3e68004730 comp rx=0 tx=0).stop 2024-04-23T18:52:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 -- 172.21.15.33:0/725311362 shutdown_connections 2024-04-23T18:52:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84c0bf0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84be2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:06.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 --2- 172.21.15.33:0/725311362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:06.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 -- 172.21.15.33:0/725311362 >> 172.21.15.33:0/725311362 conn(0x561db84afb30 msgr2=0x561db84aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.573+0000 7f3e82471740 1 -- 172.21.15.33:0/725311362 shutdown_connections 2024-04-23T18:52:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 -- 172.21.15.33:0/725311362 wait complete. 2024-04-23T18:52:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 -- start start 2024-04-23T18:52:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 Processor -- start 2024-04-23T18:52:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84be800 0x561db84bbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561db84c1430 con 0x561db84b3f40 2024-04-23T18:52:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561db83b30b0 con 0x561db84be800 2024-04-23T18:52:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.574+0000 7f3e82471740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561db84c15b0 con 0x561db84b3620 2024-04-23T18:52:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e82471740 10 client.0 _ll_register_callbacks cb 0x561db83a0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59556/0 (socket says 172.21.15.33:59556) 2024-04-23T18:52:07.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 -- 172.21.15.33:0/2285072583 learned_addr learned my addr 172.21.15.33:0/2285072583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84be800 0x561db84bbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 -- 172.21.15.33:0/2285072583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 msgr2=0x561db84be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:07.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 -- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84be800 msgr2=0x561db84bbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84be800 0x561db84bbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 -- 172.21.15.33:0/2285072583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db81f7620 con 0x561db84b3f40 2024-04-23T18:52:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:52:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400bad0 tx=0x7f3e7400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3f08f7b7d29dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:07.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74005020 con 0x561db84b3f40 2024-04-23T18:52:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f3e740051c0 con 0x561db84b3f40 2024-04-23T18:52:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e680054d0 con 0x561db84b3f40 2024-04-23T18:52:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3e740053c0 con 0x561db84b3f40 2024-04-23T18:52:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e82471740 10 client.18108 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:08.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.575+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561db84cd460 con 0x561db84b3f40 2024-04-23T18:52:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e680054d0 con 0x561db84b3f40 2024-04-23T18:52:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f3e7401b590 con 0x561db84b3f40 2024-04-23T18:52:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 10 client.18108 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 10 client.18108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:08.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 20 client.18108 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db84ce970 0x561db84d0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 4 client.18108 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:52:08.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561db8426390 con 0x561db84ce970 2024-04-23T18:52:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db84ce970 0x561db84d0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db84ce970 0x561db84d0d60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28064b384a50128a server_cookie=810d5480a01a7e90 in_seq=0 out_seq=0 2024-04-23T18:52:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.576+0000 7f3e60ff9640 10 client.18108 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e60ff9640 1 -- 172.21.15.33:0/2285072583 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561db8426390 con 0x561db84ce970 2024-04-23T18:52:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e60ff9640 10 client.18108 handle_command_reply: tid=0 2024-04-23T18:52:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 client.18108 shutdown 2024-04-23T18:52:09.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 msgr2=0x561db84bb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400bad0 tx=0x7f3e7400bf90 comp rx=0 tx=0).stop 2024-04-23T18:52:09.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 shutdown_connections 2024-04-23T18:52:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db84ce970 0x561db84d0d60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84be800 0x561db84bbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:09.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3f40 0x561db84bb4b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e60ff9640 0 client.18108 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:09.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 --2- 172.21.15.33:0/2285072583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.577+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 >> 172.21.15.33:0/2285072583 conn(0x561db84afb30 msgr2=0x561db84b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.578+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 shutdown_connections 2024-04-23T18:52:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.578+0000 7f3e82471740 1 -- 172.21.15.33:0/2285072583 wait complete. 2024-04-23T18:52:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.578+0000 7f3e82471740 20 client.18108 trim_cache size 0 max 16384 2024-04-23T18:52:09.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 Processor -- start 2024-04-23T18:52:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 -- start start 2024-04-23T18:52:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:09.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:09.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84be800 0x561db84b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:09.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b5a30 con 0x561db84b3620 2024-04-23T18:52:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b3820 con 0x561db84b3f40 2024-04-23T18:52:09.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e82471740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b5360 con 0x561db84be800 2024-04-23T18:52:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59572/0 (socket says 172.21.15.33:59572) 2024-04-23T18:52:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4161808088 learned_addr learned my addr 172.21.15.33:0/4161808088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:10.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4161808088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84be800 msgr2=0x561db84b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:10.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84be800 0x561db84b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4161808088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 msgr2=0x561db84ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:10.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4161808088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561db84b3d70 con 0x561db84b3620 2024-04-23T18:52:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:52:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.593+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009530 tx=0x7f3e68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e912d3aa01c4ee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:10.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e62ffd640 1 -- 172.21.15.33:0/4161808088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680054b0 con 0x561db84b3620 2024-04-23T18:52:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e62ffd640 1 -- 172.21.15.33:0/4161808088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e680044f0 con 0x561db84b3620 2024-04-23T18:52:10.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e62ffd640 1 -- 172.21.15.33:0/4161808088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004920 con 0x561db84b3620 2024-04-23T18:52:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 -- 172.21.15.33:0/4161808088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 msgr2=0x561db84ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009530 tx=0x7f3e68004730 comp rx=0 tx=0).stop 2024-04-23T18:52:10.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 -- 172.21.15.33:0/4161808088 shutdown_connections 2024-04-23T18:52:10.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84be800 0x561db84b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 --2- 172.21.15.33:0/4161808088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84b3620 0x561db84ca2f0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 -- 172.21.15.33:0/4161808088 >> 172.21.15.33:0/4161808088 conn(0x561db84afb30 msgr2=0x561db84b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:11.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 -- 172.21.15.33:0/4161808088 shutdown_connections 2024-04-23T18:52:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.594+0000 7f3e82471740 1 -- 172.21.15.33:0/4161808088 wait complete. 2024-04-23T18:52:11.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 -- start start 2024-04-23T18:52:11.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 Processor -- start 2024-04-23T18:52:11.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:11.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b45c0 con 0x561db84be800 2024-04-23T18:52:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b3ef0 con 0x561db84b3f40 2024-04-23T18:52:11.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781b4c90 con 0x561db84b3620 2024-04-23T18:52:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e82471740 10 client.0 _ll_register_callbacks cb 0x7f3e74004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e7ccf7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59588/0 (socket says 172.21.15.33:59588) 2024-04-23T18:52:11.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4156404472 learned_addr learned my addr 172.21.15.33:0/4156404472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.595+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4156404472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 msgr2=0x561db84c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:11.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:11.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 msgr2=0x561db84b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:11.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 -- 172.21.15.33:0/4156404472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561db84b87c0 con 0x561db84be800 2024-04-23T18:52:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e7ccf7640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009fd0 tx=0x7f3e68009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdef587f017bf570 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:11.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:11.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004240 con 0x561db84be800 2024-04-23T18:52:11.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f3e68004aa0 con 0x561db84be800 2024-04-23T18:52:11.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e680044f0 con 0x561db84be800 2024-04-23T18:52:11.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3e68004ca0 con 0x561db84be800 2024-04-23T18:52:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e82471740 10 client.18114 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:12.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561db8534b30 con 0x561db84be800 2024-04-23T18:52:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e680044f0 con 0x561db84be800 2024-04-23T18:52:12.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.596+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f3e6802ece0 con 0x561db84be800 2024-04-23T18:52:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 10 client.18114 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 10 client.18114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 20 client.18114 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db8534eb0 0x561db85372d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 4 client.18114 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:52:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561db8420200 con 0x561db8534eb0 2024-04-23T18:52:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db8534eb0 0x561db85372d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e60ff9640 10 client.18114 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.597+0000 7f3e7d4f8640 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db8534eb0 0x561db85372d0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6567d897768f2f server_cookie=d2347c069ea6957c in_seq=0 out_seq=0 2024-04-23T18:52:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.601+0000 7f3e60ff9640 1 -- 172.21.15.33:0/4156404472 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561db8420200 con 0x561db8534eb0 2024-04-23T18:52:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.601+0000 7f3e60ff9640 10 client.18114 handle_command_reply: tid=0 2024-04-23T18:52:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.601+0000 7f3e82471740 1 client.18114 shutdown 2024-04-23T18:52:12.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.601+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 msgr2=0x561db84b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009fd0 tx=0x7f3e68009c10 comp rx=0 tx=0).stop 2024-04-23T18:52:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 shutdown_connections 2024-04-23T18:52:12.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x561db8534eb0 0x561db85372d0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:12.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561db84be800 0x561db84b8280 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:12.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561db84b3f40 0x561db84b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e60ff9640 0 client.18114 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:12.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 --2- 172.21.15.33:0/4156404472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561db84b3620 0x561db84c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 >> 172.21.15.33:0/4156404472 conn(0x561db84afb30 msgr2=0x561db84b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 shutdown_connections 2024-04-23T18:52:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 1 -- 172.21.15.33:0/4156404472 wait complete. 2024-04-23T18:52:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e82471740 20 client.18114 trim_cache size 0 max 16384 2024-04-23T18:52:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:52:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:52:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:52:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:52:13.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 msgr2=0x7f3e640a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:13.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 0x7f3e640a6300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3e700ee490 tx=0x7f3e7008e8d0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 msgr2=0x7f3e7819ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:13.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009df0 tx=0x7f3e700093d0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 shutdown_connections 2024-04-23T18:52:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3e640a3e40 0x7f3e640a6300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e781063f0 0x7f3e7819ea70 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78101910 0x7f3e781a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 --2- 172.21.15.33:0/3577459887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e78100ff0 0x7f3e781a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 >> 172.21.15.33:0/3577459887 conn(0x7f3e780771d0 msgr2=0x7f3e78103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.602+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 shutdown_connections 2024-04-23T18:52:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:42.603+0000 7f3e7f75c640 1 -- 172.21.15.33:0/3577459887 wait complete. 2024-04-23T18:52:13.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:43 smithi033 ceph-mon[29274]: pgmap v780: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:13.957 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:45 smithi033 ceph-mon[29274]: pgmap v781: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:13.964 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:47 smithi033 ceph-mon[29274]: pgmap v782: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:13.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:49 smithi033 ceph-mon[29274]: pgmap v783: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:14.049 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:51 smithi033 ceph-mon[29274]: pgmap v784: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:14.097 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:53 smithi033 ceph-mon[29274]: pgmap v785: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:14.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:55 smithi033 ceph-mon[29274]: pgmap v786: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:14.154 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:03 smithi122 ceph-mon[32546]: pgmap v823: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:14.373 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:03 smithi122 ceph-mon[32546]: pgmap v824: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:52:14.454 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:04 smithi122 ceph-mon[32546]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:52:14.462 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:05 smithi122 ceph-mon[32546]: pgmap v826: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:52:14.471 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:07 smithi122 ceph-mon[32546]: pgmap v827: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:52:14.541 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:09 smithi122 ceph-mon[32546]: pgmap v828: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:52:14.550 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:11 smithi122 ceph-mon[32546]: pgmap v829: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:52:14.641 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:11 smithi122 ceph-mon[32546]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:52:14.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:13 smithi122 ceph-mon[32546]: pgmap v831: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:52:14.766 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:03 smithi106 ceph-mon[32716]: pgmap v823: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:14.883 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:03 smithi106 ceph-mon[32716]: pgmap v824: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:52:14.979 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:04 smithi106 ceph-mon[32716]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:52:15.051 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:05 smithi106 ceph-mon[32716]: pgmap v826: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:52:15.060 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:07 smithi106 ceph-mon[32716]: pgmap v827: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:52:15.069 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:09 smithi106 ceph-mon[32716]: pgmap v828: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:52:15.106 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:11 smithi106 ceph-mon[32716]: pgmap v829: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:52:15.130 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:11 smithi106 ceph-mon[32716]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:52:15.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:13 smithi106 ceph-mon[32716]: pgmap v831: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:52:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 -- 172.21.15.33:0/3010399157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c1021d0 msgr2=0x7f660c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:15.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 --2- 172.21.15.33:0/3010399157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c1021d0 0x7f660c1025b0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f65fc009ca0 tx=0x7f65fc02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:52:15.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 -- 172.21.15.33:0/3010399157 shutdown_connections 2024-04-23T18:52:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 --2- 172.21.15.33:0/3010399157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1072a0 0x7f660c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 --2- 172.21.15.33:0/3010399157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c102b80 0x7f660c106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:15.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 --2- 172.21.15.33:0/3010399157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c1021d0 0x7f660c1025b0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 -- 172.21.15.33:0/3010399157 >> 172.21.15.33:0/3010399157 conn(0x7f660c0793e0 msgr2=0x7f660c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 -- 172.21.15.33:0/3010399157 shutdown_connections 2024-04-23T18:52:15.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 -- 172.21.15.33:0/3010399157 wait complete. 2024-04-23T18:52:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.704+0000 7f6612035640 1 Processor -- start 2024-04-23T18:52:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 -- start start 2024-04-23T18:52:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 0x7f660c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 0x7f660c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1121e0 con 0x7f660c074cd0 2024-04-23T18:52:15.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f660c112060 con 0x7f660c1072a0 2024-04-23T18:52:15.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f6612035640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f660c112360 con 0x7f660c1021d0 2024-04-23T18:52:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:15.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60274/0 (socket says 172.21.15.33:60274) 2024-04-23T18:52:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 -- 172.21.15.33:0/965145241 learned_addr learned my addr 172.21.15.33:0/965145241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660b7fe640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 0x7f660c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660affd640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 0x7f660c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 -- 172.21.15.33:0/965145241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 msgr2=0x7f660c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 0x7f660c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 -- 172.21.15.33:0/965145241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 msgr2=0x7f660c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:16.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 0x7f660c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660bfff640 1 -- 172.21.15.33:0/965145241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c1a6f70 con 0x7f660c074cd0 2024-04-23T18:52:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660b7fe640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 0x7f660c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.705+0000 7f660affd640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 0x7f660c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:16.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f660bfff640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f660400d700 tx=0x7f660400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:16.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f6608ff9640 1 -- 172.21.15.33:0/965145241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604017030 con 0x7f660c074cd0 2024-04-23T18:52:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f6608ff9640 1 -- 172.21.15.33:0/965145241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6604009820 con 0x7f660c074cd0 2024-04-23T18:52:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f6608ff9640 1 -- 172.21.15.33:0/965145241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6604003ac0 con 0x7f660c074cd0 2024-04-23T18:52:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c1a7260 con 0x7f660c074cd0 2024-04-23T18:52:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.706+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c1a7760 con 0x7f660c074cd0 2024-04-23T18:52:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.708+0000 7f6608ff9640 1 -- 172.21.15.33:0/965145241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f660401a070 con 0x7f660c074cd0 2024-04-23T18:52:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 Processor -- start 2024-04-23T18:52:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 -- start start 2024-04-23T18:52:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b402332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:17.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40233800 0x556b40235bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b4018d970 con 0x556b40228620 2024-04-23T18:52:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556b4012a4a0 con 0x556b40233800 2024-04-23T18:52:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6614e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b40189930 con 0x556b40228f40 2024-04-23T18:52:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60278/0 (socket says 172.21.15.33:60278) 2024-04-23T18:52:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 -- 172.21.15.33:0/4629039 learned_addr learned my addr 172.21.15.33:0/4629039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:17.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6608ff9640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 0x7f65f40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:17.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660bfff640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 0x7f65f40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:17.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660b7fe640 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b402332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:17.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40233800 0x556b40235bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:17.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f6608ff9640 1 -- 172.21.15.33:0/965145241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f66040cff80 con 0x7f660c074cd0 2024-04-23T18:52:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 -- 172.21.15.33:0/4629039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 msgr2=0x556b402332c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:17.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b402332c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 -- 172.21.15.33:0/4629039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40233800 msgr2=0x556b40235bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:17.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40233800 0x556b40235bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:17.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.709+0000 7f660affd640 1 -- 172.21.15.33:0/4629039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3fea0210 con 0x556b40228620 2024-04-23T18:52:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f660affd640 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f66000135a0 tx=0x7f6600009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747b764cd93b58fc server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:17.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f660bfff640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 0x7f65f40a62b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f660409bbd0 tx=0x7f660406b7b0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f65f1ffb640 1 -- 172.21.15.33:0/4629039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600011470 con 0x556b40228620 2024-04-23T18:52:18.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f65f1ffb640 1 -- 172.21.15.33:0/4629039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6600011ac0 con 0x556b40228620 2024-04-23T18:52:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f65f1ffb640 1 -- 172.21.15.33:0/4629039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000f940 con 0x556b40228620 2024-04-23T18:52:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f6614e2e740 1 -- 172.21.15.33:0/4629039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 msgr2=0x556b40228a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.710+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f66000135a0 tx=0x7f6600009790 comp rx=0 tx=0).stop 2024-04-23T18:52:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- 172.21.15.33:0/4629039 shutdown_connections 2024-04-23T18:52:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40233800 0x556b40235bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b402332c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4629039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40228620 0x556b40228a00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- 172.21.15.33:0/4629039 >> 172.21.15.33:0/4629039 conn(0x556b40224b30 msgr2=0x556b40224f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- 172.21.15.33:0/4629039 shutdown_connections 2024-04-23T18:52:19.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- 172.21.15.33:0/4629039 wait complete. 2024-04-23T18:52:19.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- start start 2024-04-23T18:52:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 Processor -- start 2024-04-23T18:52:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b402331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:19.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40230490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.711+0000 7f6614e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b40236430 con 0x556b40233800 2024-04-23T18:52:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f6614e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556b401280b0 con 0x556b40228f40 2024-04-23T18:52:19.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f6614e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b402365b0 con 0x556b40228620 2024-04-23T18:52:19.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f6614e2e740 10 client.0 _ll_register_callbacks cb 0x556b40115d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:19.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60282/0 (socket says 172.21.15.33:60282) 2024-04-23T18:52:19.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 -- 172.21.15.33:0/4233584584 learned_addr learned my addr 172.21.15.33:0/4233584584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660b7fe640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40230490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b402331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 -- 172.21.15.33:0/4233584584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 msgr2=0x556b402331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b402331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 -- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 msgr2=0x556b40230490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40230490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 -- 172.21.15.33:0/4233584584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3ff6c620 con 0x556b40233800 2024-04-23T18:52:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f660affd640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6600011f50 tx=0x7f660000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389f502fe4167b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f65eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600011860 con 0x556b40233800 2024-04-23T18:52:20.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f6600004050 con 0x556b40233800 2024-04-23T18:52:20.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.712+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6600011ac0 con 0x556b40233800 2024-04-23T18:52:20.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6600004250 con 0x556b40233800 2024-04-23T18:52:20.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 10 client.18126 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556b402423a0 con 0x556b40233800 2024-04-23T18:52:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6600011ac0 con 0x556b40233800 2024-04-23T18:52:20.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f660002c030 con 0x556b40233800 2024-04-23T18:52:20.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 10 client.18126 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:20.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 10 client.18126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:20.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 20 client.18126 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402436f0 0x556b40245ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 4 client.18126 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:52:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4019b390 con 0x556b402436f0 2024-04-23T18:52:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.713+0000 7f660b7fe640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402436f0 0x556b40245ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:20.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.714+0000 7f65eaffd640 10 client.18126 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:20.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.714+0000 7f660b7fe640 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402436f0 0x556b40245ae0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11e44e77005c869 server_cookie=ce25fad10657de0 in_seq=0 out_seq=0 2024-04-23T18:52:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.714+0000 7f65eaffd640 1 -- 172.21.15.33:0/4233584584 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b4019b390 con 0x556b402436f0 2024-04-23T18:52:21.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.714+0000 7f65eaffd640 10 client.18126 handle_command_reply: tid=0 2024-04-23T18:52:21.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.714+0000 7f6614e2e740 1 client.18126 shutdown 2024-04-23T18:52:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 msgr2=0x556b40230a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:21.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6600011f50 tx=0x7f660000f5e0 comp rx=0 tx=0).stop 2024-04-23T18:52:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 shutdown_connections 2024-04-23T18:52:21.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402436f0 0x556b40245ae0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40230a30 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:21.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40230490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f65eaffd640 0 client.18126 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 --2- 172.21.15.33:0/4233584584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b402331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:21.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 >> 172.21.15.33:0/4233584584 conn(0x556b40224b30 msgr2=0x556b4022a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 shutdown_connections 2024-04-23T18:52:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 1 -- 172.21.15.33:0/4233584584 wait complete. 2024-04-23T18:52:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.715+0000 7f6614e2e740 20 client.18126 trim_cache size 0 max 16384 2024-04-23T18:52:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 Processor -- start 2024-04-23T18:52:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 -- start start 2024-04-23T18:52:21.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228620 0x556b4022fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b4022b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1b1780 con 0x556b40233800 2024-04-23T18:52:21.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1af570 con 0x556b40228620 2024-04-23T18:52:22.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f6614e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1b10b0 con 0x556b40228f40 2024-04-23T18:52:22.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:22.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60290/0 (socket says 172.21.15.33:60290) 2024-04-23T18:52:22.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f660affd640 1 -- 172.21.15.33:0/642453834 learned_addr learned my addr 172.21.15.33:0/642453834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.730+0000 7f660affd640 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228620 0x556b4022fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660b7fe640 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b4022b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:22.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 -- 172.21.15.33:0/642453834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 msgr2=0x556b4022b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:22.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b4022b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 -- 172.21.15.33:0/642453834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228620 msgr2=0x556b4022fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:22.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228620 0x556b4022fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 -- 172.21.15.33:0/642453834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4023fe20 con 0x556b40233800 2024-04-23T18:52:22.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f660affd640 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6600004c10 tx=0x7f66000043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c64990a0f66b671 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f65f17fa640 1 -- 172.21.15.33:0/642453834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600011d20 con 0x556b40233800 2024-04-23T18:52:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f65f17fa640 1 -- 172.21.15.33:0/642453834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6600004c90 con 0x556b40233800 2024-04-23T18:52:22.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.731+0000 7f65f17fa640 1 -- 172.21.15.33:0/642453834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000044a0 con 0x556b40233800 2024-04-23T18:52:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- 172.21.15.33:0/642453834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 msgr2=0x556b40232cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6600004c10 tx=0x7f66000043f0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- 172.21.15.33:0/642453834 shutdown_connections 2024-04-23T18:52:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40232cd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228f40 0x556b4022b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- 172.21.15.33:0/642453834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228620 0x556b4022fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:22.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- 172.21.15.33:0/642453834 >> 172.21.15.33:0/642453834 conn(0x556b40224b30 msgr2=0x556b40223cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:22.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- 172.21.15.33:0/642453834 shutdown_connections 2024-04-23T18:52:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- 172.21.15.33:0/642453834 wait complete. 2024-04-23T18:52:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- start start 2024-04-23T18:52:23.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 Processor -- start 2024-04-23T18:52:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b4023bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40236ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:23.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1b0310 con 0x556b40233800 2024-04-23T18:52:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1afc40 con 0x556b40228f40 2024-04-23T18:52:23.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.732+0000 7f6614e2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f660c1b09e0 con 0x556b40228620 2024-04-23T18:52:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f6614e2e740 10 client.0 _ll_register_callbacks cb 0x7f6600011470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:23.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:23.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60296/0 (socket says 172.21.15.33:60296) 2024-04-23T18:52:23.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40236ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 -- 172.21.15.33:0/3176985321 learned_addr learned my addr 172.21.15.33:0/3176985321 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b4023bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:23.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 -- 172.21.15.33:0/3176985321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 msgr2=0x556b4023bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:23.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b4023bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:23.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 -- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 msgr2=0x556b40236ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:23.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40236ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:23.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 -- 172.21.15.33:0/3176985321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b40237c20 con 0x556b40233800 2024-04-23T18:52:23.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660b7fe640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40236ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:52:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f660affd640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6600004a60 tx=0x7f6600011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69c5aa82b99eb3f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f65eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:23.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000046b0 con 0x556b40233800 2024-04-23T18:52:23.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7f6600004fb0 con 0x556b40233800 2024-04-23T18:52:23.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.733+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6600004c90 con 0x556b40233800 2024-04-23T18:52:23.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6600048050 con 0x556b40233800 2024-04-23T18:52:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 10 client.18132 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:24.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x556b402a99c0 con 0x556b40233800 2024-04-23T18:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6600004c90 con 0x556b40233800 2024-04-23T18:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f660003f070 con 0x556b40233800 2024-04-23T18:52:24.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 10 client.18132 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:24.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:24.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 20 client.18132 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402aa140 0x556b402ac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 4 client.18132 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:52:24.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b40195200 con 0x556b402aa140 2024-04-23T18:52:24.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.734+0000 7f660b7fe640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402aa140 0x556b402ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:24.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.735+0000 7f65eaffd640 10 client.18132 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:24.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.735+0000 7f660b7fe640 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402aa140 0x556b402ac560 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f9b9ea1a56490f server_cookie=6011a315cb8ecf80 in_seq=0 out_seq=0 2024-04-23T18:52:24.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f65eaffd640 1 -- 172.21.15.33:0/3176985321 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b40195200 con 0x556b402aa140 2024-04-23T18:52:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f65eaffd640 10 client.18132 handle_command_reply: tid=0 2024-04-23T18:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 client.18132 shutdown 2024-04-23T18:52:24.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 msgr2=0x556b40237420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f6600004a60 tx=0x7f6600011520 comp rx=0 tx=0).stop 2024-04-23T18:52:24.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 shutdown_connections 2024-04-23T18:52:24.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556b402aa140 0x556b402ac560 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:24.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b40233800 0x556b40237420 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b40228f40 0x556b40236ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f65eaffd640 0 client.18132 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 --2- 172.21.15.33:0/3176985321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b40228620 0x556b4023bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 >> 172.21.15.33:0/3176985321 conn(0x556b40224b30 msgr2=0x556b4023e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 shutdown_connections 2024-04-23T18:52:25.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 1 -- 172.21.15.33:0/3176985321 wait complete. 2024-04-23T18:52:25.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.739+0000 7f6614e2e740 20 client.18132 trim_cache size 0 max 16384 2024-04-23T18:52:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:52:25.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:52:25.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:52:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:52:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 msgr2=0x7f65f40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 0x7f65f40a62b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f660409bbd0 tx=0x7f660406b7b0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 msgr2=0x7f660c1a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f660400d700 tx=0x7f660400dbc0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 shutdown_connections 2024-04-23T18:52:25.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f65f40a3df0 0x7f65f40a62b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f660c074cd0 0x7f660c1a6870 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f660c1072a0 0x7f660c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 --2- 172.21.15.33:0/965145241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f660c1021d0 0x7f660c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:25.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 >> 172.21.15.33:0/965145241 conn(0x7f660c0793e0 msgr2=0x7f660c0ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:25.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 shutdown_connections 2024-04-23T18:52:25.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:50:55.740+0000 7f6612035640 1 -- 172.21.15.33:0/965145241 wait complete. 2024-04-23T18:52:25.837 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:57 smithi033 ceph-mon[29274]: pgmap v787: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:25.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:50:59 smithi033 ceph-mon[29274]: pgmap v788: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:26.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:01 smithi033 ceph-mon[29274]: pgmap v789: 97 pgs: 97 active+clean; 15 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:26.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:01 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:52:26.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:01 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:52:26.116 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:01 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:52:26.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:01 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:52:26.135 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:02 smithi033 ceph-mon[29274]: pgmap v790: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 49 KiB/s wr, 33 op/s 2024-04-23T18:52:26.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:04 smithi033 ceph-mon[29274]: pgmap v791: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:26.197 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:06 smithi033 ceph-mon[29274]: pgmap v792: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:26.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:08 smithi033 ceph-mon[29274]: pgmap v793: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:26.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:15 smithi106 ceph-mon[32716]: pgmap v832: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:52:26.496 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:16 smithi106 ceph-mon[32716]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-23T18:52:26.507 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:52:26.544 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:17 smithi106 ceph-mon[32716]: pgmap v833: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:52:26.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:19 smithi106 ceph-mon[32716]: pgmap v834: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:52:26.560 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:21 smithi106 ceph-mon[32716]: pgmap v835: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:52:26.616 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:23 smithi106 ceph-mon[32716]: pgmap v836: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 9 op/s 2024-04-23T18:52:26.625 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:25 smithi106 ceph-mon[32716]: pgmap v837: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:26.634 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:15 smithi122 ceph-mon[32546]: pgmap v832: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:52:26.811 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:16 smithi122 ceph-mon[32546]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-23T18:52:26.865 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:52:26.875 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:17 smithi122 ceph-mon[32546]: pgmap v833: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:52:26.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:19 smithi122 ceph-mon[32546]: pgmap v834: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:52:26.974 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:21 smithi122 ceph-mon[32546]: pgmap v835: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:52:27.075 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:23 smithi122 ceph-mon[32546]: pgmap v836: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 9 op/s 2024-04-23T18:52:27.117 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:25 smithi122 ceph-mon[32546]: pgmap v837: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 -- 172.21.15.33:0/3304774192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 msgr2=0x7fd7a8101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 --2- 172.21.15.33:0/3304774192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a8101160 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd798009ca0 tx=0x7fd79802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:52:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 -- 172.21.15.33:0/3304774192 shutdown_connections 2024-04-23T18:52:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 --2- 172.21.15.33:0/3304774192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a810e930 0x7fd7a8110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 --2- 172.21.15.33:0/3304774192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a81016a0 0x7fd7a810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 --2- 172.21.15.33:0/3304774192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a8101160 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 -- 172.21.15.33:0/3304774192 >> 172.21.15.33:0/3304774192 conn(0x7fd7a8077fd0 msgr2=0x7fd7a80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:27.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 -- 172.21.15.33:0/3304774192 shutdown_connections 2024-04-23T18:52:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.841+0000 7fd7ad329640 1 -- 172.21.15.33:0/3304774192 wait complete. 2024-04-23T18:52:27.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 Processor -- start 2024-04-23T18:52:27.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 -- start start 2024-04-23T18:52:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:27.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a81016a0 0x7fd7a81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a810e930 0x7fd7a819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a8114430 con 0x7fd7a8100d80 2024-04-23T18:52:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81142b0 con 0x7fd7a81016a0 2024-04-23T18:52:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7ad329640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81145b0 con 0x7fd7a810e930 2024-04-23T18:52:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:28.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48242/0 (socket says 172.21.15.33:48242) 2024-04-23T18:52:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.842+0000 7fd7a6ffd640 1 -- 172.21.15.33:0/2229743966 learned_addr learned my addr 172.21.15.33:0/2229743966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a810e930 0x7fd7a819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a81016a0 0x7fd7a81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 -- 172.21.15.33:0/2229743966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a810e930 msgr2=0x7fd7a819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a810e930 0x7fd7a819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 -- 172.21.15.33:0/2229743966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a81016a0 msgr2=0x7fd7a81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a81016a0 0x7fd7a81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 -- 172.21.15.33:0/2229743966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a819b030 con 0x7fd7a8100d80 2024-04-23T18:52:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd798005e30 tx=0x7fd798031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd78ffff640 1 -- 172.21.15.33:0/2229743966 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798003b70 con 0x7fd7a8100d80 2024-04-23T18:52:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd78ffff640 1 -- 172.21.15.33:0/2229743966 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd798003d10 con 0x7fd7a8100d80 2024-04-23T18:52:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a819b2c0 con 0x7fd7a8100d80 2024-04-23T18:52:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd78ffff640 1 -- 172.21.15.33:0/2229743966 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79802fa50 con 0x7fd7a8100d80 2024-04-23T18:52:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.843+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a81a7630 con 0x7fd7a8100d80 2024-04-23T18:52:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.845+0000 7fd78ffff640 1 -- 172.21.15.33:0/2229743966 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd79802fbf0 con 0x7fd7a8100d80 2024-04-23T18:52:28.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 Processor -- start 2024-04-23T18:52:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 -- start start 2024-04-23T18:52:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ebba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:29.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a579e20970 con 0x55a579ec6800 2024-04-23T18:52:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a579dbd4a0 con 0x55a579ebbf40 2024-04-23T18:52:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7b0042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a579e1c930 con 0x55a579ebb620 2024-04-23T18:52:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48258/0 (socket says 172.21.15.33:48258) 2024-04-23T18:52:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a67fc640 1 -- 172.21.15.33:0/1076590971 learned_addr learned my addr 172.21.15.33:0/1076590971 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd78ffff640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 0x7fd7900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ebba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.846+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 0x7fd7900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:29.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 -- 172.21.15.33:0/1076590971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 msgr2=0x55a579ebba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:29.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ebba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:29.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 -- 172.21.15.33:0/1076590971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 msgr2=0x55a579ec62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 -- 172.21.15.33:0/1076590971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a579b33210 con 0x55a579ec6800 2024-04-23T18:52:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd78ffff640 1 -- 172.21.15.33:0/2229743966 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fd7980f1b00 con 0x7fd7a8100d80 2024-04-23T18:52:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009c30 tx=0x7fd79c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813ff57a171cb363 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:29.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 0x7fd7900a6300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0005fd0 tx=0x7fd7a0005c50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:29.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd78d7fa640 1 -- 172.21.15.33:0/1076590971 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c012450 con 0x55a579ec6800 2024-04-23T18:52:30.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd78d7fa640 1 -- 172.21.15.33:0/1076590971 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c012aa0 con 0x55a579ec6800 2024-04-23T18:52:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.847+0000 7fd78d7fa640 1 -- 172.21.15.33:0/1076590971 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00fae0 con 0x55a579ec6800 2024-04-23T18:52:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- 172.21.15.33:0/1076590971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 msgr2=0x55a579ec8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:30.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009c30 tx=0x7fd79c0117b0 comp rx=0 tx=0).stop 2024-04-23T18:52:30.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- 172.21.15.33:0/1076590971 shutdown_connections 2024-04-23T18:52:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec8bf0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- 172.21.15.33:0/1076590971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ebba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- 172.21.15.33:0/1076590971 >> 172.21.15.33:0/1076590971 conn(0x55a579eb7b30 msgr2=0x55a579eb7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:30.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- 172.21.15.33:0/1076590971 shutdown_connections 2024-04-23T18:52:30.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- 172.21.15.33:0/1076590971 wait complete. 2024-04-23T18:52:30.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- start start 2024-04-23T18:52:30.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 Processor -- start 2024-04-23T18:52:30.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ec6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a579ec9430 con 0x55a579ec6800 2024-04-23T18:52:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a579dbb0b0 con 0x55a579ebbf40 2024-04-23T18:52:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.848+0000 7fd7b0042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a579ec95b0 con 0x55a579ebb620 2024-04-23T18:52:31.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7b0042740 10 client.0 _ll_register_callbacks cb 0x55a579da8d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:31.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48268/0 (socket says 172.21.15.33:48268) 2024-04-23T18:52:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ec6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 -- 172.21.15.33:0/2010569031 learned_addr learned my addr 172.21.15.33:0/2010569031 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 -- 172.21.15.33:0/2010569031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 msgr2=0x55a579ec6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:31.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ec6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 -- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 msgr2=0x55a579ec3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:31.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 -- 172.21.15.33:0/2010569031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a579bff620 con 0x55a579ec6800 2024-04-23T18:52:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:31.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0004630 tx=0x7fd7a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db762ba4e884922 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:31.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd78aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0022070 con 0x55a579ec6800 2024-04-23T18:52:31.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fd7a00119a0 con 0x55a579ec6800 2024-04-23T18:52:31.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79c012aa0 con 0x55a579ec6800 2024-04-23T18:52:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.849+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd7a002e050 con 0x55a579ec6800 2024-04-23T18:52:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 10 client.18144 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a579ed53b0 con 0x55a579ec6800 2024-04-23T18:52:32.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c012aa0 con 0x55a579ec6800 2024-04-23T18:52:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7fd7a001a070 con 0x55a579ec6800 2024-04-23T18:52:32.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 10 client.18144 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 10 client.18144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 20 client.18144 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579ed6870 0x55a579ed8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 4 client.18144 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:52:32.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a579e2e390 con 0x55a579ed6870 2024-04-23T18:52:32.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.850+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579ed6870 0x55a579ed8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.851+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579ed6870 0x55a579ed8c60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6590c33a7363ef server_cookie=67467498e809b168 in_seq=0 out_seq=0 2024-04-23T18:52:32.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.851+0000 7fd78aff5640 10 client.18144 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.851+0000 7fd78aff5640 1 -- 172.21.15.33:0/2010569031 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a579e2e390 con 0x55a579ed6870 2024-04-23T18:52:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.851+0000 7fd78aff5640 10 client.18144 handle_command_reply: tid=0 2024-04-23T18:52:33.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 client.18144 shutdown 2024-04-23T18:52:33.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 msgr2=0x55a579ec3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0004630 tx=0x7fd7a00046c0 comp rx=0 tx=0).stop 2024-04-23T18:52:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 shutdown_connections 2024-04-23T18:52:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579ed6870 0x55a579ed8c60 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ec3950 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebbf40 0x55a579ec3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 --2- 172.21.15.33:0/2010569031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579ec6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:33.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd78aff5640 0 client.18144 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 >> 172.21.15.33:0/2010569031 conn(0x55a579eb7b30 msgr2=0x55a579ebd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 shutdown_connections 2024-04-23T18:52:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 1 -- 172.21.15.33:0/2010569031 wait complete. 2024-04-23T18:52:33.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.852+0000 7fd7b0042740 20 client.18144 trim_cache size 0 max 16384 2024-04-23T18:52:33.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 Processor -- start 2024-04-23T18:52:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 -- start start 2024-04-23T18:52:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 0x55a579ed22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:34.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebbf40 0x55a579eb6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:34.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81b1840 con 0x55a579ec6800 2024-04-23T18:52:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81af630 con 0x55a579ebb620 2024-04-23T18:52:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7b0042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81b1170 con 0x55a579ebbf40 2024-04-23T18:52:34.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48278/0 (socket says 172.21.15.33:48278) 2024-04-23T18:52:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.867+0000 7fd7a77fe640 1 -- 172.21.15.33:0/444094350 learned_addr learned my addr 172.21.15.33:0/444094350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 0x55a579ed22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebbf40 0x55a579eb6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 -- 172.21.15.33:0/444094350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebbf40 msgr2=0x55a579eb6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:34.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebbf40 0x55a579eb6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:34.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 -- 172.21.15.33:0/444094350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 msgr2=0x55a579ed22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 0x55a579ed22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 -- 172.21.15.33:0/444094350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a579ebefe0 con 0x55a579ec6800 2024-04-23T18:52:34.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 0x55a579ed22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:52:34.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0003a30 tx=0x7fd7a0026540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bb39b414a98ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd78cff9640 1 -- 172.21.15.33:0/444094350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0036030 con 0x55a579ec6800 2024-04-23T18:52:35.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd78cff9640 1 -- 172.21.15.33:0/444094350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a000e710 con 0x55a579ec6800 2024-04-23T18:52:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd78cff9640 1 -- 172.21.15.33:0/444094350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0022070 con 0x55a579ec6800 2024-04-23T18:52:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7b0042740 1 -- 172.21.15.33:0/444094350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 msgr2=0x55a579ede570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.868+0000 7fd7b0042740 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0003a30 tx=0x7fd7a0026540 comp rx=0 tx=0).stop 2024-04-23T18:52:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- 172.21.15.33:0/444094350 shutdown_connections 2024-04-23T18:52:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ec6800 0x55a579ede570 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:35.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebbf40 0x55a579eb6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- 172.21.15.33:0/444094350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ebb620 0x55a579ed22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:35.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- 172.21.15.33:0/444094350 >> 172.21.15.33:0/444094350 conn(0x55a579eb7b30 msgr2=0x55a579eb6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- 172.21.15.33:0/444094350 shutdown_connections 2024-04-23T18:52:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- 172.21.15.33:0/444094350 wait complete. 2024-04-23T18:52:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- start start 2024-04-23T18:52:35.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 Processor -- start 2024-04-23T18:52:35.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579eceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:35.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:35.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ec6800 0x55a579ecc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:35.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81b03d0 con 0x55a579ebbf40 2024-04-23T18:52:35.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81afd00 con 0x55a579ec6800 2024-04-23T18:52:35.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a81b0aa0 con 0x55a579ebb620 2024-04-23T18:52:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7b0042740 10 client.0 _ll_register_callbacks cb 0x7fd7a000eee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48288/0 (socket says 172.21.15.33:48288) 2024-04-23T18:52:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7a67fc640 1 -- 172.21.15.33:0/3470969142 learned_addr learned my addr 172.21.15.33:0/3470969142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:36.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.869+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579eceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a77fe640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ec6800 0x55a579ecc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 -- 172.21.15.33:0/3470969142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 msgr2=0x55a579eceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579eceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:36.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 -- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ec6800 msgr2=0x55a579ecc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ec6800 0x55a579ecc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 -- 172.21.15.33:0/3470969142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a579eccaf0 con 0x55a579ebbf40 2024-04-23T18:52:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7a67fc640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd79c011cb0 tx=0x7fd79c0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84292a3744b6bab server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd78aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:36.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00f850 con 0x55a579ebbf40 2024-04-23T18:52:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(134..134 src has 1..134) ==== 8498+0+0 (secure 0 0 0) 0x7fd79c00f9f0 con 0x55a579ebbf40 2024-04-23T18:52:36.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a579ec3ce0 con 0x55a579ebbf40 2024-04-23T18:52:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd79c035050 con 0x55a579ebbf40 2024-04-23T18:52:37.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7b0042740 10 client.18150 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:37.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a579f3cca0 con 0x55a579ebbf40 2024-04-23T18:52:37.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.870+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a579ec3ce0 con 0x55a579ebbf40 2024-04-23T18:52:37.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7fd79c012450 con 0x55a579ebbf40 2024-04-23T18:52:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 10 client.18150 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:37.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 10 client.18150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 20 client.18150 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579f3d220 0x55a579f3f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:37.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 4 client.18150 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:52:37.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a579e28200 con 0x55a579f3d220 2024-04-23T18:52:37.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579f3d220 0x55a579f3f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd78aff5640 10 client.18150 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:37.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.871+0000 7fd7a6ffd640 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579f3d220 0x55a579f3f640 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a99eb3513f2630 server_cookie=da42db6e4da069d0 in_seq=0 out_seq=0 2024-04-23T18:52:37.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd78aff5640 1 -- 172.21.15.33:0/3470969142 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a579e28200 con 0x55a579f3d220 2024-04-23T18:52:37.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd78aff5640 10 client.18150 handle_command_reply: tid=0 2024-04-23T18:52:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 client.18150 shutdown 2024-04-23T18:52:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 msgr2=0x55a579ecbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fd79c011cb0 tx=0x7fd79c0053b0 comp rx=0 tx=0).stop 2024-04-23T18:52:37.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 shutdown_connections 2024-04-23T18:52:37.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a579f3d220 0x55a579f3f640 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:37.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a579ec6800 0x55a579ecc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a579ebbf40 0x55a579ecbe70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd78aff5640 0 client.18150 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 --2- 172.21.15.33:0/3470969142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a579ebb620 0x55a579eceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:37.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 >> 172.21.15.33:0/3470969142 conn(0x55a579eb7b30 msgr2=0x55a579eb6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 shutdown_connections 2024-04-23T18:52:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 1 -- 172.21.15.33:0/3470969142 wait complete. 2024-04-23T18:52:38.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.878+0000 7fd7b0042740 20 client.18150 trim_cache size 0 max 16384 2024-04-23T18:52:38.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:52:38.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:52:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:52:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:52:38.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 msgr2=0x7fd7900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:38.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 0x7fd7900a6300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0005fd0 tx=0x7fd7a0005c50 comp rx=0 tx=0).stop 2024-04-23T18:52:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 msgr2=0x7fd7a81a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:38.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd798005e30 tx=0x7fd798031840 comp rx=0 tx=0).stop 2024-04-23T18:52:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 shutdown_connections 2024-04-23T18:52:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd7900a3e40 0x7fd7900a6300 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a810e930 0x7fd7a819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7a81016a0 0x7fd7a81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 --2- 172.21.15.33:0/2229743966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a8100d80 0x7fd7a81a06b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 >> 172.21.15.33:0/2229743966 conn(0x7fd7a8077fd0 msgr2=0x7fd7a810fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 shutdown_connections 2024-04-23T18:52:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:08.879+0000 7fd7ad329640 1 -- 172.21.15.33:0/2229743966 wait complete. 2024-04-23T18:52:39.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:10 smithi033 ceph-mon[29274]: pgmap v794: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:52:39.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:10 smithi033 ceph-mon[29274]: osdmap e135: 12 total, 12 up, 12 in 2024-04-23T18:52:39.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:12 smithi033 ceph-mon[29274]: pgmap v796: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:52:39.509 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:14 smithi033 ceph-mon[29274]: pgmap v797: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:52:39.517 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:16 smithi033 ceph-mon[29274]: pgmap v798: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:52:39.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:16 smithi033 ceph-mon[29274]: osdmap e136: 12 total, 12 up, 12 in 2024-04-23T18:52:39.586 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:18 smithi033 ceph-mon[29274]: pgmap v800: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-04-23T18:52:39.665 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:20 smithi033 ceph-mon[29274]: pgmap v801: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 950 B/s wr, 11 op/s 2024-04-23T18:52:39.675 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:27 smithi122 ceph-mon[32546]: pgmap v838: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:39.683 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:29 smithi122 ceph-mon[32546]: pgmap v839: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:52:39.690 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:31 smithi122 ceph-mon[32546]: pgmap v840: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:52:39.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:33 smithi122 ceph-mon[32546]: pgmap v841: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:52:39.785 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:35 smithi122 ceph-mon[32546]: pgmap v842: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:39.833 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:37 smithi122 ceph-mon[32546]: pgmap v843: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:39.844 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:27 smithi106 ceph-mon[32716]: pgmap v838: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:39.918 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:29 smithi106 ceph-mon[32716]: pgmap v839: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:52:39.991 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:31 smithi106 ceph-mon[32716]: pgmap v840: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:52:40.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:33 smithi106 ceph-mon[32716]: pgmap v841: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:52:40.102 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:35 smithi106 ceph-mon[32716]: pgmap v842: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:40.110 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:37 smithi106 ceph-mon[32716]: pgmap v843: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:40.120 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:39 smithi106 ceph-mon[32716]: pgmap v844: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:40.131 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:39 smithi122 ceph-mon[32546]: pgmap v844: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 -- 172.21.15.33:0/540141658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a4102b70 msgr2=0x7f49a4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/540141658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a4102b70 0x7f49a4106bb0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4990009900 tx=0x7f499002a410 comp rx=0 tx=0).stop 2024-04-23T18:52:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 -- 172.21.15.33:0/540141658 shutdown_connections 2024-04-23T18:52:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/540141658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 0x7f49a410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/540141658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a4102b70 0x7f49a4106bb0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:40.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/540141658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a41021c0 0x7f49a41025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:40.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 -- 172.21.15.33:0/540141658 >> 172.21.15.33:0/540141658 conn(0x7f49a40fda20 msgr2=0x7f49a40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 -- 172.21.15.33:0/540141658 shutdown_connections 2024-04-23T18:52:40.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.978+0000 7f49aaa7c640 1 -- 172.21.15.33:0/540141658 wait complete. 2024-04-23T18:52:40.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 Processor -- start 2024-04-23T18:52:40.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 -- start start 2024-04-23T18:52:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 0x7f49a41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:40.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 0x7f49a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41121d0 con 0x7f49a41021c0 2024-04-23T18:52:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4112050 con 0x7f49a4107290 2024-04-23T18:52:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49aaa7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a4112350 con 0x7f49a419a7a0 2024-04-23T18:52:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:40.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57544/0 (socket says 172.21.15.33:57544) 2024-04-23T18:52:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.979+0000 7f49a8818640 1 -- 172.21.15.33:0/2555440301 learned_addr learned my addr 172.21.15.33:0/2555440301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a9019640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 0x7f49a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:40.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a3fff640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 0x7f49a41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 -- 172.21.15.33:0/2555440301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 msgr2=0x7f49a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:41.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 0x7f49a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 -- 172.21.15.33:0/2555440301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 msgr2=0x7f49a41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:41.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 0x7f49a41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:41.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 -- 172.21.15.33:0/2555440301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a419b330 con 0x7f49a41021c0 2024-04-23T18:52:41.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a9019640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 0x7f49a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:41.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a8818640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f499800d770 tx=0x7f499800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:41.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a1ffb640 1 -- 172.21.15.33:0/2555440301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998017070 con 0x7f49a41021c0 2024-04-23T18:52:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a1ffb640 1 -- 172.21.15.33:0/2555440301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49980054d0 con 0x7f49a41021c0 2024-04-23T18:52:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a419b620 con 0x7f49a41021c0 2024-04-23T18:52:41.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49a1ffb640 1 -- 172.21.15.33:0/2555440301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998004c30 con 0x7f49a41021c0 2024-04-23T18:52:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.980+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a41a7780 con 0x7f49a41021c0 2024-04-23T18:52:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a1ffb640 1 -- 172.21.15.33:0/2555440301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4998004e10 con 0x7f49a41021c0 2024-04-23T18:52:41.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 Processor -- start 2024-04-23T18:52:41.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 -- start start 2024-04-23T18:52:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77480a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:41.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e7748dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e773e5970 con 0x563e77480620 2024-04-23T18:52:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563e773824a0 con 0x563e7748b800 2024-04-23T18:52:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49ad798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e773e1930 con 0x563e77480f40 2024-04-23T18:52:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a1ffb640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 0x7f49940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a9019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77480a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:42.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a9019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77480a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57546/0 (socket says 172.21.15.33:57546) 2024-04-23T18:52:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a9019640 1 -- 172.21.15.33:0/25347135 learned_addr learned my addr 172.21.15.33:0/25347135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.983+0000 7f49a9019640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 0x7f49940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:42.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a8818640 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:42.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e7748dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a1ffb640 1 -- 172.21.15.33:0/2555440301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f49980d02e0 con 0x7f49a41021c0 2024-04-23T18:52:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 -- 172.21.15.33:0/25347135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 msgr2=0x563e7748b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 -- 172.21.15.33:0/25347135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 msgr2=0x563e77480a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:42.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77480a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 -- 172.21.15.33:0/25347135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e770f8210 con 0x563e7748b800 2024-04-23T18:52:42.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a9019640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 0x7f49940a6280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f499c014ce0 tx=0x7f499c009a90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f49a3fff640 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e7748dbf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4990006010 tx=0x7f4990005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cf5df5b3060cf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:42.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f498effd640 1 -- 172.21.15.33:0/25347135 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990040070 con 0x563e7748b800 2024-04-23T18:52:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f498effd640 1 -- 172.21.15.33:0/25347135 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4990004030 con 0x563e7748b800 2024-04-23T18:52:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.984+0000 7f498effd640 1 -- 172.21.15.33:0/25347135 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990035040 con 0x563e7748b800 2024-04-23T18:52:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- 172.21.15.33:0/25347135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 msgr2=0x563e7748dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e7748dbf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4990006010 tx=0x7f4990005c00 comp rx=0 tx=0).stop 2024-04-23T18:52:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- 172.21.15.33:0/25347135 shutdown_connections 2024-04-23T18:52:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e7748dbf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:43.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 --2- 172.21.15.33:0/25347135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77480a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:43.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- 172.21.15.33:0/25347135 >> 172.21.15.33:0/25347135 conn(0x563e7747cb30 msgr2=0x563e7747cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- 172.21.15.33:0/25347135 shutdown_connections 2024-04-23T18:52:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- 172.21.15.33:0/25347135 wait complete. 2024-04-23T18:52:43.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.985+0000 7f49ad798740 1 -- start start 2024-04-23T18:52:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 Processor -- start 2024-04-23T18:52:43.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:43.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e77488470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e7748b800 0x563e774889f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:43.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e7748e3d0 con 0x563e77480620 2024-04-23T18:52:43.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563e773800b0 con 0x563e77480f40 2024-04-23T18:52:43.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e7748e550 con 0x563e7748b800 2024-04-23T18:52:43.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49ad798740 10 client.0 _ll_register_callbacks cb 0x563e7736dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:44.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57548/0 (socket says 172.21.15.33:57548) 2024-04-23T18:52:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49a8818640 1 -- 172.21.15.33:0/3333630753 learned_addr learned my addr 172.21.15.33:0/3333630753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:44.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49a9019640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e7748b800 0x563e774889f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.986+0000 7f49a3fff640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e77488470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 -- 172.21.15.33:0/3333630753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e7748b800 msgr2=0x563e774889f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:44.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e7748b800 0x563e774889f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 -- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 msgr2=0x563e77488470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e77488470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:44.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 -- 172.21.15.33:0/3333630753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e771c4620 con 0x563e77480620 2024-04-23T18:52:44.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a3fff640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e77488470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:44.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49a8818640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f49980986d0 tx=0x7f49980209b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b23b6b83f0fec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980cf070 con 0x563e77480620 2024-04-23T18:52:44.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f49980988e0 con 0x563e77480620 2024-04-23T18:52:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4990004030 con 0x563e77480620 2024-04-23T18:52:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4998005680 con 0x563e77480620 2024-04-23T18:52:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49ad798740 10 client.18162 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:45.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563e7749a380 con 0x563e77480620 2024-04-23T18:52:45.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4990004030 con 0x563e77480620 2024-04-23T18:52:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.987+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f4998018b40 con 0x563e77480620 2024-04-23T18:52:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 10 client.18162 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:45.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 20 client.18162 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e7749b840 0x563e7749dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 4 client.18162 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:52:45.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e773f3390 con 0x563e7749b840 2024-04-23T18:52:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49a3fff640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e7749b840 0x563e7749dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f4983fff640 10 client.18162 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.988+0000 7f49a3fff640 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e7749b840 0x563e7749dc30 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=856ed0ee46f7d8b server_cookie=829975ccc7a6041e in_seq=0 out_seq=0 2024-04-23T18:52:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f4983fff640 1 -- 172.21.15.33:0/3333630753 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e773f3390 con 0x563e7749b840 2024-04-23T18:52:45.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f4983fff640 10 client.18162 handle_command_reply: tid=0 2024-04-23T18:52:45.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 client.18162 shutdown 2024-04-23T18:52:45.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 msgr2=0x563e7748b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:45.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f49980986d0 tx=0x7f49980209b0 comp rx=0 tx=0).stop 2024-04-23T18:52:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 shutdown_connections 2024-04-23T18:52:45.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e7749b840 0x563e7749dc30 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e7748b800 0x563e774889f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e77488470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f4983fff640 0 client.18162 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 --2- 172.21.15.33:0/3333630753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e7748b180 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 >> 172.21.15.33:0/3333630753 conn(0x563e7747cb30 msgr2=0x563e774828b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 shutdown_connections 2024-04-23T18:52:46.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.989+0000 7f49ad798740 1 -- 172.21.15.33:0/3333630753 wait complete. 2024-04-23T18:52:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:21.990+0000 7f49ad798740 20 client.18162 trim_cache size 0 max 16384 2024-04-23T18:52:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 Processor -- start 2024-04-23T18:52:46.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 -- start start 2024-04-23T18:52:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e774a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41b1860 con 0x563e77480620 2024-04-23T18:52:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41af650 con 0x563e7748b800 2024-04-23T18:52:46.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49ad798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41b1190 con 0x563e77480f40 2024-04-23T18:52:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57550/0 (socket says 172.21.15.33:57550) 2024-04-23T18:52:46.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 -- 172.21.15.33:0/4006681367 learned_addr learned my addr 172.21.15.33:0/4006681367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:46.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a9019640 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e774a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:46.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 -- 172.21.15.33:0/4006681367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 msgr2=0x563e7748acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a3fff640 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:47.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:47.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 -- 172.21.15.33:0/4006681367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 msgr2=0x563e774a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:47.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e774a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 -- 172.21.15.33:0/4006681367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e77488d60 con 0x563e77480620 2024-04-23T18:52:47.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.005+0000 7f49a8818640 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4998009610 tx=0x7f499809a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7886dd495a3a579f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f498e7fc640 1 -- 172.21.15.33:0/4006681367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49980cf070 con 0x563e77480620 2024-04-23T18:52:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f498e7fc640 1 -- 172.21.15.33:0/4006681367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499809c970 con 0x563e77480620 2024-04-23T18:52:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f498e7fc640 1 -- 172.21.15.33:0/4006681367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998056040 con 0x563e77480620 2024-04-23T18:52:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- 172.21.15.33:0/4006681367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 msgr2=0x563e77497290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4998009610 tx=0x7f499809a780 comp rx=0 tx=0).stop 2024-04-23T18:52:47.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- 172.21.15.33:0/4006681367 shutdown_connections 2024-04-23T18:52:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e7748b800 0x563e774a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480f40 0x563e7748acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:47.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 --2- 172.21.15.33:0/4006681367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e77480620 0x563e77497290 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- 172.21.15.33:0/4006681367 >> 172.21.15.33:0/4006681367 conn(0x563e7747cb30 msgr2=0x563e77483af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- 172.21.15.33:0/4006681367 shutdown_connections 2024-04-23T18:52:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- 172.21.15.33:0/4006681367 wait complete. 2024-04-23T18:52:48.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.006+0000 7f49ad798740 1 -- start start 2024-04-23T18:52:48.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 Processor -- start 2024-04-23T18:52:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 0x563e77493ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:48.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e7748ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41b03f0 con 0x563e7748b800 2024-04-23T18:52:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41afd20 con 0x563e77480f40 2024-04-23T18:52:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a41b0ac0 con 0x563e77480620 2024-04-23T18:52:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49ad798740 10 client.0 _ll_register_callbacks cb 0x7f499809c110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49a9019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:48.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49a9019640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57554/0 (socket says 172.21.15.33:57554) 2024-04-23T18:52:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49a9019640 1 -- 172.21.15.33:0/4103393155 learned_addr learned my addr 172.21.15.33:0/4103393155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49a3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e7748ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.007+0000 7f49a8818640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 0x563e77493ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 -- 172.21.15.33:0/4103393155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 msgr2=0x563e77493ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 0x563e77493ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 -- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 msgr2=0x563e7748ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:50.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e7748ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:50.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 -- 172.21.15.33:0/4103393155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e7748fb80 con 0x563e7748b800 2024-04-23T18:52:50.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a3fff640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e7748ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:52:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a8818640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 0x563e77493ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:50.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49a9019640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f499c011c40 tx=0x7f499c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37817cabce3ac9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004b20 con 0x563e7748b800 2024-04-23T18:52:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f499c004cc0 con 0x563e7748b800 2024-04-23T18:52:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f499809c970 con 0x563e7748b800 2024-04-23T18:52:50.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f499c003970 con 0x563e7748b800 2024-04-23T18:52:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 10 client.18168 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563e77501d30 con 0x563e7748b800 2024-04-23T18:52:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499809c970 con 0x563e7748b800 2024-04-23T18:52:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f499c004690 con 0x563e7748b800 2024-04-23T18:52:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 10 client.18168 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 10 client.18168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 20 client.18168 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.008+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e77502530 0x563e77504950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.009+0000 7f49ad798740 4 client.18168 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:52:51.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.009+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e773ed200 con 0x563e77502530 2024-04-23T18:52:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.009+0000 7f49a8818640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e77502530 0x563e77504950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.009+0000 7f49a8818640 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e77502530 0x563e77504950 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5366bef52f6bedf7 server_cookie=299b1d6d66e63821 in_seq=0 out_seq=0 2024-04-23T18:52:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.009+0000 7f4983fff640 10 client.18168 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:51.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f4983fff640 1 -- 172.21.15.33:0/4103393155 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e773ed200 con 0x563e77502530 2024-04-23T18:52:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f4983fff640 10 client.18168 handle_command_reply: tid=0 2024-04-23T18:52:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 client.18168 shutdown 2024-04-23T18:52:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 msgr2=0x563e7748f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f499c011c40 tx=0x7f499c01b000 comp rx=0 tx=0).stop 2024-04-23T18:52:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 shutdown_connections 2024-04-23T18:52:51.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563e77502530 0x563e77504950 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e7748b800 0x563e7748f380 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:51.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f4983fff640 0 client.18168 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563e77480f40 0x563e7748ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:51.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 --2- 172.21.15.33:0/4103393155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e77480620 0x563e77493ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:51.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 >> 172.21.15.33:0/4103393155 conn(0x563e7747cb30 msgr2=0x563e77496280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 shutdown_connections 2024-04-23T18:52:51.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 1 -- 172.21.15.33:0/4103393155 wait complete. 2024-04-23T18:52:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.016+0000 7f49ad798740 20 client.18168 trim_cache size 0 max 16384 2024-04-23T18:52:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:52:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:52:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:52:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:52:52.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 msgr2=0x7f49940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 0x7f49940a6280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f499c014ce0 tx=0x7f499c009a90 comp rx=0 tx=0).stop 2024-04-23T18:52:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 msgr2=0x7f49a41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:52.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f499800d770 tx=0x7f499800dc30 comp rx=0 tx=0).stop 2024-04-23T18:52:52.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 shutdown_connections 2024-04-23T18:52:52.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f49940a3dc0 0x7f49940a6280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:52.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a419a7a0 0x7f49a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49a4107290 0x7f49a41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 --2- 172.21.15.33:0/2555440301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49a41021c0 0x7f49a41a0630 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 >> 172.21.15.33:0/2555440301 conn(0x7f49a40fda20 msgr2=0x7f49a40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:52.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 shutdown_connections 2024-04-23T18:52:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:22.017+0000 7f49aaa7c640 1 -- 172.21.15.33:0/2555440301 wait complete. 2024-04-23T18:52:52.730 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:41 smithi122 ceph-mon[32546]: pgmap v845: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:52.806 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:43 smithi122 ceph-mon[32546]: pgmap v846: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:52.817 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:45 smithi122 ceph-mon[32546]: pgmap v847: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:52.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:47 smithi122 ceph-mon[32546]: pgmap v848: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:52.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:49 smithi122 ceph-mon[32546]: pgmap v849: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T18:52:52.921 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:51 smithi122 ceph-mon[32546]: pgmap v850: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:52:53.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:41 smithi106 ceph-mon[32716]: pgmap v845: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:52:53.244 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:43 smithi106 ceph-mon[32716]: pgmap v846: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:52:53.304 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:45 smithi106 ceph-mon[32716]: pgmap v847: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:53.355 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:47 smithi106 ceph-mon[32716]: pgmap v848: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:52:53.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:49 smithi106 ceph-mon[32716]: pgmap v849: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T18:52:53.470 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:51 smithi106 ceph-mon[32716]: pgmap v850: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:52:53.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:22 smithi033 ceph-mon[29274]: pgmap v802: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:53.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:24 smithi033 ceph-mon[29274]: pgmap v803: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:53.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:26 smithi033 ceph-mon[29274]: pgmap v804: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:52:53.682 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:28 smithi033 ceph-mon[29274]: pgmap v805: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 378 B/s wr, 9 op/s 2024-04-23T18:52:53.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:30 smithi033 ceph-mon[29274]: pgmap v806: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:53.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:32 smithi033 ceph-mon[29274]: pgmap v807: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:52:53.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:34 smithi033 ceph-mon[29274]: pgmap v808: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:52:54.004 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:53 smithi122 ceph-mon[32546]: pgmap v851: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:52:54.044 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:53 smithi106 ceph-mon[32716]: pgmap v851: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:52:54.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 -- 172.21.15.33:0/337734052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0684100d80 msgr2=0x7f0684101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 --2- 172.21.15.33:0/337734052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0684100d80 0x7f0684101160 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0680009900 tx=0x7f068002a410 comp rx=0 tx=0).stop 2024-04-23T18:52:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 -- 172.21.15.33:0/337734052 shutdown_connections 2024-04-23T18:52:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 --2- 172.21.15.33:0/337734052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f068410e930 0x7f0684110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 --2- 172.21.15.33:0/337734052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06841016a0 0x7f068410e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 --2- 172.21.15.33:0/337734052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0684100d80 0x7f0684101160 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.117+0000 7f068abaf640 1 -- 172.21.15.33:0/337734052 >> 172.21.15.33:0/337734052 conn(0x7f06840771d0 msgr2=0x7f06840775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- 172.21.15.33:0/337734052 shutdown_connections 2024-04-23T18:52:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- 172.21.15.33:0/337734052 wait complete. 2024-04-23T18:52:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 Processor -- start 2024-04-23T18:52:54.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- start start 2024-04-23T18:52:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0684100d80 0x7f06841a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:54.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06841016a0 0x7f06841a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0684114450 con 0x7f068410e930 2024-04-23T18:52:54.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06841142d0 con 0x7f0684100d80 2024-04-23T18:52:54.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.118+0000 7f068abaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06841145d0 con 0x7f06841016a0 2024-04-23T18:52:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:54.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57588/0 (socket says 172.21.15.33:57588) 2024-04-23T18:52:54.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 -- 172.21.15.33:0/1334963644 learned_addr learned my addr 172.21.15.33:0/1334963644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068894b640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0684100d80 0x7f06841a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f067bfff640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06841016a0 0x7f06841a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 -- 172.21.15.33:0/1334963644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06841016a0 msgr2=0x7f06841a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06841016a0 0x7f06841a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:55.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 -- 172.21.15.33:0/1334963644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0684100d80 msgr2=0x7f06841a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0684100d80 0x7f06841a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:55.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 -- 172.21.15.33:0/1334963644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068419af70 con 0x7f068410e930 2024-04-23T18:52:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068914c640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f067c009fd0 tx=0x7f067c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:55.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f0679ffb640 1 -- 172.21.15.33:0/1334963644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c004d20 con 0x7f068410e930 2024-04-23T18:52:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f0679ffb640 1 -- 172.21.15.33:0/1334963644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f067c005020 con 0x7f068410e930 2024-04-23T18:52:55.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068419b260 con 0x7f068410e930 2024-04-23T18:52:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.119+0000 7f0679ffb640 1 -- 172.21.15.33:0/1334963644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0116b0 con 0x7f068410e930 2024-04-23T18:52:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.120+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06841a7570 con 0x7f068410e930 2024-04-23T18:52:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.122+0000 7f0679ffb640 1 -- 172.21.15.33:0/1334963644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f067c0051d0 con 0x7f068410e930 2024-04-23T18:52:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 Processor -- start 2024-04-23T18:52:55.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 -- start start 2024-04-23T18:52:55.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace348a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace355bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ace2ad970 con 0x562ace348f40 2024-04-23T18:52:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562ace24a4a0 con 0x562ace348620 2024-04-23T18:52:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068d99c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ace2a9930 con 0x562ace353800 2024-04-23T18:52:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57592/0 (socket says 172.21.15.33:57592) 2024-04-23T18:52:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068894b640 1 -- 172.21.15.33:0/2509793749 learned_addr learned my addr 172.21.15.33:0/2509793749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f0679ffb640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 0x7f066c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:55.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f067bfff640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace348a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.123+0000 7f068914c640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 0x7f066c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f067bfff640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace355bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 -- 172.21.15.33:0/2509793749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 msgr2=0x562ace355bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:56.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace355bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 -- 172.21.15.33:0/2509793749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 msgr2=0x562ace348a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f0679ffb640 1 -- 172.21.15.33:0/1334963644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f067c0d0c90 con 0x7f068410e930 2024-04-23T18:52:56.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace348a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 -- 172.21.15.33:0/2509793749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562acdfc0210 con 0x562ace348f40 2024-04-23T18:52:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f067bfff640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace355bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:52:56.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068894b640 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0680009900 tx=0x7f0680032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25a67e8b151bd06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:56.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f068914c640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 0x7f066c0a6280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f067c0990d0 tx=0x7f067c0677c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f066affd640 1 -- 172.21.15.33:0/2509793749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680040070 con 0x562ace348f40 2024-04-23T18:52:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f066affd640 1 -- 172.21.15.33:0/2509793749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680004030 con 0x562ace348f40 2024-04-23T18:52:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.124+0000 7f066affd640 1 -- 172.21.15.33:0/2509793749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680035040 con 0x562ace348f40 2024-04-23T18:52:56.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- 172.21.15.33:0/2509793749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 msgr2=0x562ace3532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0680009900 tx=0x7f0680032bd0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- 172.21.15.33:0/2509793749 shutdown_connections 2024-04-23T18:52:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace355bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace3532c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 --2- 172.21.15.33:0/2509793749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace348a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- 172.21.15.33:0/2509793749 >> 172.21.15.33:0/2509793749 conn(0x562ace344b30 msgr2=0x562ace344f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- 172.21.15.33:0/2509793749 shutdown_connections 2024-04-23T18:52:56.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- 172.21.15.33:0/2509793749 wait complete. 2024-04-23T18:52:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 -- start start 2024-04-23T18:52:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.125+0000 7f068d99c740 1 Processor -- start 2024-04-23T18:52:57.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace353170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:57.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 0x562ace350460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ace3563d0 con 0x562ace353800 2024-04-23T18:52:57.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562ace2480b0 con 0x562ace348f40 2024-04-23T18:52:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ace356550 con 0x562ace348620 2024-04-23T18:52:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068d99c740 10 client.0 _ll_register_callbacks cb 0x562ace235d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:52:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:57.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57602/0 (socket says 172.21.15.33:57602) 2024-04-23T18:52:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 -- 172.21.15.33:0/143855857 learned_addr learned my addr 172.21.15.33:0/143855857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:57.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068894b640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 0x562ace350460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace353170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 -- 172.21.15.33:0/143855857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 msgr2=0x562ace353170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:57.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace353170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 -- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 msgr2=0x562ace350460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:52:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 0x562ace350460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f067bfff640 1 -- 172.21.15.33:0/143855857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ace08c620 con 0x562ace353800 2024-04-23T18:52:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.126+0000 7f068894b640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 0x562ace350460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:52:57.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f067bfff640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0674012db0 tx=0x7f0674004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690ccda0eaee878a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:52:58.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:52:58.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067401b040 con 0x562ace353800 2024-04-23T18:52:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f0674012250 con 0x562ace353800 2024-04-23T18:52:58.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0680004030 con 0x562ace353800 2024-04-23T18:52:58.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0674004030 con 0x562ace353800 2024-04-23T18:52:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 10 client.18180 fetch_fsmap learned FSMap version 44 2024-04-23T18:52:58.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562ace3623a0 con 0x562ace353800 2024-04-23T18:52:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680004030 con 0x562ace353800 2024-04-23T18:52:58.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f0674003b90 con 0x562ace353800 2024-04-23T18:52:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 10 client.18180 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:52:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 10 client.18180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:52:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 20 client.18180 populate_metadata read hostname 'smithi033' 2024-04-23T18:52:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace363810 0x562ace365c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 4 client.18180 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:52:58.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.127+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ace2bb390 con 0x562ace363810 2024-04-23T18:52:58.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.128+0000 7f068894b640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace363810 0x562ace365c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:58.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.128+0000 7f068894b640 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace363810 0x562ace365c00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc45c3c51f5c7c7f server_cookie=55afbaadb1545eca in_seq=0 out_seq=0 2024-04-23T18:52:58.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.128+0000 7f0663fff640 10 client.18180 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f0663fff640 1 -- 172.21.15.33:0/143855857 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ace2bb390 con 0x562ace363810 2024-04-23T18:52:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f0663fff640 10 client.18180 handle_command_reply: tid=0 2024-04-23T18:52:58.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 client.18180 shutdown 2024-04-23T18:52:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 msgr2=0x562ace3509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:52:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0674012db0 tx=0x7f0674004730 comp rx=0 tx=0).stop 2024-04-23T18:52:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 shutdown_connections 2024-04-23T18:52:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace363810 0x562ace365c00 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace353800 0x562ace3509c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:59.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348f40 0x562ace350460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f0663fff640 0 client.18180 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:52:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 --2- 172.21.15.33:0/143855857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace353170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:52:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 >> 172.21.15.33:0/143855857 conn(0x562ace344b30 msgr2=0x562ace34a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:52:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 shutdown_connections 2024-04-23T18:52:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 1 -- 172.21.15.33:0/143855857 wait complete. 2024-04-23T18:52:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.129+0000 7f068d99c740 20 client.18180 trim_cache size 0 max 16384 2024-04-23T18:52:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 Processor -- start 2024-04-23T18:52:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 -- start start 2024-04-23T18:52:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace35f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:59.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace353800 0x562ace352ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:52:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06841b1940 con 0x562ace348f40 2024-04-23T18:52:59.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06841af730 con 0x562ace353800 2024-04-23T18:52:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068d99c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06841b1270 con 0x562ace348620 2024-04-23T18:52:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57618/0 (socket says 172.21.15.33:57618) 2024-04-23T18:52:59.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 -- 172.21.15.33:0/1287745274 learned_addr learned my addr 172.21.15.33:0/1287745274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:52:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f067bfff640 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace35f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:52:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 -- 172.21.15.33:0/1287745274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 msgr2=0x562ace35f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace35f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:00.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f067bfff640 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace353800 0x562ace352ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 -- 172.21.15.33:0/1287745274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace353800 msgr2=0x562ace352ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.145+0000 7f068894b640 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace353800 0x562ace352ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068894b640 1 -- 172.21.15.33:0/1287745274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ace348d70 con 0x562ace348f40 2024-04-23T18:53:00.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068894b640 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x562ace349bc0 tx=0x7f0680031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13a99dcd7afa5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:00.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f066a7fc640 1 -- 172.21.15.33:0/1287745274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680040040 con 0x562ace348f40 2024-04-23T18:53:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f066a7fc640 1 -- 172.21.15.33:0/1287745274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680005150 con 0x562ace348f40 2024-04-23T18:53:00.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f066a7fc640 1 -- 172.21.15.33:0/1287745274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680035040 con 0x562ace348f40 2024-04-23T18:53:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 -- 172.21.15.33:0/1287745274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 msgr2=0x562ace343b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x562ace349bc0 tx=0x7f0680031400 comp rx=0 tx=0).stop 2024-04-23T18:53:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 -- 172.21.15.33:0/1287745274 shutdown_connections 2024-04-23T18:53:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace353800 0x562ace352ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace343b30 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:00.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.146+0000 7f068d99c740 1 --2- 172.21.15.33:0/1287745274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace348620 0x562ace35f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:00.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- 172.21.15.33:0/1287745274 >> 172.21.15.33:0/1287745274 conn(0x562ace344b30 msgr2=0x562ace343420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- 172.21.15.33:0/1287745274 shutdown_connections 2024-04-23T18:53:01.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- 172.21.15.33:0/1287745274 wait complete. 2024-04-23T18:53:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- start start 2024-04-23T18:53:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 Processor -- start 2024-04-23T18:53:01.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace35b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace356bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06841b04d0 con 0x562ace348f40 2024-04-23T18:53:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06841afe00 con 0x562ace348620 2024-04-23T18:53:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06841b0ba0 con 0x562ace353800 2024-04-23T18:53:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.147+0000 7f068d99c740 10 client.0 _ll_register_callbacks cb 0x7f06740129c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57622/0 (socket says 172.21.15.33:57622) 2024-04-23T18:53:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 -- 172.21.15.33:0/712581276 learned_addr learned my addr 172.21.15.33:0/712581276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f067bfff640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace35b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:01.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f067bfff640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace356bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 -- 172.21.15.33:0/712581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 msgr2=0x562ace356bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace356bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:01.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 -- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 msgr2=0x562ace35b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:02.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace35b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 -- 172.21.15.33:0/712581276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ace3573e0 con 0x562ace348f40 2024-04-23T18:53:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068894b640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c00 tx=0x7f0680032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a69da0aed7567 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f0663fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:02.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0680055070 con 0x562ace348f40 2024-04-23T18:53:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f06800314a0 con 0x562ace348f40 2024-04-23T18:53:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0680005150 con 0x562ace348f40 2024-04-23T18:53:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0680040070 con 0x562ace348f40 2024-04-23T18:53:02.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068d99c740 10 client.18186 fetch_fsmap learned FSMap version 44 2024-04-23T18:53:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.148+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562ace3c98b0 con 0x562ace348f40 2024-04-23T18:53:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0680005150 con 0x562ace348f40 2024-04-23T18:53:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f068004e040 con 0x562ace348f40 2024-04-23T18:53:02.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 10 client.18186 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:53:02.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 10 client.18186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:02.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 20 client.18186 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:02.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace3c9be0 0x562ace3cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 4 client.18186 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:53:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ace2b5200 con 0x562ace3c9be0 2024-04-23T18:53:02.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f067bfff640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace3c9be0 0x562ace3cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f067bfff640 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace3c9be0 0x562ace3cbfd0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b00dc523efe44b3 server_cookie=67f2402382da67a1 in_seq=0 out_seq=0 2024-04-23T18:53:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.149+0000 7f0663fff640 10 client.18186 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.153+0000 7f0663fff640 1 -- 172.21.15.33:0/712581276 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ace2b5200 con 0x562ace3c9be0 2024-04-23T18:53:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.153+0000 7f0663fff640 10 client.18186 handle_command_reply: tid=0 2024-04-23T18:53:03.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.153+0000 7f068d99c740 1 client.18186 shutdown 2024-04-23T18:53:03.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 msgr2=0x562ace35bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:03.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0680000c00 tx=0x7f0680032bd0 comp rx=0 tx=0).stop 2024-04-23T18:53:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 shutdown_connections 2024-04-23T18:53:03.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562ace3c9be0 0x562ace3cbfd0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ace353800 0x562ace356bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ace348f40 0x562ace35bc00 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f0663fff640 0 client.18186 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:03.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 --2- 172.21.15.33:0/712581276 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562ace348620 0x562ace35b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 >> 172.21.15.33:0/712581276 conn(0x562ace344b30 msgr2=0x562ace343cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 shutdown_connections 2024-04-23T18:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 1 -- 172.21.15.33:0/712581276 wait complete. 2024-04-23T18:53:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068d99c740 20 client.18186 trim_cache size 0 max 16384 2024-04-23T18:53:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:53:03.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:53:03.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:53:03.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:53:03.868 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:55 smithi122 ceph-mon[32546]: pgmap v852: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:53:03.983 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:57 smithi122 ceph-mon[32546]: pgmap v853: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:53:03.994 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:52:59 smithi122 ceph-mon[32546]: pgmap v854: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:53:04.032 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:01 smithi122 ceph-mon[32546]: pgmap v855: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:04.081 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:02 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:04.093 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:03 smithi122 ceph-mon[32546]: pgmap v856: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:53:04.105 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:55 smithi106 ceph-mon[32716]: pgmap v852: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:53:04.114 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:57 smithi106 ceph-mon[32716]: pgmap v853: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:53:04.124 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:52:59 smithi106 ceph-mon[32716]: pgmap v854: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:53:04.186 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:01 smithi106 ceph-mon[32716]: pgmap v855: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:04.234 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:02 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:04.347 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:03 smithi106 ceph-mon[32716]: pgmap v856: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 msgr2=0x7f066c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 0x7f066c0a6280 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f067c0990d0 tx=0x7f067c0677c0 comp rx=0 tx=0).stop 2024-04-23T18:53:04.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 msgr2=0x7f068419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:04.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.154+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f067c009fd0 tx=0x7f067c00ee60 comp rx=0 tx=0).stop 2024-04-23T18:53:04.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 shutdown_connections 2024-04-23T18:53:04.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f066c0a3dc0 0x7f066c0a6280 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:04.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f068410e930 0x7f068419a6b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06841016a0 0x7f06841a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:04.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 --2- 172.21.15.33:0/1334963644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0684100d80 0x7f06841a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 >> 172.21.15.33:0/1334963644 conn(0x7f06840771d0 msgr2=0x7f068410fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 shutdown_connections 2024-04-23T18:53:04.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:35.155+0000 7f068abaf640 1 -- 172.21.15.33:0/1334963644 wait complete. 2024-04-23T18:53:04.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:36 smithi033 ceph-mon[29274]: pgmap v809: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:04.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:38 smithi033 ceph-mon[29274]: pgmap v810: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:53:04.972 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:40 smithi033 ceph-mon[29274]: pgmap v811: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:05.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:42 smithi033 ceph-mon[29274]: pgmap v812: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:05.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:44 smithi033 ceph-mon[29274]: pgmap v813: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:05.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:46 smithi033 ceph-mon[29274]: pgmap v814: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 -- 172.21.15.33:0/447386681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0102b70 msgr2=0x7f18e0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 --2- 172.21.15.33:0/447386681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0102b70 0x7f18e0106bb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009900 tx=0x7f18d002a410 comp rx=0 tx=0).stop 2024-04-23T18:53:05.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 -- 172.21.15.33:0/447386681 shutdown_connections 2024-04-23T18:53:05.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 --2- 172.21.15.33:0/447386681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0107290 0x7f18e010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:05.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 --2- 172.21.15.33:0/447386681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e0102b70 0x7f18e0106bb0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 --2- 172.21.15.33:0/447386681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e01021c0 0x7f18e01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.255+0000 7f18e6dba640 1 -- 172.21.15.33:0/447386681 >> 172.21.15.33:0/447386681 conn(0x7f18e00fda20 msgr2=0x7f18e00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:05.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- 172.21.15.33:0/447386681 shutdown_connections 2024-04-23T18:53:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- 172.21.15.33:0/447386681 wait complete. 2024-04-23T18:53:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 Processor -- start 2024-04-23T18:53:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- start start 2024-04-23T18:53:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e0107290 0x7f18e0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0074cd0 0x7f18e01a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01121b0 con 0x7f18e01021c0 2024-04-23T18:53:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0112030 con 0x7f18e0107290 2024-04-23T18:53:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.256+0000 7f18e6dba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18e0112330 con 0x7f18e0074cd0 2024-04-23T18:53:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41732/0 (socket says 172.21.15.33:41732) 2024-04-23T18:53:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 -- 172.21.15.33:0/1571015096 learned_addr learned my addr 172.21.15.33:0/1571015096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18d7fff640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e0107290 0x7f18e0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e5357640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0074cd0 0x7f18e01a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 -- 172.21.15.33:0/1571015096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0074cd0 msgr2=0x7f18e01a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:06.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0074cd0 0x7f18e01a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 -- 172.21.15.33:0/1571015096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e0107290 msgr2=0x7f18e0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e0107290 0x7f18e0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 -- 172.21.15.33:0/1571015096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18e01a6f60 con 0x7f18e01021c0 2024-04-23T18:53:06.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e4b56640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d950 tx=0x7f18dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:06.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18d5ffb640 1 -- 172.21.15.33:0/1571015096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc017070 con 0x7f18e01021c0 2024-04-23T18:53:06.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18d5ffb640 1 -- 172.21.15.33:0/1571015096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc009c30 con 0x7f18e01021c0 2024-04-23T18:53:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18e01a7250 con 0x7f18e01021c0 2024-04-23T18:53:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.257+0000 7f18d5ffb640 1 -- 172.21.15.33:0/1571015096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc00f620 con 0x7f18e01021c0 2024-04-23T18:53:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.258+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18e01a7770 con 0x7f18e01021c0 2024-04-23T18:53:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18d5ffb640 1 -- 172.21.15.33:0/1571015096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f18dc010460 con 0x7f18e01021c0 2024-04-23T18:53:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 Processor -- start 2024-04-23T18:53:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 -- start start 2024-04-23T18:53:07.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18d5ffb640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 0x7f18c80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c7bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559728bd3970 con 0x559728c6e620 2024-04-23T18:53:07.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559728b704a0 con 0x559728c6ef40 2024-04-23T18:53:07.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.260+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559728bcf930 con 0x559728c79800 2024-04-23T18:53:07.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e5357640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 0x7f18c80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41734/0 (socket says 172.21.15.33:41734) 2024-04-23T18:53:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 -- 172.21.15.33:0/102977907 learned_addr learned my addr 172.21.15.33:0/102977907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18d5ffb640 1 -- 172.21.15.33:0/1571015096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f18dc0d0480 con 0x7f18e01021c0 2024-04-23T18:53:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18d7fff640 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e5357640 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c7bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 -- 172.21.15.33:0/102977907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 msgr2=0x559728c7bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:07.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c7bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 -- 172.21.15.33:0/102977907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 msgr2=0x559728c792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:07.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 -- 172.21.15.33:0/102977907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597288e6210 con 0x559728c6e620 2024-04-23T18:53:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e5357640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 0x7f18c80a62b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f18d80092f0 tx=0x7f18d8014a70 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18e4b56640 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f18dc015e50 tx=0x7f18dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2680e4c79654e2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18c6ffd640 1 -- 172.21.15.33:0/102977907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc020030 con 0x559728c6e620 2024-04-23T18:53:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.261+0000 7f18c6ffd640 1 -- 172.21.15.33:0/102977907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc09c3b0 con 0x559728c6e620 2024-04-23T18:53:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18c6ffd640 1 -- 172.21.15.33:0/102977907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18dc098a20 con 0x559728c6e620 2024-04-23T18:53:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 -- 172.21.15.33:0/102977907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 msgr2=0x559728c6ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f18dc015e50 tx=0x7f18dc005880 comp rx=0 tx=0).stop 2024-04-23T18:53:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 -- 172.21.15.33:0/102977907 shutdown_connections 2024-04-23T18:53:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c7bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/102977907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6e620 0x559728c6ea00 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 -- 172.21.15.33:0/102977907 >> 172.21.15.33:0/102977907 conn(0x559728c6ab30 msgr2=0x559728c6af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 -- 172.21.15.33:0/102977907 shutdown_connections 2024-04-23T18:53:08.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.262+0000 7f18e9ba8740 1 -- 172.21.15.33:0/102977907 wait complete. 2024-04-23T18:53:08.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 -- start start 2024-04-23T18:53:08.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 Processor -- start 2024-04-23T18:53:08.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:08.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559728c7c3d0 con 0x559728c79800 2024-04-23T18:53:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559728b6e0b0 con 0x559728c6ef40 2024-04-23T18:53:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559728c7c550 con 0x559728c6e620 2024-04-23T18:53:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e9ba8740 10 client.0 _ll_register_callbacks cb 0x559728b5bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e5357640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e5357640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41748/0 (socket says 172.21.15.33:41748) 2024-04-23T18:53:09.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18e5357640 1 -- 172.21.15.33:0/2236930531 learned_addr learned my addr 172.21.15.33:0/2236930531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:09.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.263+0000 7f18d7fff640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:09.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 -- 172.21.15.33:0/2236930531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 msgr2=0x559728c791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:09.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 -- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 msgr2=0x559728c764c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:09.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c764c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 -- 172.21.15.33:0/2236930531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597289b2620 con 0x559728c79800 2024-04-23T18:53:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e5357640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f18d80128f0 tx=0x7f18d801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2142a8e4bd24ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18bffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8026070 con 0x559728c79800 2024-04-23T18:53:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f18d8004d20 con 0x559728c79800 2024-04-23T18:53:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18dc09c3b0 con 0x559728c79800 2024-04-23T18:53:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f18d8030050 con 0x559728c79800 2024-04-23T18:53:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e9ba8740 10 client.18198 fetch_fsmap learned FSMap version 44 2024-04-23T18:53:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.264+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559728c90bc0 con 0x559728c79800 2024-04-23T18:53:09.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18dc09c3b0 con 0x559728c79800 2024-04-23T18:53:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f18d800f040 con 0x559728c79800 2024-04-23T18:53:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 10 client.18198 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:53:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:10.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 20 client.18198 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728c920d0 0x559728c944c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:10.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 4 client.18198 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:53:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559728be1390 con 0x559728c920d0 2024-04-23T18:53:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728c920d0 0x559728c944c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:10.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728c920d0 0x559728c944c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d86c0ab9d8bbb640 server_cookie=926c107f9fb2f5a in_seq=0 out_seq=0 2024-04-23T18:53:10.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.265+0000 7f18bffff640 10 client.18198 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:10.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18bffff640 1 -- 172.21.15.33:0/2236930531 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559728be1390 con 0x559728c920d0 2024-04-23T18:53:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18bffff640 10 client.18198 handle_command_reply: tid=0 2024-04-23T18:53:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 client.18198 shutdown 2024-04-23T18:53:10.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 msgr2=0x559728c76ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:10.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f18d80128f0 tx=0x7f18d801a000 comp rx=0 tx=0).stop 2024-04-23T18:53:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 shutdown_connections 2024-04-23T18:53:10.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728c920d0 0x559728c944c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:10.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c79800 0x559728c76ad0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:10.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6ef40 0x559728c764c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:10.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18bffff640 0 client.18198 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:10.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2236930531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:10.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.266+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 >> 172.21.15.33:0/2236930531 conn(0x559728c6ab30 msgr2=0x559728c708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.267+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 shutdown_connections 2024-04-23T18:53:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.267+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2236930531 wait complete. 2024-04-23T18:53:10.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.267+0000 7f18e9ba8740 20 client.18198 trim_cache size 0 max 16384 2024-04-23T18:53:10.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 Processor -- start 2024-04-23T18:53:10.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 -- start start 2024-04-23T18:53:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6e620 0x559728c99dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:11.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:11.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:11.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ad400 con 0x559728c6ef40 2024-04-23T18:53:11.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ab1f0 con 0x559728c6e620 2024-04-23T18:53:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01acd30 con 0x559728c79800 2024-04-23T18:53:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41764/0 (socket says 172.21.15.33:41764) 2024-04-23T18:53:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 -- 172.21.15.33:0/1152979455 learned_addr learned my addr 172.21.15.33:0/1152979455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18e4b56640 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6e620 0x559728c99dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 -- 172.21.15.33:0/1152979455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 msgr2=0x559728c73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:11.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:11.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 -- 172.21.15.33:0/1152979455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6e620 msgr2=0x559728c99dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:11.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6e620 0x559728c99dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:11.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.282+0000 7f18d7fff640 1 -- 172.21.15.33:0/1152979455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559728c770d0 con 0x559728c6ef40 2024-04-23T18:53:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18d7fff640 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009900 tx=0x7f18d0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e066f7dd0e9d7533 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18c67fc640 1 -- 172.21.15.33:0/1152979455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0035040 con 0x559728c6ef40 2024-04-23T18:53:11.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18c67fc640 1 -- 172.21.15.33:0/1152979455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d00317e0 con 0x559728c6ef40 2024-04-23T18:53:11.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18c67fc640 1 -- 172.21.15.33:0/1152979455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0004750 con 0x559728c6ef40 2024-04-23T18:53:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 -- 172.21.15.33:0/1152979455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 msgr2=0x559728c9a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009900 tx=0x7f18d0005c00 comp rx=0 tx=0).stop 2024-04-23T18:53:11.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 -- 172.21.15.33:0/1152979455 shutdown_connections 2024-04-23T18:53:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c79800 0x559728c73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:12.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c9a310 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:12.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.283+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/1152979455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c6e620 0x559728c99dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- 172.21.15.33:0/1152979455 >> 172.21.15.33:0/1152979455 conn(0x559728c6ab30 msgr2=0x559728c69cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- 172.21.15.33:0/1152979455 shutdown_connections 2024-04-23T18:53:12.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- 172.21.15.33:0/1152979455 wait complete. 2024-04-23T18:53:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- start start 2024-04-23T18:53:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 Processor -- start 2024-04-23T18:53:12.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c87a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:12.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 0x559728c88510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01abf90 con 0x559728c6ef40 2024-04-23T18:53:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ab8c0 con 0x559728c79800 2024-04-23T18:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f18e01ac660 con 0x559728c6e620 2024-04-23T18:53:12.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.284+0000 7f18e9ba8740 10 client.0 _ll_register_callbacks cb 0x7f18d80126e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:12.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41780/0 (socket says 172.21.15.33:41780) 2024-04-23T18:53:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 -- 172.21.15.33:0/2819029384 learned_addr learned my addr 172.21.15.33:0/2819029384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e5357640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 0x559728c88510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c87a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:12.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 -- 172.21.15.33:0/2819029384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 msgr2=0x559728c87a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:12.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c87a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:13.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 -- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 msgr2=0x559728c88510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 0x559728c88510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 -- 172.21.15.33:0/2819029384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559728cf76a0 con 0x559728c6ef40 2024-04-23T18:53:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c87a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e5357640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 0x559728c88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18d7fff640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f18d0004620 tx=0x7f18d0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fa148b32cf6626 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18bffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:13.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d0035040 con 0x559728c6ef40 2024-04-23T18:53:13.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f18d0047890 con 0x559728c6ef40 2024-04-23T18:53:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18d00317e0 con 0x559728c6ef40 2024-04-23T18:53:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f18d0047a90 con 0x559728c6ef40 2024-04-23T18:53:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e9ba8740 10 client.18204 fetch_fsmap learned FSMap version 44 2024-04-23T18:53:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.285+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x559728cf89b0 con 0x559728c6ef40 2024-04-23T18:53:13.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18d00317e0 con 0x559728c6ef40 2024-04-23T18:53:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f18d003dce0 con 0x559728c6ef40 2024-04-23T18:53:13.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 10 client.18204 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:53:13.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 10 client.18204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 20 client.18204 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728cf9100 0x559728cfb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 4 client.18204 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:53:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559728bdb200 con 0x559728cf9100 2024-04-23T18:53:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.286+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728cf9100 0x559728cfb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.287+0000 7f18e4b56640 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728cf9100 0x559728cfb520 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cde06ad35997863 server_cookie=3f8908bcfd16021a in_seq=0 out_seq=0 2024-04-23T18:53:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.287+0000 7f18bffff640 10 client.18204 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18bffff640 1 -- 172.21.15.33:0/2819029384 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559728bdb200 con 0x559728cf9100 2024-04-23T18:53:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18bffff640 10 client.18204 handle_command_reply: tid=0 2024-04-23T18:53:14.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 client.18204 shutdown 2024-04-23T18:53:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 msgr2=0x559728c87fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:14.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f18d0004620 tx=0x7f18d0005760 comp rx=0 tx=0).stop 2024-04-23T18:53:14.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 shutdown_connections 2024-04-23T18:53:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559728cf9100 0x559728cfb520 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559728c79800 0x559728c88510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559728c6ef40 0x559728c87fd0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 --2- 172.21.15.33:0/2819029384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559728c6e620 0x559728c87a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:14.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 >> 172.21.15.33:0/2819029384 conn(0x559728c6ab30 msgr2=0x559728c8cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:14.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18bffff640 0 client.18204 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:14.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 shutdown_connections 2024-04-23T18:53:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 1 -- 172.21.15.33:0/2819029384 wait complete. 2024-04-23T18:53:14.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.291+0000 7f18e9ba8740 20 client.18204 trim_cache size 0 max 16384 2024-04-23T18:53:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:53:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:53:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:53:14.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:53:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 msgr2=0x7f18c80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 0x7f18c80a62b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f18d80092f0 tx=0x7f18d8014a70 comp rx=0 tx=0).stop 2024-04-23T18:53:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 msgr2=0x7f18e0074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:15.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00d950 tx=0x7f18dc00de10 comp rx=0 tx=0).stop 2024-04-23T18:53:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 shutdown_connections 2024-04-23T18:53:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f18c80a3df0 0x7f18c80a62b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:15.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f18e0074cd0 0x7f18e01a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18e0107290 0x7f18e0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 --2- 172.21.15.33:0/1571015096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18e01021c0 0x7f18e0074250 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 >> 172.21.15.33:0/1571015096 conn(0x7f18e00fda20 msgr2=0x7f18e00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 shutdown_connections 2024-04-23T18:53:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:51:48.292+0000 7f18e6dba640 1 -- 172.21.15.33:0/1571015096 wait complete. 2024-04-23T18:53:15.500 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:48 smithi033 ceph-mon[29274]: pgmap v815: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:15.508 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:50 smithi033 ceph-mon[29274]: pgmap v816: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:53:15.517 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:52 smithi033 ceph-mon[29274]: pgmap v817: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:15.576 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:54 smithi033 ceph-mon[29274]: pgmap v818: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:15.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:56 smithi033 ceph-mon[29274]: pgmap v819: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:15.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:51:58 smithi033 ceph-mon[29274]: pgmap v820: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:15.671 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:00 smithi033 ceph-mon[29274]: pgmap v821: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:15.782 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:05 smithi122 ceph-mon[32546]: pgmap v857: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:53:15.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: pgmap v858: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:53:15.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:15.897 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:16.014 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:16.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:16.113 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: pgmap v859: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-04-23T18:53:16.121 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: pgmap v860: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:53:16.128 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:06 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:16.136 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:09 smithi122 ceph-mon[32546]: pgmap v861: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:53:16.204 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:10 smithi122 ceph-mon[32546]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:53:16.225 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:11 smithi122 ceph-mon[32546]: pgmap v863: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-23T18:53:16.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:13 smithi122 ceph-mon[32546]: pgmap v864: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-23T18:53:16.363 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:15 smithi122 ceph-mon[32546]: pgmap v865: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:53:16.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:05 smithi106 ceph-mon[32716]: pgmap v857: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:53:16.387 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: pgmap v858: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:53:16.395 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:16.435 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:16.492 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:16.503 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:16.538 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: pgmap v859: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-04-23T18:53:16.607 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: pgmap v860: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:53:16.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:06 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:16.784 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:09 smithi106 ceph-mon[32716]: pgmap v861: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:53:16.850 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:10 smithi106 ceph-mon[32716]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:53:16.949 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:11 smithi106 ceph-mon[32716]: pgmap v863: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-23T18:53:16.998 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:13 smithi106 ceph-mon[32716]: pgmap v864: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-23T18:53:17.092 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:15 smithi106 ceph-mon[32716]: pgmap v865: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:53:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.382+0000 7f564082a640 1 -- 172.21.15.33:0/1843865935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 msgr2=0x7f563c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:17.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.382+0000 7f564082a640 1 --2- 172.21.15.33:0/1843865935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1014d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5630009900 tx=0x7f563002a410 comp rx=0 tx=0).stop 2024-04-23T18:53:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.382+0000 7f564082a640 1 -- 172.21.15.33:0/1843865935 shutdown_connections 2024-04-23T18:53:17.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.382+0000 7f564082a640 1 --2- 172.21.15.33:0/1843865935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c106170 0x7f563c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.382+0000 7f564082a640 1 --2- 172.21.15.33:0/1843865935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c101aa0 0x7f563c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.383+0000 7f564082a640 1 --2- 172.21.15.33:0/1843865935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1014d0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:17.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.383+0000 7f564082a640 1 -- 172.21.15.33:0/1843865935 >> 172.21.15.33:0/1843865935 conn(0x7f563c077fd0 msgr2=0x7f563c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.383+0000 7f564082a640 1 -- 172.21.15.33:0/1843865935 shutdown_connections 2024-04-23T18:53:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.383+0000 7f564082a640 1 -- 172.21.15.33:0/1843865935 wait complete. 2024-04-23T18:53:17.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.383+0000 7f564082a640 1 Processor -- start 2024-04-23T18:53:17.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 -- start start 2024-04-23T18:53:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c106170 0x7f563c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:17.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c19be60 0x7f563c19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1110e0 con 0x7f563c1010f0 2024-04-23T18:53:17.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f563c110f60 con 0x7f563c106170 2024-04-23T18:53:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f564082a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f563c111260 con 0x7f563c19be60 2024-04-23T18:53:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58370/0 (socket says 172.21.15.33:58370) 2024-04-23T18:53:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 -- 172.21.15.33:0/738880358 learned_addr learned my addr 172.21.15.33:0/738880358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:18.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f5639d9b640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c106170 0x7f563c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563ad9d640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c19be60 0x7f563c19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 -- 172.21.15.33:0/738880358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c19be60 msgr2=0x7f563c19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c19be60 0x7f563c19c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:18.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 -- 172.21.15.33:0/738880358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c106170 msgr2=0x7f563c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c106170 0x7f563c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.384+0000 7f563a59c640 1 -- 172.21.15.33:0/738880358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c19c950 con 0x7f563c1010f0 2024-04-23T18:53:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f563a59c640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5630005940 tx=0x7f56300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f562b7fe640 1 -- 172.21.15.33:0/738880358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563003f030 con 0x7f563c1010f0 2024-04-23T18:53:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f562b7fe640 1 -- 172.21.15.33:0/738880358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300044c0 con 0x7f563c1010f0 2024-04-23T18:53:18.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c19cbe0 con 0x7f563c1010f0 2024-04-23T18:53:18.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f562b7fe640 1 -- 172.21.15.33:0/738880358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630035070 con 0x7f563c1010f0 2024-04-23T18:53:18.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.385+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1a4400 con 0x7f563c1010f0 2024-04-23T18:53:18.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.387+0000 7f562b7fe640 1 -- 172.21.15.33:0/738880358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5630042030 con 0x7f563c1010f0 2024-04-23T18:53:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.387+0000 7f5643619740 1 Processor -- start 2024-04-23T18:53:18.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.387+0000 7f5643619740 1 -- start start 2024-04-23T18:53:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab732a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:18.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 0x55fbab73d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbab697970 con 0x55fbab732620 2024-04-23T18:53:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fbab6344a0 con 0x55fbab732f40 2024-04-23T18:53:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5643619740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fbab693930 con 0x55fbab73d800 2024-04-23T18:53:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab732a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5639d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 0x55fbab73d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5639d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:41808/0 (socket says 172.21.15.33:41808) 2024-04-23T18:53:19.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5639d9b640 1 -- 172.21.15.33:0/709102246 learned_addr learned my addr 172.21.15.33:0/709102246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:19.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f562b7fe640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 0x7f56240a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:19.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563ad9d640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 0x7f56240a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 -- 172.21.15.33:0/709102246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 msgr2=0x55fbab73fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 -- 172.21.15.33:0/709102246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 msgr2=0x55fbab732a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab732a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f563a59c640 1 -- 172.21.15.33:0/709102246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab3aa210 con 0x55fbab732f40 2024-04-23T18:53:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f5639d9b640 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:53:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.388+0000 7f562b7fe640 1 -- 172.21.15.33:0/738880358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f56300f0b30 con 0x7f563c1010f0 2024-04-23T18:53:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f563a59c640 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 0x55fbab73d2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56300ee210 tx=0x7f563003b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cf8c5020cbb54b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f563ad9d640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 0x7f56240a6300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5634009fd0 tx=0x7f5634004ab0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:19.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f5628ff9640 1 -- 172.21.15.33:0/709102246 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300bcb10 con 0x55fbab732f40 2024-04-23T18:53:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f5628ff9640 1 -- 172.21.15.33:0/709102246 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300c0e10 con 0x55fbab732f40 2024-04-23T18:53:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f5628ff9640 1 -- 172.21.15.33:0/709102246 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56300b9310 con 0x55fbab732f40 2024-04-23T18:53:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.389+0000 7f5643619740 1 -- 172.21.15.33:0/709102246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 msgr2=0x55fbab73d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 0x55fbab73d2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f56300ee210 tx=0x7f563003b460 comp rx=0 tx=0).stop 2024-04-23T18:53:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- 172.21.15.33:0/709102246 shutdown_connections 2024-04-23T18:53:20.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:20.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732f40 0x55fbab73d2c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- 172.21.15.33:0/709102246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab732a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:20.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- 172.21.15.33:0/709102246 >> 172.21.15.33:0/709102246 conn(0x55fbab72eb30 msgr2=0x55fbab72ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- 172.21.15.33:0/709102246 shutdown_connections 2024-04-23T18:53:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- 172.21.15.33:0/709102246 wait complete. 2024-04-23T18:53:20.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- start start 2024-04-23T18:53:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 Processor -- start 2024-04-23T18:53:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 0x55fbab7454e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:20.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:20.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fbab7401f0 con 0x55fbab732f40 2024-04-23T18:53:20.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fbab6320b0 con 0x55fbab732620 2024-04-23T18:53:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fbab740370 con 0x55fbab73d800 2024-04-23T18:53:21.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.390+0000 7f5643619740 10 client.0 _ll_register_callbacks cb 0x55fbab61fd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5639d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 0x55fbab7454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:21.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5639d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 0x55fbab7454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58382/0 (socket says 172.21.15.33:58382) 2024-04-23T18:53:21.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:54676/0 (socket says 172.21.15.33:54676) 2024-04-23T18:53:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5639d9b640 1 -- 172.21.15.33:0/1174332263 learned_addr learned my addr 172.21.15.33:0/1174332263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:21.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 -- 172.21.15.33:0/1174332263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 msgr2=0x55fbab73c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 -- 172.21.15.33:0/1174332263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 msgr2=0x55fbab7454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 0x55fbab7454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 -- 172.21.15.33:0/1174332263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbab476620 con 0x55fbab732620 2024-04-23T18:53:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f563a59c640 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56300ee210 tx=0x7f563002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911f92fe1b5ad7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:53:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630049040 con 0x55fbab732620 2024-04-23T18:53:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f563002c140 con 0x55fbab732620 2024-04-23T18:53:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56300c0e10 con 0x55fbab732620 2024-04-23T18:53:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f56300ee6a0 con 0x55fbab732620 2024-04-23T18:53:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5643619740 10 client.27410 fetch_fsmap learned FSMap version 44 2024-04-23T18:53:22.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.391+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fbab7401f0 con 0x55fbab732620 2024-04-23T18:53:22.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56300c0e10 con 0x55fbab732620 2024-04-23T18:53:22.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f56300c5240 con 0x55fbab732620 2024-04-23T18:53:22.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 10 client.27410 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:53:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 10 client.27410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 20 client.27410 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:22.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab74f3c0 0x55fbab7517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:22.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 4 client.27410 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:53:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbab6a5390 con 0x55fbab74f3c0 2024-04-23T18:53:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5639d9b640 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab74f3c0 0x55fbab7517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5621ffb640 10 client.27410 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:22.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.392+0000 7f5639d9b640 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab74f3c0 0x55fbab7517b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f4b6c2d44616a8 server_cookie=5cdc03506d4c8c9e in_seq=0 out_seq=0 2024-04-23T18:53:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5621ffb640 1 -- 172.21.15.33:0/1174332263 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbab6a5390 con 0x55fbab74f3c0 2024-04-23T18:53:22.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5621ffb640 10 client.27410 handle_command_reply: tid=0 2024-04-23T18:53:22.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 client.27410 shutdown 2024-04-23T18:53:22.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 msgr2=0x55fbab744fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:22.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56300ee210 tx=0x7f563002a410 comp rx=0 tx=0).stop 2024-04-23T18:53:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 shutdown_connections 2024-04-23T18:53:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab74f3c0 0x55fbab7517b0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab73d800 0x55fbab73c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5621ffb640 0 client.27410 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:22.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732f40 0x55fbab7454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 --2- 172.21.15.33:0/1174332263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab744fa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 >> 172.21.15.33:0/1174332263 conn(0x55fbab72eb30 msgr2=0x55fbab7348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 shutdown_connections 2024-04-23T18:53:23.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 1 -- 172.21.15.33:0/1174332263 wait complete. 2024-04-23T18:53:23.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.393+0000 7f5643619740 20 client.27410 trim_cache size 0 max 16384 2024-04-23T18:53:23.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 Processor -- start 2024-04-23T18:53:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 -- start start 2024-04-23T18:53:23.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab7453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:23.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab72db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1ad3c0 con 0x55fbab732620 2024-04-23T18:53:23.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1ab1b0 con 0x55fbab73d800 2024-04-23T18:53:23.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5643619740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1accf0 con 0x55fbab732f40 2024-04-23T18:53:23.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:23.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:54680/0 (socket says 172.21.15.33:54680) 2024-04-23T18:53:23.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f563ad9d640 1 -- 172.21.15.33:0/793845221 learned_addr learned my addr 172.21.15.33:0/793845221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:23.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f5639d9b640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab72db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.416+0000 7f563a59c640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab7453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:23.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 -- 172.21.15.33:0/793845221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 msgr2=0x55fbab757020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:23.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:23.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 -- 172.21.15.33:0/793845221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 msgr2=0x55fbab7453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab7453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:23.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 -- 172.21.15.33:0/793845221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab751d90 con 0x55fbab732f40 2024-04-23T18:53:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f563ad9d640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:53:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f563a59c640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab7453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5639d9b640 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab72db30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55fbab632200 tx=0x7f562c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d36cb9f5c09c9d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:24.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5623fff640 1 -- 172.21.15.33:0/793845221 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0200f0 con 0x55fbab732f40 2024-04-23T18:53:24.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5623fff640 1 -- 172.21.15.33:0/793845221 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c020740 con 0x55fbab732f40 2024-04-23T18:53:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.417+0000 7f5623fff640 1 -- 172.21.15.33:0/793845221 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c010b50 con 0x55fbab732f40 2024-04-23T18:53:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- 172.21.15.33:0/793845221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 msgr2=0x55fbab72db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:24.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab72db30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55fbab632200 tx=0x7f562c005f90 comp rx=0 tx=0).stop 2024-04-23T18:53:24.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- 172.21.15.33:0/793845221 shutdown_connections 2024-04-23T18:53:24.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab73d800 0x55fbab757020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab72db30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:24.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 --2- 172.21.15.33:0/793845221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab732620 0x55fbab7453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- 172.21.15.33:0/793845221 >> 172.21.15.33:0/793845221 conn(0x55fbab72eb30 msgr2=0x55fbab72d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:24.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- 172.21.15.33:0/793845221 shutdown_connections 2024-04-23T18:53:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- 172.21.15.33:0/793845221 wait complete. 2024-04-23T18:53:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 -- start start 2024-04-23T18:53:24.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.418+0000 7f5643619740 1 Processor -- start 2024-04-23T18:53:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab7a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:24.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab73d800 0x55fbab7aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1abf50 con 0x55fbab73d800 2024-04-23T18:53:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1ab880 con 0x55fbab732620 2024-04-23T18:53:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1ac620 con 0x55fbab732f40 2024-04-23T18:53:25.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 10 client.0 _ll_register_callbacks cb 0x7f5630086bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:25.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:25.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab73d800 0x55fbab7aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:25.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.33:54684/0 (socket says 172.21.15.33:54684) 2024-04-23T18:53:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 -- 172.21.15.33:0/3693406432 learned_addr learned my addr 172.21.15.33:0/3693406432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5639d9b640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab7a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 -- 172.21.15.33:0/3693406432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 msgr2=0x55fbab7a9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab7a9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 -- 172.21.15.33:0/3693406432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab73d800 msgr2=0x55fbab7aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab73d800 0x55fbab7aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 -- 172.21.15.33:0/3693406432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbab74bd30 con 0x55fbab732620 2024-04-23T18:53:25.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f563a59c640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56300c51a0 tx=0x7f5630047fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa9cdd64b6d154b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:25.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5621ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-23T18:53:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630049040 con 0x55fbab732620 2024-04-23T18:53:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 8542+0+0 (secure 0 0 0) 0x7f56300bcd70 con 0x55fbab732620 2024-04-23T18:53:25.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562c020740 con 0x55fbab732620 2024-04-23T18:53:25.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.419+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f56300c0e10 con 0x55fbab732620 2024-04-23T18:53:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 10 client.27416 fetch_fsmap learned FSMap version 44 2024-04-23T18:53:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fbab74bd30 con 0x55fbab732620 2024-04-23T18:53:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c020740 con 0x55fbab732620 2024-04-23T18:53:26.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 44) ==== 7749+0+0 (secure 0 0 0) 0x7f56300b8a40 con 0x55fbab732620 2024-04-23T18:53:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 10 client.27416 fetch_fsmap finished waiting for FSMap version 44 2024-04-23T18:53:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 10 client.27416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 20 client.27416 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab7b5f30 0x55fbab7b8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 4 client.27416 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:53:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbab69f200 con 0x55fbab7b5f30 2024-04-23T18:53:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5639d9b640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab7b5f30 0x55fbab7b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:26.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5639d9b640 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab7b5f30 0x55fbab7b8320 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84da06adf8734c1d server_cookie=2c5d1c81f8817462 in_seq=0 out_seq=0 2024-04-23T18:53:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:01.420+0000 7f5621ffb640 10 client.27416 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:26.608 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:01 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:53:26.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:02 smithi033 ceph-mon[29274]: pgmap v822: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T18:53:26.695 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:02 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:53:26.719 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:02 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:53:26.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:02 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:53:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5621ffb640 1 -- 172.21.15.33:0/3693406432 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbab69f200 con 0x55fbab7b5f30 2024-04-23T18:53:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5621ffb640 10 client.27416 handle_command_reply: tid=0 2024-04-23T18:53:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 client.27416 shutdown 2024-04-23T18:53:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 msgr2=0x55fbab7a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56300c51a0 tx=0x7f5630047fa0 comp rx=0 tx=0).stop 2024-04-23T18:53:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 shutdown_connections 2024-04-23T18:53:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55fbab7b5f30 0x55fbab7b8320 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fbab73d800 0x55fbab7aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:27.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fbab732f40 0x55fbab7a9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5621ffb640 0 client.27416 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:27.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 --2- 172.21.15.33:0/3693406432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fbab732620 0x55fbab7a9830 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 >> 172.21.15.33:0/3693406432 conn(0x55fbab72eb30 msgr2=0x55fbab72dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 shutdown_connections 2024-04-23T18:53:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 1 -- 172.21.15.33:0/3693406432 wait complete. 2024-04-23T18:53:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.197+0000 7f5643619740 20 client.27416 trim_cache size 0 max 16384 2024-04-23T18:53:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:53:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:53:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:53:27.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:53:27.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:17 smithi122 ceph-mon[32546]: pgmap v866: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:53:27.784 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:17 smithi122 ceph-mon[32546]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:53:27.838 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:19 smithi122 ceph-mon[32546]: pgmap v868: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 KiB/s wr, 12 op/s 2024-04-23T18:53:27.847 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:21 smithi122 ceph-mon[32546]: pgmap v869: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:53:27.908 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:23 smithi122 ceph-mon[32546]: pgmap v870: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:27.993 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:25 smithi122 ceph-mon[32546]: pgmap v871: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:28.002 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:27 smithi122 ceph-mon[32546]: pgmap v872: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:53:28.010 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:17 smithi106 ceph-mon[32716]: pgmap v866: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:53:28.093 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:17 smithi106 ceph-mon[32716]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:53:28.102 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:19 smithi106 ceph-mon[32716]: pgmap v868: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 KiB/s wr, 12 op/s 2024-04-23T18:53:28.164 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:21 smithi106 ceph-mon[32716]: pgmap v869: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:53:28.173 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:23 smithi106 ceph-mon[32716]: pgmap v870: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:28.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:25 smithi106 ceph-mon[32716]: pgmap v871: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:53:28.263 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:27 smithi106 ceph-mon[32716]: pgmap v872: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:53:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 msgr2=0x7f56240a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 0x7f56240a6300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5634009fd0 tx=0x7f5634004ab0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 msgr2=0x7f563c1975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5630005940 tx=0x7f56300040c0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 shutdown_connections 2024-04-23T18:53:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f56240a3e40 0x7f56240a6300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f563c19be60 0x7f563c19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f563c106170 0x7f563c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 --2- 172.21.15.33:0/738880358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c1010f0 0x7f563c1975a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 >> 172.21.15.33:0/738880358 conn(0x7f563c077fd0 msgr2=0x7f563c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 shutdown_connections 2024-04-23T18:53:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:03.198+0000 7f564082a640 1 -- 172.21.15.33:0/738880358 wait complete. 2024-04-23T18:53:28.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-mon[29274]: pgmap v823: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 49 KiB/s wr, 28 op/s 2024-04-23T18:53:28.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-mon[29274]: pgmap v824: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 60 KiB/s wr, 34 op/s 2024-04-23T18:53:28.798 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:53:28.857 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: in thread 7f3560cb9700 thread_name:md_log_replay 2024-04-23T18:53:28.901 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:53:28.988 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f357207ad20] 2024-04-23T18:53:29.054 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f9446a038f] 2024-04-23T18:53:29.070 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f9446a3b23] 2024-04-23T18:53:29.159 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 4: (MDLog::_replay_thread()+0x617) [0x55f9446fe647] 2024-04-23T18:53:29.260 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f944707d31] 2024-04-23T18:53:29.267 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f35720701ca] 2024-04-23T18:53:29.276 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 7: clone() 2024-04-23T18:53:29.366 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:52:03.203+0000 7f3560cb9700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:53:29.373 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: in thread 7f3560cb9700 thread_name:md_log_replay 2024-04-23T18:53:29.382 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:29.390 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:53:29.397 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f357207ad20] 2024-04-23T18:53:29.404 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f9446a038f] 2024-04-23T18:53:29.411 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f9446a3b23] 2024-04-23T18:53:29.418 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 4: (MDLog::_replay_thread()+0x617) [0x55f9446fe647] 2024-04-23T18:53:29.493 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f944707d31] 2024-04-23T18:53:29.596 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f35720701ca] 2024-04-23T18:53:29.657 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 7: clone() 2024-04-23T18:53:29.698 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:53:29.706 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:29.816 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 0> 2024-04-23T18:52:03.203+0000 7f3560cb9700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:53:29.874 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: in thread 7f3560cb9700 thread_name:md_log_replay 2024-04-23T18:53:29.912 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:29.921 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:53:30.001 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f357207ad20] 2024-04-23T18:53:30.065 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f9446a038f] 2024-04-23T18:53:30.154 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f9446a3b23] 2024-04-23T18:53:30.161 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 4: (MDLog::_replay_thread()+0x617) [0x55f9446fe647] 2024-04-23T18:53:30.225 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f944707d31] 2024-04-23T18:53:30.232 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f35720701ca] 2024-04-23T18:53:30.337 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 7: clone() 2024-04-23T18:53:31.124 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:53:31.246 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:31.254 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 0> 2024-04-23T18:52:03.203+0000 7f3560cb9700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:53:31.304 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: in thread 7f3560cb9700 thread_name:md_log_replay 2024-04-23T18:53:31.354 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:31.427 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:53:31.436 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f357207ad20] 2024-04-23T18:53:31.547 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55f9446a038f] 2024-04-23T18:53:31.607 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55f9446a3b23] 2024-04-23T18:53:31.683 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 4: (MDLog::_replay_thread()+0x617) [0x55f9446fe647] 2024-04-23T18:53:31.775 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55f944707d31] 2024-04-23T18:53:31.784 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f35720701ca] 2024-04-23T18:53:31.793 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 7: clone() 2024-04-23T18:53:31.896 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:53:31.905 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d[96572]: 2024-04-23T18:53:31.971 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:03 smithi033 podman[97311]: 2024-04-23 18:52:03.263534558 +0000 UTC m=+0.024528514 container died bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T18:53:31.981 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:04 smithi033 ceph-mon[29274]: osdmap e137: 12 total, 12 up, 12 in 2024-04-23T18:53:32.045 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:29 smithi122 ceph-mon[32546]: pgmap v873: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:32.071 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:31 smithi122 ceph-mon[32546]: pgmap v874: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:32.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:29 smithi106 ceph-mon[32716]: pgmap v873: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:32.319 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:31 smithi106 ceph-mon[32716]: pgmap v874: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:32.329 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:04 smithi033 podman[97311]: 2024-04-23 18:52:04.070418056 +0000 UTC m=+0.831412015 container remove bf8e3dec13b9b309a2a93067328078ce10a069f28fda7cf650eccbe1a2902e9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T18:53:32.421 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:53:32.432 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:53:32.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:05 smithi033 ceph-mon[29274]: pgmap v826: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:53:32.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:07 smithi033 ceph-mon[29274]: pgmap v827: 97 pgs: 97 active+clean; 15 MiB data, 472 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 78 KiB/s wr, 37 op/s 2024-04-23T18:53:32.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:09 smithi033 ceph-mon[29274]: pgmap v828: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:53:32.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:11 smithi033 ceph-mon[29274]: pgmap v829: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:53:32.639 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:11 smithi033 ceph-mon[29274]: osdmap e138: 12 total, 12 up, 12 in 2024-04-23T18:53:32.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:13 smithi033 ceph-mon[29274]: pgmap v831: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-04-23T18:53:32.756 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 6. 2024-04-23T18:53:32.764 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:14 smithi033 systemd[1]: Stopped Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:53:32.824 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Start request repeated too quickly. 2024-04-23T18:53:32.872 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-23T18:53:32.935 INFO:journalctl@ceph.mds.d.smithi033.stdout:Apr 23 18:52:14 smithi033 systemd[1]: Failed to start Ceph mds.d for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:53:32.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:15 smithi033 ceph-mon[29274]: pgmap v832: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:53:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 -- 172.21.15.33:0/2064265500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807410c5d0 msgr2=0x7f807410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2064265500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807410c5d0 0x7f807410ea00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f806c009ca0 tx=0x7f806c02a950 comp rx=0 tx=0).stop 2024-04-23T18:53:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 -- 172.21.15.33:0/2064265500 shutdown_connections 2024-04-23T18:53:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2064265500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f807410c5d0 0x7f807410ea00 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2064265500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8074103b20 0x7f807410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:33.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2064265500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103200 0x7f80741035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:33.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.296+0000 7f807c2a0640 1 -- 172.21.15.33:0/2064265500 >> 172.21.15.33:0/2064265500 conn(0x7f80740771d0 msgr2=0x7f80740775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.297+0000 7f807c2a0640 1 -- 172.21.15.33:0/2064265500 shutdown_connections 2024-04-23T18:53:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.297+0000 7f807c2a0640 1 -- 172.21.15.33:0/2064265500 wait complete. 2024-04-23T18:53:33.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.297+0000 7f807c2a0640 1 Processor -- start 2024-04-23T18:53:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.297+0000 7f807c2a0640 1 -- start start 2024-04-23T18:53:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 0x7f80741a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807410c5d0 0x7f807419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80741121d0 con 0x7f8074103200 2024-04-23T18:53:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8074112050 con 0x7f807410c5d0 2024-04-23T18:53:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8074112350 con 0x7f8074103b20 2024-04-23T18:53:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:33.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47114/0 (socket says 172.21.15.33:47114) 2024-04-23T18:53:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 -- 172.21.15.33:0/2127897620 learned_addr learned my addr 172.21.15.33:0/2127897620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:33.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a83d640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807410c5d0 0x7f807419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807983b640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 0x7f80741a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:33.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 -- 172.21.15.33:0/2127897620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 msgr2=0x7f80741a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 0x7f80741a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 -- 172.21.15.33:0/2127897620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807410c5d0 msgr2=0x7f807419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:33.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807410c5d0 0x7f807419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 -- 172.21.15.33:0/2127897620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807419af40 con 0x7f8074103200 2024-04-23T18:53:34.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807983b640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 0x7f80741a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807a03c640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f807000c890 tx=0x7f807000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f806b7fe640 1 -- 172.21.15.33:0/2127897620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070004220 con 0x7f8074103200 2024-04-23T18:53:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807419b230 con 0x7f8074103200 2024-04-23T18:53:34.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f806b7fe640 1 -- 172.21.15.33:0/2127897620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80700043c0 con 0x7f8074103200 2024-04-23T18:53:34.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.298+0000 7f806b7fe640 1 -- 172.21.15.33:0/2127897620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070004a00 con 0x7f8074103200 2024-04-23T18:53:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.299+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80741a7540 con 0x7f8074103200 2024-04-23T18:53:34.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.301+0000 7f806b7fe640 1 -- 172.21.15.33:0/2127897620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f80700079b0 con 0x7f8074103200 2024-04-23T18:53:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.301+0000 7f807efbb740 1 Processor -- start 2024-04-23T18:53:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.301+0000 7f807efbb740 1 -- start start 2024-04-23T18:53:34.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a307039970 con 0x55a3070d4620 2024-04-23T18:53:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a306fd64a0 con 0x55a3070df800 2024-04-23T18:53:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807efbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a307035930 con 0x55a3070d4f40 2024-04-23T18:53:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f806b7fe640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 0x7f80600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:34.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47130/0 (socket says 172.21.15.33:47130) 2024-04-23T18:53:35.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 -- 172.21.15.33:0/1709644455 learned_addr learned my addr 172.21.15.33:0/1709644455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807983b640 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:35.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a03c640 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 0x7f80600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f806b7fe640 1 -- 172.21.15.33:0/2127897620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f80700d0580 con 0x7f8074103200 2024-04-23T18:53:35.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 -- 172.21.15.33:0/1709644455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 msgr2=0x55a3070df2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070df2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:35.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 -- 172.21.15.33:0/1709644455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 msgr2=0x55a3070e1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:35.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 -- 172.21.15.33:0/1709644455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a306d4c210 con 0x55a3070d4620 2024-04-23T18:53:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.302+0000 7f807a83d640 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f806c004ad0 tx=0x7f806c02fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b7ed46a79650ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:35.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f8068bfe640 1 -- 172.21.15.33:0/1709644455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c043050 con 0x55a3070d4620 2024-04-23T18:53:35.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f8068bfe640 1 -- 172.21.15.33:0/1709644455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806c044040 con 0x55a3070d4620 2024-04-23T18:53:35.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f8068bfe640 1 -- 172.21.15.33:0/1709644455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c04d020 con 0x55a3070d4620 2024-04-23T18:53:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 -- 172.21.15.33:0/1709644455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 msgr2=0x55a3070d4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:35.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807a83d640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 0x7f80600a6280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f806c04e000 tx=0x7f806c04c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:35.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f806c004ad0 tx=0x7f806c02fc40 comp rx=0 tx=0).stop 2024-04-23T18:53:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 -- 172.21.15.33:0/1709644455 shutdown_connections 2024-04-23T18:53:35.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:35.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070df2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:35.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 --2- 172.21.15.33:0/1709644455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070d4a00 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 -- 172.21.15.33:0/1709644455 >> 172.21.15.33:0/1709644455 conn(0x55a3070d0b30 msgr2=0x55a3070d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:36.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 -- 172.21.15.33:0/1709644455 shutdown_connections 2024-04-23T18:53:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.303+0000 7f807efbb740 1 -- 172.21.15.33:0/1709644455 wait complete. 2024-04-23T18:53:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 -- start start 2024-04-23T18:53:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 Processor -- start 2024-04-23T18:53:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 0x55a3070df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:36.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a3070e2430 con 0x55a3070df800 2024-04-23T18:53:36.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a306fd40b0 con 0x55a3070d4620 2024-04-23T18:53:36.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a3070e25b0 con 0x55a3070d4f40 2024-04-23T18:53:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807efbb740 10 client.0 _ll_register_callbacks cb 0x55a306fc1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47140/0 (socket says 172.21.15.33:47140) 2024-04-23T18:53:36.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 -- 172.21.15.33:0/1461904817 learned_addr learned my addr 172.21.15.33:0/1461904817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:36.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a03c640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 0x55a3070df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:36.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807983b640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 -- 172.21.15.33:0/1461904817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 msgr2=0x55a3070dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070dc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:37.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 -- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 msgr2=0x55a3070df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:37.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 0x55a3070df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:37.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.304+0000 7f807a83d640 1 -- 172.21.15.33:0/1461904817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a306e18620 con 0x55a3070df800 2024-04-23T18:53:37.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807a03c640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 0x55a3070df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:37.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807a83d640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f806c0045c0 tx=0x7f806c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69075ab71f8d91e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:37.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c04d070 con 0x55a3070df800 2024-04-23T18:53:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f806c0497f0 con 0x55a3070df800 2024-04-23T18:53:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f806c044040 con 0x55a3070df800 2024-04-23T18:53:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f806c034040 con 0x55a3070df800 2024-04-23T18:53:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 10 client.18234 fetch_fsmap learned FSMap version 45 2024-04-23T18:53:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a3070ee380 con 0x55a3070df800 2024-04-23T18:53:37.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f806c044040 con 0x55a3070df800 2024-04-23T18:53:37.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f806c05b630 con 0x55a3070df800 2024-04-23T18:53:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 10 client.18234 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:53:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 10 client.18234 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 20 client.18234 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:37.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a3070f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 4 client.18234 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:53:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.305+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a307047390 con 0x55a3070ef6d0 2024-04-23T18:53:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.306+0000 7f807a03c640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a3070f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.306+0000 7f805dffb640 10 client.18234 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.306+0000 7f807a03c640 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a3070f1ac0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f3f4ce56ca60c8b server_cookie=cd3fad6dea2aced2 in_seq=0 out_seq=0 2024-04-23T18:53:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f805dffb640 1 -- 172.21.15.33:0/1461904817 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a307047390 con 0x55a3070ef6d0 2024-04-23T18:53:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f805dffb640 10 client.18234 handle_command_reply: tid=0 2024-04-23T18:53:38.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 client.18234 shutdown 2024-04-23T18:53:38.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 msgr2=0x55a3070dc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:38.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f806c0045c0 tx=0x7f806c005f70 comp rx=0 tx=0).stop 2024-04-23T18:53:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 shutdown_connections 2024-04-23T18:53:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a3070f1ac0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:38.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070df800 0x55a3070dc960 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:38.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f805dffb640 0 client.18234 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:38.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 --2- 172.21.15.33:0/1461904817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4620 0x55a3070df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 >> 172.21.15.33:0/1461904817 conn(0x55a3070d0b30 msgr2=0x55a3070d6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 shutdown_connections 2024-04-23T18:53:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 1 -- 172.21.15.33:0/1461904817 wait complete. 2024-04-23T18:53:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.307+0000 7f807efbb740 20 client.18234 trim_cache size 0 max 16384 2024-04-23T18:53:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 Processor -- start 2024-04-23T18:53:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 -- start start 2024-04-23T18:53:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:39.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 0x55a3070d7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 0x55a3070dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80741ae8d0 con 0x55a3070d4620 2024-04-23T18:53:39.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80741b11b0 con 0x55a3070d4f40 2024-04-23T18:53:39.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807efbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80741aefa0 con 0x55a3070df800 2024-04-23T18:53:39.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47144/0 (socket says 172.21.15.33:47144) 2024-04-23T18:53:39.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 -- 172.21.15.33:0/1528370277 learned_addr learned my addr 172.21.15.33:0/1528370277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:39.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807983b640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 0x55a3070d7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 -- 172.21.15.33:0/1528370277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 msgr2=0x55a3070dec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a83d640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 0x55a3070dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:39.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 0x55a3070dec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 -- 172.21.15.33:0/1528370277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 msgr2=0x55a3070d7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 0x55a3070d7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a03c640 1 -- 172.21.15.33:0/1528370277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3070f1e30 con 0x55a3070d4620 2024-04-23T18:53:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807983b640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 0x55a3070d7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:53:39.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.322+0000 7f807a83d640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 0x55a3070dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:53:39.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807a03c640 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f807006bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d76d389d29b30f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:39.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f805ffff640 1 -- 172.21.15.33:0/1528370277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807001d040 con 0x55a3070d4620 2024-04-23T18:53:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f805ffff640 1 -- 172.21.15.33:0/1528370277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807009cbf0 con 0x55a3070d4620 2024-04-23T18:53:39.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f805ffff640 1 -- 172.21.15.33:0/1528370277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070098b60 con 0x55a3070d4620 2024-04-23T18:53:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 -- 172.21.15.33:0/1528370277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 msgr2=0x55a3070eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c00 tx=0x7f807006bdd0 comp rx=0 tx=0).stop 2024-04-23T18:53:40.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 -- 172.21.15.33:0/1528370277 shutdown_connections 2024-04-23T18:53:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070df800 0x55a3070dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070d4f40 0x55a3070d7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 --2- 172.21.15.33:0/1528370277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070eb2f0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:40.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 -- 172.21.15.33:0/1528370277 >> 172.21.15.33:0/1528370277 conn(0x55a3070d0b30 msgr2=0x55a3070d7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 -- 172.21.15.33:0/1528370277 shutdown_connections 2024-04-23T18:53:40.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.323+0000 7f807efbb740 1 -- 172.21.15.33:0/1528370277 wait complete. 2024-04-23T18:53:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 -- start start 2024-04-23T18:53:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 Processor -- start 2024-04-23T18:53:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:40.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:40.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80741af670 con 0x55a3070d4620 2024-04-23T18:53:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80741b0ae0 con 0x55a3070df800 2024-04-23T18:53:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80741afd40 con 0x55a3070d4f40 2024-04-23T18:53:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807efbb740 10 client.0 _ll_register_callbacks cb 0x7f806c02fa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:40.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47158/0 (socket says 172.21.15.33:47158) 2024-04-23T18:53:40.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 -- 172.21.15.33:0/64089425 learned_addr learned my addr 172.21.15.33:0/64089425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a83d640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:40.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 -- 172.21.15.33:0/64089425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 msgr2=0x55a3070e2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:41.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:41.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 -- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 msgr2=0x55a3070e34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:41.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 -- 172.21.15.33:0/64089425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3070e3ce0 con 0x55a3070d4620 2024-04-23T18:53:41.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a83d640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:41.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.324+0000 7f807a03c640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f80700a4c30 tx=0x7f80700a51d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c732876470ff38 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:41.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807001d070 con 0x55a3070d4620 2024-04-23T18:53:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f8070098b60 con 0x55a3070d4620 2024-04-23T18:53:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f807009cbf0 con 0x55a3070d4620 2024-04-23T18:53:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f807009c4f0 con 0x55a3070d4620 2024-04-23T18:53:41.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 10 client.18240 fetch_fsmap learned FSMap version 45 2024-04-23T18:53:41.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a307155a90 con 0x55a3070d4620 2024-04-23T18:53:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f807009cbf0 con 0x55a3070d4620 2024-04-23T18:53:41.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f80700b2be0 con 0x55a3070d4620 2024-04-23T18:53:41.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 10 client.18240 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:53:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 10 client.18240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 20 client.18240 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:41.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a307158060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 4 client.18240 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:53:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3070dccf0 con 0x55a3070ef6d0 2024-04-23T18:53:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.325+0000 7f807983b640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a307158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.326+0000 7f807983b640 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a307158060 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eee66930f6792ee server_cookie=fbce92be7ed1bafe in_seq=0 out_seq=0 2024-04-23T18:53:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.326+0000 7f805dffb640 10 client.18240 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f805dffb640 1 -- 172.21.15.33:0/64089425 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3070dccf0 con 0x55a3070ef6d0 2024-04-23T18:53:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f805dffb640 10 client.18240 handle_command_reply: tid=0 2024-04-23T18:53:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 client.18240 shutdown 2024-04-23T18:53:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 msgr2=0x55a3070e7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:42.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f80700a4c30 tx=0x7f80700a51d0 comp rx=0 tx=0).stop 2024-04-23T18:53:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 shutdown_connections 2024-04-23T18:53:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a3070ef6d0 0x55a307158060 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:42.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a3070df800 0x55a3070e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:42.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3070d4f40 0x55a3070e2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 --2- 172.21.15.33:0/64089425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3070d4620 0x55a3070e7bb0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.332+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 >> 172.21.15.33:0/64089425 conn(0x55a3070d0b30 msgr2=0x55a3070cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:42.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f805dffb640 0 client.18240 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 shutdown_connections 2024-04-23T18:53:42.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807efbb740 1 -- 172.21.15.33:0/64089425 wait complete. 2024-04-23T18:53:42.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807efbb740 20 client.18240 trim_cache size 0 max 16384 2024-04-23T18:53:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:53:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:53:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:53:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:53:43.034 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:33 smithi122 ceph-mon[32546]: pgmap v875: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:53:43.042 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:35 smithi122 ceph-mon[32546]: pgmap v876: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:43.050 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:37 smithi122 ceph-mon[32546]: pgmap v877: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:53:43.059 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:39 smithi122 ceph-mon[32546]: pgmap v878: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:43.132 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:41 smithi122 ceph-mon[32546]: pgmap v879: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:43.210 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:33 smithi106 ceph-mon[32716]: pgmap v875: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:53:43.249 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:35 smithi106 ceph-mon[32716]: pgmap v876: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:43.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:37 smithi106 ceph-mon[32716]: pgmap v877: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:53:43.400 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:39 smithi106 ceph-mon[32716]: pgmap v878: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:43.456 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:41 smithi106 ceph-mon[32716]: pgmap v879: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:43.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 msgr2=0x7f80600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 0x7f80600a6280 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f806c04e000 tx=0x7f806c04c000 comp rx=0 tx=0).stop 2024-04-23T18:53:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 msgr2=0x7f80741a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:43.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f807000c890 tx=0x7f807000cd50 comp rx=0 tx=0).stop 2024-04-23T18:53:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 shutdown_connections 2024-04-23T18:53:43.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80600a3dc0 0x7f80600a6280 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:43.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f807410c5d0 0x7f807419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:43.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8074103b20 0x7f80741a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:43.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 --2- 172.21.15.33:0/2127897620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8074103200 0x7f80741a05c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:43.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 >> 172.21.15.33:0/2127897620 conn(0x7f80740771d0 msgr2=0x7f8074101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 shutdown_connections 2024-04-23T18:53:43.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:16.333+0000 7f807c2a0640 1 -- 172.21.15.33:0/2127897620 wait complete. 2024-04-23T18:53:43.832 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:16 smithi033 ceph-mon[29274]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-23T18:53:43.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:53:43.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:17 smithi033 ceph-mon[29274]: pgmap v833: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 921 B/s wr, 9 op/s 2024-04-23T18:53:44.110 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:19 smithi033 ceph-mon[29274]: pgmap v834: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:53:44.122 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:21 smithi033 ceph-mon[29274]: pgmap v835: 97 pgs: 97 active+clean; 15 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 102 B/s wr, 8 op/s 2024-04-23T18:53:44.172 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:23 smithi033 ceph-mon[29274]: pgmap v836: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 386 B/s wr, 9 op/s 2024-04-23T18:53:44.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:25 smithi033 ceph-mon[29274]: pgmap v837: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:44.401 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:27 smithi033 ceph-mon[29274]: pgmap v838: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:53:44.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:43 smithi122 ceph-mon[32546]: pgmap v880: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:44.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:43 smithi106 ceph-mon[32716]: pgmap v880: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:44.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 -- 172.21.15.33:0/3881564881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc103b20 msgr2=0x7f66dc10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 --2- 172.21.15.33:0/3881564881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc103b20 0x7f66dc10c000 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009900 tx=0x7f66d002a480 comp rx=0 tx=0).stop 2024-04-23T18:53:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 -- 172.21.15.33:0/3881564881 shutdown_connections 2024-04-23T18:53:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 --2- 172.21.15.33:0/3881564881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc10c5d0 0x7f66dc10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 --2- 172.21.15.33:0/3881564881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc103b20 0x7f66dc10c000 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:44.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 --2- 172.21.15.33:0/3881564881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103200 0x7f66dc1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 -- 172.21.15.33:0/3881564881 >> 172.21.15.33:0/3881564881 conn(0x7f66dc0771d0 msgr2=0x7f66dc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 -- 172.21.15.33:0/3881564881 shutdown_connections 2024-04-23T18:53:45.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.432+0000 7f66e268a640 1 -- 172.21.15.33:0/3881564881 wait complete. 2024-04-23T18:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.433+0000 7f66e268a640 1 Processor -- start 2024-04-23T18:53:45.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.433+0000 7f66e268a640 1 -- start start 2024-04-23T18:53:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc103200 0x7f66dc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:45.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103b20 0x7f66dc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:45.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112210 con 0x7f66dc10c5d0 2024-04-23T18:53:45.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112090 con 0x7f66dc103b20 2024-04-23T18:53:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e268a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc112390 con 0x7f66dc103200 2024-04-23T18:53:45.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:45.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46330/0 (socket says 172.21.15.33:46330) 2024-04-23T18:53:45.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 -- 172.21.15.33:0/466588646 learned_addr learned my addr 172.21.15.33:0/466588646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:45.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66dbfff640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc103200 0x7f66dc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:45.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66db7fe640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103b20 0x7f66dc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:45.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 -- 172.21.15.33:0/466588646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc103200 msgr2=0x7f66dc1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc103200 0x7f66dc1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 -- 172.21.15.33:0/466588646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103b20 msgr2=0x7f66dc1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103b20 0x7f66dc1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:46.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 -- 172.21.15.33:0/466588646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc19af70 con 0x7f66dc10c5d0 2024-04-23T18:53:46.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.434+0000 7f66e0c27640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f66d400eb40 tx=0x7f66d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:46.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.435+0000 7f66d97fa640 1 -- 172.21.15.33:0/466588646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4004b20 con 0x7f66dc10c5d0 2024-04-23T18:53:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.435+0000 7f66d97fa640 1 -- 172.21.15.33:0/466588646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d4004cc0 con 0x7f66dc10c5d0 2024-04-23T18:53:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.435+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc19b260 con 0x7f66dc10c5d0 2024-04-23T18:53:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.435+0000 7f66d97fa640 1 -- 172.21.15.33:0/466588646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4010720 con 0x7f66dc10c5d0 2024-04-23T18:53:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.435+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc1a7570 con 0x7f66dc10c5d0 2024-04-23T18:53:46.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.437+0000 7f66d97fa640 1 -- 172.21.15.33:0/466588646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66d4005560 con 0x7f66dc10c5d0 2024-04-23T18:53:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.437+0000 7f66e53a3740 1 Processor -- start 2024-04-23T18:53:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.437+0000 7f66e53a3740 1 -- start start 2024-04-23T18:53:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66d97fa640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 0x7f66c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05ef5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:46.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05f02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:46.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a05e5a970 con 0x562a05ef5f40 2024-04-23T18:53:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562a05df74a0 con 0x562a05f00800 2024-04-23T18:53:47.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e53a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562a05e56930 con 0x562a05ef5620 2024-04-23T18:53:47.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 0x7f66c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:47.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66d97fa640 1 -- 172.21.15.33:0/466588646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f66d40d0fb0 con 0x7f66dc10c5d0 2024-04-23T18:53:47.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46344/0 (socket says 172.21.15.33:46344) 2024-04-23T18:53:47.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 -- 172.21.15.33:0/521726223 learned_addr learned my addr 172.21.15.33:0/521726223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66e0c27640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05f02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05ef5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 -- 172.21.15.33:0/521726223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 msgr2=0x562a05f02bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:47.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05f02bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 -- 172.21.15.33:0/521726223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 msgr2=0x562a05f002c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66dbfff640 1 -- 172.21.15.33:0/521726223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a05b6d210 con 0x562a05ef5620 2024-04-23T18:53:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:53:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.438+0000 7f66db7fe640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 0x7f66c40a6280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66d002a9a0 tx=0x7f66d0005e80 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:47.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66dbfff640 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05ef5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005fd0 tx=0x7f66cc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe953a0af3da3f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:47.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66c27fc640 1 -- 172.21.15.33:0/521726223 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc01c480 con 0x562a05ef5620 2024-04-23T18:53:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66c27fc640 1 -- 172.21.15.33:0/521726223 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc01cad0 con 0x562a05ef5620 2024-04-23T18:53:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66c27fc640 1 -- 172.21.15.33:0/521726223 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc011af0 con 0x562a05ef5620 2024-04-23T18:53:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66e53a3740 1 -- 172.21.15.33:0/521726223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 msgr2=0x562a05ef5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.439+0000 7f66e53a3740 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05ef5a00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005fd0 tx=0x7f66cc0107a0 comp rx=0 tx=0).stop 2024-04-23T18:53:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 -- 172.21.15.33:0/521726223 shutdown_connections 2024-04-23T18:53:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05f02bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5f40 0x562a05f002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:48.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 --2- 172.21.15.33:0/521726223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05ef5a00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 -- 172.21.15.33:0/521726223 >> 172.21.15.33:0/521726223 conn(0x562a05ef1b30 msgr2=0x562a05ef1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 -- 172.21.15.33:0/521726223 shutdown_connections 2024-04-23T18:53:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 -- 172.21.15.33:0/521726223 wait complete. 2024-04-23T18:53:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 -- start start 2024-04-23T18:53:48.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 Processor -- start 2024-04-23T18:53:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.440+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05efd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05f00800 0x562a05efd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562a05f03430 con 0x562a05ef5620 2024-04-23T18:53:48.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562a05df50b0 con 0x562a05ef5f40 2024-04-23T18:53:48.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562a05f035b0 con 0x562a05f00800 2024-04-23T18:53:49.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 10 client.0 _ll_register_callbacks cb 0x562a05de2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:49.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:49.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46358/0 (socket says 172.21.15.33:46358) 2024-04-23T18:53:49.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 -- 172.21.15.33:0/3207122788 learned_addr learned my addr 172.21.15.33:0/3207122788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:49.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66db7fe640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05efd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:49.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e0c27640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05f00800 0x562a05efd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 -- 172.21.15.33:0/3207122788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05f00800 msgr2=0x562a05efd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05f00800 0x562a05efd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 -- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 msgr2=0x562a05efd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:49.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05efd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 -- 172.21.15.33:0/3207122788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a05c39620 con 0x562a05ef5620 2024-04-23T18:53:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66dbfff640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f66cc011750 tx=0x7f66cc01ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb07eb739df595c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc02b060 con 0x562a05ef5620 2024-04-23T18:53:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f66cc01c480 con 0x562a05ef5620 2024-04-23T18:53:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.441+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66cc01cad0 con 0x562a05ef5620 2024-04-23T18:53:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f66cc01c680 con 0x562a05ef5620 2024-04-23T18:53:49.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 10 client.18252 fetch_fsmap learned FSMap version 45 2024-04-23T18:53:49.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a05f0f360 con 0x562a05ef5620 2024-04-23T18:53:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc01cad0 con 0x562a05ef5620 2024-04-23T18:53:49.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f66cc026670 con 0x562a05ef5620 2024-04-23T18:53:49.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 10 client.18252 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:53:49.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 10 client.18252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 20 client.18252 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f10760 0x562a05f12b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:49.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 4 client.18252 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:53:49.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a05e68390 con 0x562a05f10760 2024-04-23T18:53:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.442+0000 7f66db7fe640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f10760 0x562a05f12b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.443+0000 7f66db7fe640 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f10760 0x562a05f12b50 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6ffc5b848278e8 server_cookie=e5165b8fa954a31a in_seq=0 out_seq=0 2024-04-23T18:53:50.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.443+0000 7f66bb7fe640 10 client.18252 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.443+0000 7f66bb7fe640 1 -- 172.21.15.33:0/3207122788 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a05e68390 con 0x562a05f10760 2024-04-23T18:53:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.443+0000 7f66bb7fe640 10 client.18252 handle_command_reply: tid=0 2024-04-23T18:53:50.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 client.18252 shutdown 2024-04-23T18:53:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 msgr2=0x562a05f00150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f66cc011750 tx=0x7f66cc01ceb0 comp rx=0 tx=0).stop 2024-04-23T18:53:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 shutdown_connections 2024-04-23T18:53:50.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f10760 0x562a05f12b50 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:50.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05f00800 0x562a05efd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:50.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05efd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:50.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66bb7fe640 0 client.18252 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3207122788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f00150 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 >> 172.21.15.33:0/3207122788 conn(0x562a05ef1b30 msgr2=0x562a05ef78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:50.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 shutdown_connections 2024-04-23T18:53:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 1 -- 172.21.15.33:0/3207122788 wait complete. 2024-04-23T18:53:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.444+0000 7f66e53a3740 20 client.18252 trim_cache size 0 max 16384 2024-04-23T18:53:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 Processor -- start 2024-04-23T18:53:51.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 -- start start 2024-04-23T18:53:51.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05f0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:51.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05ef89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:51.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1ae800 con 0x562a05f00800 2024-04-23T18:53:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b10e0 con 0x562a05ef5f40 2024-04-23T18:53:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e53a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1aeed0 con 0x562a05ef5620 2024-04-23T18:53:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e0c27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e0c27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46360/0 (socket says 172.21.15.33:46360) 2024-04-23T18:53:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.459+0000 7f66e0c27640 1 -- 172.21.15.33:0/3171192716 learned_addr learned my addr 172.21.15.33:0/3171192716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66dbfff640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05f0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 -- 172.21.15.33:0/3171192716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 msgr2=0x562a05f0c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05f0c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 -- 172.21.15.33:0/3171192716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 msgr2=0x562a05ef89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:51.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66db7fe640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05ef89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05ef89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 -- 172.21.15.33:0/3171192716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a05f12ec0 con 0x562a05f00800 2024-04-23T18:53:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66db7fe640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05ef89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:53:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e0c27640 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f66d4099340 tx=0x7f66d4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e417eb482c5c0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66c1ffb640 1 -- 172.21.15.33:0/3171192716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d407a0f0 con 0x562a05f00800 2024-04-23T18:53:52.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66c1ffb640 1 -- 172.21.15.33:0/3171192716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d409cee0 con 0x562a05f00800 2024-04-23T18:53:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66c1ffb640 1 -- 172.21.15.33:0/3171192716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40ce910 con 0x562a05f00800 2024-04-23T18:53:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e53a3740 1 -- 172.21.15.33:0/3171192716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 msgr2=0x562a05effc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.460+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f66d4099340 tx=0x7f66d4009590 comp rx=0 tx=0).stop 2024-04-23T18:53:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- 172.21.15.33:0/3171192716 shutdown_connections 2024-04-23T18:53:52.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05f00800 0x562a05effc80 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:52.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05ef5f40 0x562a05ef89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- 172.21.15.33:0/3171192716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5620 0x562a05f0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- 172.21.15.33:0/3171192716 >> 172.21.15.33:0/3171192716 conn(0x562a05ef1b30 msgr2=0x562a05efd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- 172.21.15.33:0/3171192716 shutdown_connections 2024-04-23T18:53:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- 172.21.15.33:0/3171192716 wait complete. 2024-04-23T18:53:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- start start 2024-04-23T18:53:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 Processor -- start 2024-04-23T18:53:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 0x562a05ef9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05efa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1af5a0 con 0x562a05ef5620 2024-04-23T18:53:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1b0a10 con 0x562a05f00800 2024-04-23T18:53:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.461+0000 7f66e53a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc1afc70 con 0x562a05ef5f40 2024-04-23T18:53:53.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66e53a3740 10 client.0 _ll_register_callbacks cb 0x7f66cc00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:53:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46368/0 (socket says 172.21.15.33:46368) 2024-04-23T18:53:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 -- 172.21.15.33:0/1393516174 learned_addr learned my addr 172.21.15.33:0/1393516174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:53.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66e0c27640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05efa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66db7fe640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 0x562a05ef9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 -- 172.21.15.33:0/1393516174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 msgr2=0x562a05ef9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:53.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 0x562a05ef9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 -- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 msgr2=0x562a05efa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05efa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 -- 172.21.15.33:0/1393516174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a05efc410 con 0x562a05ef5620 2024-04-23T18:53:53.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66db7fe640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 0x562a05ef9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66dbfff640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f66cc001eb0 tx=0x7f66cc00bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d6b6931980b7e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:53:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc02b040 con 0x562a05ef5620 2024-04-23T18:53:53.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f66cc01c480 con 0x562a05ef5620 2024-04-23T18:53:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d409cee0 con 0x562a05ef5620 2024-04-23T18:53:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f66cc03f050 con 0x562a05ef5620 2024-04-23T18:53:53.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.462+0000 7f66e53a3740 10 client.18258 fetch_fsmap learned FSMap version 45 2024-04-23T18:53:53.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a05f76920 con 0x562a05ef5620 2024-04-23T18:53:53.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d409cee0 con 0x562a05ef5620 2024-04-23T18:53:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f66cc005370 con 0x562a05ef5620 2024-04-23T18:53:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 10 client.18258 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:53:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 10 client.18258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:53:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 20 client.18258 populate_metadata read hostname 'smithi033' 2024-04-23T18:53:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f76fe0 0x562a05f793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 4 client.18258 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:53:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a05e62200 con 0x562a05f76fe0 2024-04-23T18:53:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66db7fe640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f76fe0 0x562a05f793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66bb7fe640 10 client.18258 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.463+0000 7f66db7fe640 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f76fe0 0x562a05f793d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d729fd64aeda96c server_cookie=6663e04cb4a7ace2 in_seq=0 out_seq=0 2024-04-23T18:53:54.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.467+0000 7f66bb7fe640 1 -- 172.21.15.33:0/1393516174 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a05e62200 con 0x562a05f76fe0 2024-04-23T18:53:54.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.467+0000 7f66bb7fe640 10 client.18258 handle_command_reply: tid=0 2024-04-23T18:53:54.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.467+0000 7f66e53a3740 1 client.18258 shutdown 2024-04-23T18:53:54.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.467+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 msgr2=0x562a05f08af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f66cc001eb0 tx=0x7f66cc00bf30 comp rx=0 tx=0).stop 2024-04-23T18:53:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 shutdown_connections 2024-04-23T18:53:54.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x562a05f76fe0 0x562a05f793d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:54.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x562a05f00800 0x562a05efa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:54.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562a05ef5f40 0x562a05ef9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66bb7fe640 0 client.18258 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:53:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 --2- 172.21.15.33:0/1393516174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562a05ef5620 0x562a05f08af0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 >> 172.21.15.33:0/1393516174 conn(0x562a05ef1b30 msgr2=0x562a05ef7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:55.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 shutdown_connections 2024-04-23T18:53:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 1 -- 172.21.15.33:0/1393516174 wait complete. 2024-04-23T18:53:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e53a3740 20 client.18258 trim_cache size 0 max 16384 2024-04-23T18:53:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:53:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:53:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:53:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:53:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 msgr2=0x7f66c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 0x7f66c40a6280 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66d002a9a0 tx=0x7f66d0005e80 comp rx=0 tx=0).stop 2024-04-23T18:53:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 msgr2=0x7f66dc19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f66d400eb40 tx=0x7f66d400c2d0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 shutdown_connections 2024-04-23T18:53:55.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66c40a3dc0 0x7f66c40a6280 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66dc10c5d0 0x7f66dc19a6b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66dc103b20 0x7f66dc1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 --2- 172.21.15.33:0/466588646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66dc103200 0x7f66dc1a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.468+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 >> 172.21.15.33:0/466588646 conn(0x7f66dc0771d0 msgr2=0x7f66dc0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.469+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 shutdown_connections 2024-04-23T18:53:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:29.469+0000 7f66e268a640 1 -- 172.21.15.33:0/466588646 wait complete. 2024-04-23T18:53:55.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:29 smithi033 ceph-mon[29274]: pgmap v839: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:53:56.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:31 smithi033 ceph-mon[29274]: pgmap v840: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 341 B/s wr, 9 op/s 2024-04-23T18:53:56.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:33 smithi033 ceph-mon[29274]: pgmap v841: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 597 B/s wr, 14 op/s 2024-04-23T18:53:56.095 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:35 smithi033 ceph-mon[29274]: pgmap v842: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:56.124 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:37 smithi033 ceph-mon[29274]: pgmap v843: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:56.279 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-23T18:53:56.409 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-23T18:53:56.452 INFO:tasks.quiescer:joining Quiescers 2024-04-23T18:53:56.460 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-23T18:53:56.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T18:53:56.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:39 smithi033 ceph-mon[29274]: pgmap v844: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:56.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:41 smithi033 ceph-mon[29274]: pgmap v845: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:56.651 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:45 smithi122 ceph-mon[32546]: pgmap v881: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:56.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:47 smithi122 ceph-mon[32546]: pgmap v882: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:56.807 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:49 smithi122 ceph-mon[32546]: pgmap v883: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:56.872 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:51 smithi122 ceph-mon[32546]: pgmap v884: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:56.880 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:53 smithi122 ceph-mon[32546]: pgmap v885: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:56.979 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:55 smithi122 ceph-mon[32546]: pgmap v886: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:57.077 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:45 smithi106 ceph-mon[32716]: pgmap v881: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:57.086 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:47 smithi106 ceph-mon[32716]: pgmap v882: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:57.144 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:49 smithi106 ceph-mon[32716]: pgmap v883: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:57.197 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:51 smithi106 ceph-mon[32716]: pgmap v884: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:53:57.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:53 smithi106 ceph-mon[32716]: pgmap v885: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:53:57.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:55 smithi106 ceph-mon[32716]: pgmap v886: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:53:57.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:57 smithi122 ceph-mon[32546]: pgmap v887: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:57.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:57 smithi106 ceph-mon[32716]: pgmap v887: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:53:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 -- 172.21.15.33:0/2389917935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c107290 msgr2=0x7fd61c10ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:57.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 --2- 172.21.15.33:0/2389917935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c107290 0x7fd61c10ee60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd61402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:53:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 -- 172.21.15.33:0/2389917935 shutdown_connections 2024-04-23T18:53:57.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 --2- 172.21.15.33:0/2389917935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c107290 0x7fd61c10ee60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 --2- 172.21.15.33:0/2389917935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c102b70 0x7fd61c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 --2- 172.21.15.33:0/2389917935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c1021c0 0x7fd61c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 -- 172.21.15.33:0/2389917935 >> 172.21.15.33:0/2389917935 conn(0x7fd61c0fda20 msgr2=0x7fd61c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:53:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.568+0000 7fd6222d0640 1 -- 172.21.15.33:0/2389917935 shutdown_connections 2024-04-23T18:53:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.569+0000 7fd6222d0640 1 -- 172.21.15.33:0/2389917935 wait complete. 2024-04-23T18:53:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.569+0000 7fd6222d0640 1 Processor -- start 2024-04-23T18:53:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.569+0000 7fd6222d0640 1 -- start start 2024-04-23T18:53:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:57.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c107290 0x7fd61c198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:58.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 0x7fd61c19d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c112250 con 0x7fd61c1021c0 2024-04-23T18:53:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1120d0 con 0x7fd61c19cef0 2024-04-23T18:53:58.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6222d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1123d0 con 0x7fd61c107290 2024-04-23T18:53:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56890/0 (socket says 172.21.15.33:56890) 2024-04-23T18:53:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 -- 172.21.15.33:0/13155971 learned_addr learned my addr 172.21.15.33:0/13155971 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:58.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c107290 0x7fd61c198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd62086d640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 0x7fd61c19d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:58.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 -- 172.21.15.33:0/13155971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c107290 msgr2=0x7fd61c198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c107290 0x7fd61c198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 -- 172.21.15.33:0/13155971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 msgr2=0x7fd61c19d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:53:58.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 0x7fd61c19d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:53:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 -- 172.21.15.33:0/13155971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c19d9e0 con 0x7fd61c1021c0 2024-04-23T18:53:58.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd62086d640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 0x7fd61c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:53:58.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd61bfff640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9f0 tx=0x7fd60c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:53:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.570+0000 7fd6197fa640 1 -- 172.21.15.33:0/13155971 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00cd80 con 0x7fd61c1021c0 2024-04-23T18:53:58.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.571+0000 7fd6197fa640 1 -- 172.21.15.33:0/13155971 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c0054d0 con 0x7fd61c1021c0 2024-04-23T18:53:58.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.571+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c19dcd0 con 0x7fd61c1021c0 2024-04-23T18:53:58.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.571+0000 7fd6197fa640 1 -- 172.21.15.33:0/13155971 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c010690 con 0x7fd61c1021c0 2024-04-23T18:53:58.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.571+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a5470 con 0x7fd61c1021c0 2024-04-23T18:53:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 Processor -- start 2024-04-23T18:53:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 -- start start 2024-04-23T18:53:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd6197fa640 1 -- 172.21.15.33:0/13155971 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd60c004e90 con 0x7fd61c1021c0 2024-04-23T18:53:59.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 0x558406e04a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e0f800 0x558406e11bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:53:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558406d69970 con 0x558406e04f40 2024-04-23T18:53:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558406d064a0 con 0x558406e0f800 2024-04-23T18:53:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd624fdd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558406d65930 con 0x558406e04620 2024-04-23T18:53:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56898/0 (socket says 172.21.15.33:56898) 2024-04-23T18:53:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 -- 172.21.15.33:0/2877025829 learned_addr learned my addr 172.21.15.33:0/2877025829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:53:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd62086d640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 0x558406e04a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e0f800 0x558406e11bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:53:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 -- 172.21.15.33:0/2877025829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 msgr2=0x558406e04a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:53:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 0x558406e04a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:00.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 -- 172.21.15.33:0/2877025829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e0f800 msgr2=0x558406e11bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:00.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e0f800 0x558406e11bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:00.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd61bfff640 1 -- 172.21.15.33:0/2877025829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558406a7c210 con 0x558406e04f40 2024-04-23T18:54:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.574+0000 7fd62086d640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 0x558406e04a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:00.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd6197fa640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 0x7fd6040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:00.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ce420 tx=0x7fd60c0d4e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e693a1fa33f5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd62086d640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 0x7fd6040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd6197fa640 1 -- 172.21.15.33:0/13155971 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fd60c0d0d70 con 0x7fd61c1021c0 2024-04-23T18:54:00.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd6027fc640 1 -- 172.21.15.33:0/2877025829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c098fa0 con 0x558406e04f40 2024-04-23T18:54:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd6027fc640 1 -- 172.21.15.33:0/2877025829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c09d400 con 0x558406e04f40 2024-04-23T18:54:00.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.575+0000 7fd6027fc640 1 -- 172.21.15.33:0/2877025829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09c660 con 0x558406e04f40 2024-04-23T18:54:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- 172.21.15.33:0/2877025829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 msgr2=0x558406e0f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:00.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ce420 tx=0x7fd60c0d4e30 comp rx=0 tx=0).stop 2024-04-23T18:54:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd62086d640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 0x7fd6040a6280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fe0 tx=0x7fd614004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- 172.21.15.33:0/2877025829 shutdown_connections 2024-04-23T18:54:00.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e0f800 0x558406e11bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0f2c0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:00.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2877025829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e04620 0x558406e04a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- 172.21.15.33:0/2877025829 >> 172.21.15.33:0/2877025829 conn(0x558406e00b30 msgr2=0x558406e00f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- 172.21.15.33:0/2877025829 shutdown_connections 2024-04-23T18:54:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- 172.21.15.33:0/2877025829 wait complete. 2024-04-23T18:54:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 -- start start 2024-04-23T18:54:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.576+0000 7fd624fdd740 1 Processor -- start 2024-04-23T18:54:01.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:01.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:01.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558406e123d0 con 0x558406e04f40 2024-04-23T18:54:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x558406d040b0 con 0x558406e04620 2024-04-23T18:54:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558406e12550 con 0x558406e0f800 2024-04-23T18:54:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd624fdd740 10 client.0 _ll_register_callbacks cb 0x558406cf1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:01.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:01.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56910/0 (socket says 172.21.15.33:56910) 2024-04-23T18:54:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 -- 172.21.15.33:0/4142333509 learned_addr learned my addr 172.21.15.33:0/4142333509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd62086d640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 -- 172.21.15.33:0/4142333509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 msgr2=0x558406e0ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:01.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 -- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 msgr2=0x558406e0f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e0f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 -- 172.21.15.33:0/4142333509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558406b48620 con 0x558406e04f40 2024-04-23T18:54:01.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61bfff640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e0f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:54:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd61000fe10 tx=0x7fd6100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f03aa1bb72926e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:02.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.577+0000 7fd5fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610004230 con 0x558406e04f40 2024-04-23T18:54:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fd6100043d0 con 0x558406e04f40 2024-04-23T18:54:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c09d400 con 0x558406e04f40 2024-04-23T18:54:02.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd610012660 con 0x558406e04f40 2024-04-23T18:54:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 10 client.18270 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558406e1e3a0 con 0x558406e04f40 2024-04-23T18:54:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60c09d400 con 0x558406e04f40 2024-04-23T18:54:02.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fd610003a00 con 0x558406e04f40 2024-04-23T18:54:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 10 client.18270 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 10 client.18270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 20 client.18270 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:02.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e1f8b0 0x558406e21d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:02.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 4 client.18270 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:54:02.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.578+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558406d77390 con 0x558406e1f8b0 2024-04-23T18:54:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.579+0000 7fd61bfff640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e1f8b0 0x558406e21d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:02.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.579+0000 7fd61bfff640 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e1f8b0 0x558406e21d00 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c0880757443945 server_cookie=2b350c40229afa1e in_seq=0 out_seq=0 2024-04-23T18:54:02.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.579+0000 7fd5fb7fe640 10 client.18270 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/4142333509 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558406d77390 con 0x558406e1f8b0 2024-04-23T18:54:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd5fb7fe640 10 client.18270 handle_command_reply: tid=0 2024-04-23T18:54:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 client.18270 shutdown 2024-04-23T18:54:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 msgr2=0x558406e0c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fd61000fe10 tx=0x7fd6100117b0 comp rx=0 tx=0).stop 2024-04-23T18:54:03.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 shutdown_connections 2024-04-23T18:54:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e1f8b0 0x558406e21d00 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:03.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e0c4a0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd5fb7fe640 0 client.18270 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:03.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 --2- 172.21.15.33:0/4142333509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e0f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 >> 172.21.15.33:0/4142333509 conn(0x558406e00b30 msgr2=0x558406e06860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:03.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 shutdown_connections 2024-04-23T18:54:03.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 1 -- 172.21.15.33:0/4142333509 wait complete. 2024-04-23T18:54:03.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.580+0000 7fd624fdd740 20 client.18270 trim_cache size 0 max 16384 2024-04-23T18:54:03.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 Processor -- start 2024-04-23T18:54:03.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 -- start start 2024-04-23T18:54:03.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e1b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ac670 con 0x558406e04f40 2024-04-23T18:54:04.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1aef50 con 0x558406e04620 2024-04-23T18:54:04.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd624fdd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1acd40 con 0x558406e0f800 2024-04-23T18:54:04.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56922/0 (socket says 172.21.15.33:56922) 2024-04-23T18:54:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 -- 172.21.15.33:0/2948812314 learned_addr learned my addr 172.21.15.33:0/2948812314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e1b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd62086d640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 -- 172.21.15.33:0/2948812314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 msgr2=0x558406e0ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:04.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:04.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 -- 172.21.15.33:0/2948812314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 msgr2=0x558406e1b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:04.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.595+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e1b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:04.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd61b7fe640 1 -- 172.21.15.33:0/2948812314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558406e0b450 con 0x558406e04f40 2024-04-23T18:54:04.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd62086d640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd61001c470 tx=0x7fd61000f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92290a05c0005934 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd601ffb640 1 -- 172.21.15.33:0/2948812314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610032070 con 0x558406e04f40 2024-04-23T18:54:04.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd601ffb640 1 -- 172.21.15.33:0/2948812314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd610003aa0 con 0x558406e04f40 2024-04-23T18:54:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd601ffb640 1 -- 172.21.15.33:0/2948812314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610029040 con 0x558406e04f40 2024-04-23T18:54:04.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd624fdd740 1 -- 172.21.15.33:0/2948812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 msgr2=0x558406e00f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fd61001c470 tx=0x7fd61000f5d0 comp rx=0 tx=0).stop 2024-04-23T18:54:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.596+0000 7fd624fdd740 1 -- 172.21.15.33:0/2948812314 shutdown_connections 2024-04-23T18:54:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e0ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04f40 0x558406e00f10 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:05.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2948812314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04620 0x558406e1b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- 172.21.15.33:0/2948812314 >> 172.21.15.33:0/2948812314 conn(0x558406e00b30 msgr2=0x558406e07af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- 172.21.15.33:0/2948812314 shutdown_connections 2024-04-23T18:54:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- 172.21.15.33:0/2948812314 wait complete. 2024-04-23T18:54:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- start start 2024-04-23T18:54:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 Processor -- start 2024-04-23T18:54:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:05.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 0x558406e12f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e13530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ad410 con 0x558406e04620 2024-04-23T18:54:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1ae880 con 0x558406e04f40 2024-04-23T18:54:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1adae0 con 0x558406e0f800 2024-04-23T18:54:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd624fdd740 10 client.0 _ll_register_callbacks cb 0x7fd61000f9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.597+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56936/0 (socket says 172.21.15.33:56936) 2024-04-23T18:54:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 -- 172.21.15.33:0/2890159780 learned_addr learned my addr 172.21.15.33:0/2890159780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd62086d640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e13530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 0x558406e12f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:06.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 -- 172.21.15.33:0/2890159780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 msgr2=0x558406e13530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e13530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 -- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 msgr2=0x558406e12f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 0x558406e12f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 -- 172.21.15.33:0/2890159780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558406e13d60 con 0x558406e04620 2024-04-23T18:54:06.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 0x558406e12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd61bfff640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ceab0 tx=0x7fd60c0d1070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9e01d914a1f827 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd5fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c09c840 con 0x558406e04620 2024-04-23T18:54:06.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fd60c016d90 con 0x558406e04620 2024-04-23T18:54:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd610003aa0 con 0x558406e04620 2024-04-23T18:54:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd60c09cad0 con 0x558406e04620 2024-04-23T18:54:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd624fdd740 10 client.18276 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558406e85ed0 con 0x558406e04620 2024-04-23T18:54:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.598+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd610003aa0 con 0x558406e04620 2024-04-23T18:54:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fd60c099370 con 0x558406e04620 2024-04-23T18:54:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 10 client.18276 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 10 client.18276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 20 client.18276 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e86460 0x558406e88850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 4 client.18276 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:54:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558406d71200 con 0x558406e86460 2024-04-23T18:54:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e86460 0x558406e88850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd5fb7fe640 10 client.18276 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.599+0000 7fd61b7fe640 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e86460 0x558406e88850 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb3f5e2d6dba122 server_cookie=74fd87978db08c6 in_seq=0 out_seq=0 2024-04-23T18:54:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd5fb7fe640 1 -- 172.21.15.33:0/2890159780 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558406d71200 con 0x558406e86460 2024-04-23T18:54:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd5fb7fe640 10 client.18276 handle_command_reply: tid=0 2024-04-23T18:54:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 client.18276 shutdown 2024-04-23T18:54:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 msgr2=0x558406e17c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ceab0 tx=0x7fd60c0d1070 comp rx=0 tx=0).stop 2024-04-23T18:54:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 shutdown_connections 2024-04-23T18:54:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x558406e86460 0x558406e88850 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558406e0f800 0x558406e13530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x558406e04f40 0x558406e12f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd5fb7fe640 0 client.18276 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 --2- 172.21.15.33:0/2890159780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558406e04620 0x558406e17c20 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 >> 172.21.15.33:0/2890159780 conn(0x558406e00b30 msgr2=0x558406dffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 shutdown_connections 2024-04-23T18:54:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 1 -- 172.21.15.33:0/2890159780 wait complete. 2024-04-23T18:54:08.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.614+0000 7fd624fdd740 20 client.18276 trim_cache size 0 max 16384 2024-04-23T18:54:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:54:08.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:54:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:54:08.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:54:08.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 msgr2=0x7fd6040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 0x7fd6040a6280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fe0 tx=0x7fd614004a40 comp rx=0 tx=0).stop 2024-04-23T18:54:08.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 msgr2=0x7fd61c198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:08.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00e9f0 tx=0x7fd60c00edf0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 shutdown_connections 2024-04-23T18:54:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6040a3dc0 0x7fd6040a6280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd61c19cef0 0x7fd61c19d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd61c107290 0x7fd61c198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 --2- 172.21.15.33:0/13155971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd61c1021c0 0x7fd61c198630 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 >> 172.21.15.33:0/13155971 conn(0x7fd61c0fda20 msgr2=0x7fd61c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 shutdown_connections 2024-04-23T18:54:09.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:42.615+0000 7fd6222d0640 1 -- 172.21.15.33:0/13155971 wait complete. 2024-04-23T18:54:09.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:53:59 smithi122 ceph-mon[32546]: pgmap v888: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:09.156 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:01 smithi122 ceph-mon[32546]: pgmap v889: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:09.173 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:03 smithi122 ceph-mon[32546]: pgmap v890: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:54:09.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:05 smithi122 ceph-mon[32546]: pgmap v891: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:54:09.261 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:05 smithi122 ceph-mon[32546]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:54:09.381 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:07 smithi122 ceph-mon[32546]: pgmap v893: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:09.480 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:07 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:09.505 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:53:59 smithi106 ceph-mon[32716]: pgmap v888: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:09.623 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:01 smithi106 ceph-mon[32716]: pgmap v889: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:09.727 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:03 smithi106 ceph-mon[32716]: pgmap v890: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:54:09.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:05 smithi106 ceph-mon[32716]: pgmap v891: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:54:09.744 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:05 smithi106 ceph-mon[32716]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:54:09.752 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:07 smithi106 ceph-mon[32716]: pgmap v893: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:09.817 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:07 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:09.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:43 smithi033 ceph-mon[29274]: pgmap v846: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:09.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:45 smithi033 ceph-mon[29274]: pgmap v847: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:09.889 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:47 smithi033 ceph-mon[29274]: pgmap v848: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:54:09.897 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:49 smithi033 ceph-mon[29274]: pgmap v849: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 15 op/s 2024-04-23T18:54:09.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:51 smithi033 ceph-mon[29274]: pgmap v850: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:54:09.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:53 smithi033 ceph-mon[29274]: pgmap v851: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:54:09.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:09 smithi122 ceph-mon[32546]: pgmap v894: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:10.041 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:09 smithi106 ceph-mon[32716]: pgmap v894: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 -- 172.21.15.33:0/471233952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103b20 msgr2=0x7f599810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 --2- 172.21.15.33:0/471233952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103b20 0x7f599810c000 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5984009900 tx=0x7f598402a480 comp rx=0 tx=0).stop 2024-04-23T18:54:10.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 -- 172.21.15.33:0/471233952 shutdown_connections 2024-04-23T18:54:10.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 --2- 172.21.15.33:0/471233952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 0x7f599810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:10.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 --2- 172.21.15.33:0/471233952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103b20 0x7f599810c000 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:10.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 --2- 172.21.15.33:0/471233952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103200 0x7f59981035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:10.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 -- 172.21.15.33:0/471233952 >> 172.21.15.33:0/471233952 conn(0x7f59980771d0 msgr2=0x7f59980775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:10.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 -- 172.21.15.33:0/471233952 shutdown_connections 2024-04-23T18:54:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.716+0000 7f599edfe640 1 -- 172.21.15.33:0/471233952 wait complete. 2024-04-23T18:54:10.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 Processor -- start 2024-04-23T18:54:10.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 -- start start 2024-04-23T18:54:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:10.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103b20 0x7f5998198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:10.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 0x7f599819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59981121f0 con 0x7f5998103200 2024-04-23T18:54:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5998112070 con 0x7f599810c5d0 2024-04-23T18:54:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599edfe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5998112370 con 0x7f5998103b20 2024-04-23T18:54:10.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599cb9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599cb9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36470/0 (socket says 172.21.15.33:36470) 2024-04-23T18:54:11.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.717+0000 7f599cb9a640 1 -- 172.21.15.33:0/2855618377 learned_addr learned my addr 172.21.15.33:0/2855618377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:11.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f5997fff640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103b20 0x7f5998198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599d39b640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 0x7f599819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 -- 172.21.15.33:0/2855618377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103b20 msgr2=0x7f5998198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103b20 0x7f5998198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 -- 172.21.15.33:0/2855618377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 msgr2=0x7f599819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:11.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 0x7f599819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 -- 172.21.15.33:0/2855618377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599819d690 con 0x7f5998103200 2024-04-23T18:54:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:11.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f5995ffb640 1 -- 172.21.15.33:0/2855618377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c017070 con 0x7f5998103200 2024-04-23T18:54:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f5995ffb640 1 -- 172.21.15.33:0/2855618377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598c009c30 con 0x7f5998103200 2024-04-23T18:54:11.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599819d980 con 0x7f5998103200 2024-04-23T18:54:11.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f5995ffb640 1 -- 172.21.15.33:0/2855618377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c00f620 con 0x7f5998103200 2024-04-23T18:54:11.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.718+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a5210 con 0x7f5998103200 2024-04-23T18:54:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.721+0000 7f5995ffb640 1 -- 172.21.15.33:0/2855618377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f598c010460 con 0x7f5998103200 2024-04-23T18:54:11.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 Processor -- start 2024-04-23T18:54:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 -- start start 2024-04-23T18:54:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:11.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:11.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55addc618970 con 0x55addc6b3520 2024-04-23T18:54:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55addc5b54a0 con 0x55addc6b3e40 2024-04-23T18:54:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55addc614930 con 0x55addc6c2c10 2024-04-23T18:54:12.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:12.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36476/0 (socket says 172.21.15.33:36476) 2024-04-23T18:54:12.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 -- 172.21.15.33:0/3355318926 learned_addr learned my addr 172.21.15.33:0/3355318926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f5995ffb640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 0x7f59880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f5997fff640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 -- 172.21.15.33:0/3355318926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 msgr2=0x55addc6c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 -- 172.21.15.33:0/3355318926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 msgr2=0x55addc6c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:12.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6c26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 -- 172.21.15.33:0/3355318926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addc32b210 con 0x55addc6b3520 2024-04-23T18:54:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f5997fff640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:12.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:12.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f599d39b640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 0x7f59880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:12.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.722+0000 7f5995ffb640 1 -- 172.21.15.33:0/2855618377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f598c0d03f0 con 0x7f5998103200 2024-04-23T18:54:12.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f599d39b640 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5990005c10 tx=0x7f5990010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996eefe71ed73859 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f5982ffd640 1 -- 172.21.15.33:0/3355318926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599001c900 con 0x55addc6b3520 2024-04-23T18:54:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f5982ffd640 1 -- 172.21.15.33:0/3355318926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599001b3d0 con 0x55addc6b3520 2024-04-23T18:54:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f5982ffd640 1 -- 172.21.15.33:0/3355318926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599001be60 con 0x55addc6b3520 2024-04-23T18:54:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3355318926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 msgr2=0x55addc6b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5990005c10 tx=0x7f5990010800 comp rx=0 tx=0).stop 2024-04-23T18:54:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.723+0000 7f599d39b640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 0x7f59880a6300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5990020000 tx=0x7f599001d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3355318926 shutdown_connections 2024-04-23T18:54:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:13.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3355318926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3520 0x55addc6b3900 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3355318926 >> 172.21.15.33:0/3355318926 conn(0x55addc6afb30 msgr2=0x55addc6aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3355318926 shutdown_connections 2024-04-23T18:54:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3355318926 wait complete. 2024-04-23T18:54:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 -- start start 2024-04-23T18:54:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.724+0000 7f59a1bf2740 1 Processor -- start 2024-04-23T18:54:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:13.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55addc6c5840 con 0x55addc6c2c10 2024-04-23T18:54:13.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55addc5b30b0 con 0x55addc6b3e40 2024-04-23T18:54:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55addc6c59c0 con 0x55addc6b3520 2024-04-23T18:54:13.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f59a1bf2740 10 client.0 _ll_register_callbacks cb 0x55addc5a0d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:13.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36484/0 (socket says 172.21.15.33:36484) 2024-04-23T18:54:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 -- 172.21.15.33:0/3369076169 learned_addr learned my addr 172.21.15.33:0/3369076169 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f5997fff640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:14.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 -- 172.21.15.33:0/3369076169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 msgr2=0x55addc6c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 -- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 msgr2=0x55addc6bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:14.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:14.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 -- 172.21.15.33:0/3369076169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55addc3f7620 con 0x55addc6c2c10 2024-04-23T18:54:14.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:14.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f599d39b640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5990016010 tx=0x7f599001b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1409833eab2418 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:14.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.725+0000 7f5977fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990013620 con 0x55addc6c2c10 2024-04-23T18:54:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f599001c880 con 0x55addc6c2c10 2024-04-23T18:54:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599001b3d0 con 0x55addc6c2c10 2024-04-23T18:54:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f599003d050 con 0x55addc6c2c10 2024-04-23T18:54:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 10 client.18288 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55addc6d1710 con 0x55addc6c2c10 2024-04-23T18:54:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f599001b3d0 con 0x55addc6c2c10 2024-04-23T18:54:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f599000f040 con 0x55addc6c2c10 2024-04-23T18:54:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 10 client.18288 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 10 client.18288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 20 client.18288 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc6d2b10 0x55addc6d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 4 client.18288 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:54:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55addc626390 con 0x55addc6d2b10 2024-04-23T18:54:15.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.726+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc6d2b10 0x55addc6d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:15.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.727+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc6d2b10 0x55addc6d4f00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64887148e16a87d2 server_cookie=5cdcd7c9da44243e in_seq=0 out_seq=0 2024-04-23T18:54:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.727+0000 7f5977fff640 10 client.18288 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f5977fff640 1 -- 172.21.15.33:0/3369076169 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55addc626390 con 0x55addc6d2b10 2024-04-23T18:54:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f5977fff640 10 client.18288 handle_command_reply: tid=0 2024-04-23T18:54:15.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 client.18288 shutdown 2024-04-23T18:54:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 msgr2=0x55addc6bfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:15.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5990016010 tx=0x7f599001b7b0 comp rx=0 tx=0).stop 2024-04-23T18:54:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 shutdown_connections 2024-04-23T18:54:15.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc6d2b10 0x55addc6d4f00 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:15.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6bfd50 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:15.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3369076169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:16.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 >> 172.21.15.33:0/3369076169 conn(0x55addc6afb30 msgr2=0x55addc6b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:16.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f5977fff640 0 client.18288 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 shutdown_connections 2024-04-23T18:54:16.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3369076169 wait complete. 2024-04-23T18:54:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.728+0000 7f59a1bf2740 20 client.18288 trim_cache size 0 max 16384 2024-04-23T18:54:16.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:55 smithi033 ceph-mon[29274]: pgmap v852: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:54:16.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: pgmap v895: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:16.358 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:16.388 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:16.397 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:16.406 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:16.413 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:16.452 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:16.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:16.562 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:11 smithi122 ceph-mon[32546]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:54:16.637 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:12 smithi122 ceph-mon[32546]: pgmap v896: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:54:16.682 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:12 smithi122 ceph-mon[32546]: pgmap v897: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-23T18:54:16.778 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:14 smithi122 ceph-mon[32546]: pgmap v899: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:54:16.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:15 smithi122 ceph-mon[32546]: daemon mds.j restarted 2024-04-23T18:54:16.931 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: pgmap v895: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:54:17.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:17.109 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:17.118 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:17.131 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:17.139 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:17.226 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:17.235 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:17.244 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:11 smithi106 ceph-mon[32716]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:54:17.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:12 smithi106 ceph-mon[32716]: pgmap v896: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:54:17.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:12 smithi106 ceph-mon[32716]: pgmap v897: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-23T18:54:17.439 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:14 smithi106 ceph-mon[32716]: pgmap v899: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:54:17.541 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:15 smithi106 ceph-mon[32716]: daemon mds.j restarted 2024-04-23T18:54:17.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: pgmap v900: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:54:17.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:54:17.820 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6834/2623838743,v1:172.21.15.33:6835/2623838743] up:boot 2024-04-23T18:54:17.863 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:54:17.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:54:17.927 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:54:17.939 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: pgmap v900: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:54:17.948 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:54:17.960 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6834/2623838743,v1:172.21.15.33:6835/2623838743] up:boot 2024-04-23T18:54:18.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:54:18.124 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:54:18.215 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:54:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.743+0000 7f59a1bf2740 1 Processor -- start 2024-04-23T18:54:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.743+0000 7f59a1bf2740 1 -- start start 2024-04-23T18:54:18.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.743+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:18.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aa440 con 0x55addc6c2c10 2024-04-23T18:54:18.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59981acd20 con 0x55addc6b3e40 2024-04-23T18:54:18.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aab10 con 0x55addc6b3520 2024-04-23T18:54:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:18.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36500/0 (socket says 172.21.15.33:36500) 2024-04-23T18:54:18.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 -- 172.21.15.33:0/3957325711 learned_addr learned my addr 172.21.15.33:0/3957325711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f5997fff640 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599cb9a640 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 -- 172.21.15.33:0/3957325711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 msgr2=0x55addc6ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 -- 172.21.15.33:0/3957325711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 msgr2=0x55addc6bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:18.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 -- 172.21.15.33:0/3957325711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55addc6ab500 con 0x55addc6c2c10 2024-04-23T18:54:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f599d39b640 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5990024360 tx=0x7f5990026b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96dc86a074e1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59827fc640 1 -- 172.21.15.33:0/3957325711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990045070 con 0x55addc6c2c10 2024-04-23T18:54:19.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59827fc640 1 -- 172.21.15.33:0/3957325711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5990034070 con 0x55addc6c2c10 2024-04-23T18:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.744+0000 7f59827fc640 1 -- 172.21.15.33:0/3957325711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900100b0 con 0x55addc6c2c10 2024-04-23T18:54:19.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3957325711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 msgr2=0x55addc6c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:19.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f5990024360 tx=0x7f5990026b10 comp rx=0 tx=0).stop 2024-04-23T18:54:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3957325711 shutdown_connections 2024-04-23T18:54:19.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6c2c10 0x55addc6c2050 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:19.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3e40 0x55addc6bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/3957325711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6b3520 0x55addc6ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3957325711 >> 172.21.15.33:0/3957325711 conn(0x55addc6afb30 msgr2=0x55addc6ba670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3957325711 shutdown_connections 2024-04-23T18:54:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- 172.21.15.33:0/3957325711 wait complete. 2024-04-23T18:54:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 -- start start 2024-04-23T18:54:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.745+0000 7f59a1bf2740 1 Processor -- start 2024-04-23T18:54:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3520 0x55addc6cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6bba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ab1e0 con 0x55addc6b3e40 2024-04-23T18:54:20.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ac650 con 0x55addc6b3520 2024-04-23T18:54:20.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ab8b0 con 0x55addc6c2c10 2024-04-23T18:54:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f59a1bf2740 10 client.0 _ll_register_callbacks cb 0x7f5990009240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:20.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36516/0 (socket says 172.21.15.33:36516) 2024-04-23T18:54:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 -- 172.21.15.33:0/2914590614 learned_addr learned my addr 172.21.15.33:0/2914590614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3520 0x55addc6cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f599d39b640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6bba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 -- 172.21.15.33:0/2914590614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 msgr2=0x55addc6bba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6bba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:20.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 -- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3520 msgr2=0x55addc6cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3520 0x55addc6cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.746+0000 7f5997fff640 1 -- 172.21.15.33:0/2914590614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55addc6bc040 con 0x55addc6b3e40 2024-04-23T18:54:20.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5997fff640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f598402a9a0 tx=0x7f5984031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89c58ebf6b299ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984035040 con 0x55addc6b3e40 2024-04-23T18:54:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7f5984031d10 con 0x55addc6b3e40 2024-04-23T18:54:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5990034070 con 0x55addc6b3e40 2024-04-23T18:54:21.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5984004040 con 0x55addc6b3e40 2024-04-23T18:54:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 10 client.18294 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55addc738910 con 0x55addc6b3e40 2024-04-23T18:54:21.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5990034070 con 0x55addc6b3e40 2024-04-23T18:54:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f5984004870 con 0x55addc6b3e40 2024-04-23T18:54:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 10 client.18294 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 10 client.18294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 20 client.18294 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc7390c0 0x55addc73b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:21.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 4 client.18294 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:54:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.747+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55addc620200 con 0x55addc7390c0 2024-04-23T18:54:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.748+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc7390c0 0x55addc73b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.748+0000 7f5977fff640 10 client.18294 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:21.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.748+0000 7f599cb9a640 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc7390c0 0x55addc73b4e0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd79b944af379db server_cookie=a12edee3139d4924 in_seq=0 out_seq=0 2024-04-23T18:54:21.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f5977fff640 1 -- 172.21.15.33:0/2914590614 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55addc620200 con 0x55addc7390c0 2024-04-23T18:54:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f5977fff640 10 client.18294 handle_command_reply: tid=0 2024-04-23T18:54:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 client.18294 shutdown 2024-04-23T18:54:21.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 msgr2=0x55addc6cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:22.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f598402a9a0 tx=0x7f5984031740 comp rx=0 tx=0).stop 2024-04-23T18:54:22.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 shutdown_connections 2024-04-23T18:54:22.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55addc7390c0 0x55addc73b4e0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:22.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55addc6c2c10 0x55addc6bba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:22.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f5977fff640 0 client.18294 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55addc6b3e40 0x55addc6cb150 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 --2- 172.21.15.33:0/2914590614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55addc6b3520 0x55addc6cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.755+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 >> 172.21.15.33:0/2914590614 conn(0x55addc6afb30 msgr2=0x55addc6b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 shutdown_connections 2024-04-23T18:54:22.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f59a1bf2740 1 -- 172.21.15.33:0/2914590614 wait complete. 2024-04-23T18:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f59a1bf2740 20 client.18294 trim_cache size 0 max 16384 2024-04-23T18:54:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:54:22.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:54:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:54:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:54:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 msgr2=0x7f59880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 0x7f59880a6300 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5990020000 tx=0x7f599001d000 comp rx=0 tx=0).stop 2024-04-23T18:54:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 msgr2=0x7f59981986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:22.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00de10 comp rx=0 tx=0).stop 2024-04-23T18:54:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 shutdown_connections 2024-04-23T18:54:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f59880a3e40 0x7f59880a6300 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f599810c5d0 0x7f599819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5998103b20 0x7f5998198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:23.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 --2- 172.21.15.33:0/2855618377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5998103200 0x7f59981986f0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:23.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 >> 172.21.15.33:0/2855618377 conn(0x7f59980771d0 msgr2=0x7f59980fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:23.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 shutdown_connections 2024-04-23T18:54:23.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:52:55.756+0000 7f599edfe640 1 -- 172.21.15.33:0/2855618377 wait complete. 2024-04-23T18:54:23.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:57 smithi033 ceph-mon[29274]: pgmap v853: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-23T18:54:23.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:52:59 smithi033 ceph-mon[29274]: pgmap v854: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-04-23T18:54:23.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:01 smithi033 ceph-mon[29274]: pgmap v855: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:54:23.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:02 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:54:23.554 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:03 smithi033 ceph-mon[29274]: pgmap v856: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 30 op/s 2024-04-23T18:54:23.606 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:05 smithi033 ceph-mon[29274]: pgmap v857: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:54:23.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: pgmap v858: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 KiB/s wr, 25 op/s 2024-04-23T18:54:23.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:23.800 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:23.844 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:54:23.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:54:23.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: pgmap v859: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 52 KiB/s wr, 35 op/s 2024-04-23T18:54:23.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: pgmap v860: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:54:23.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:06 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:54:24.070 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:18 smithi106 ceph-mon[32716]: pgmap v902: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 170 B/s wr, 15 op/s 2024-04-23T18:54:24.083 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:20 smithi106 ceph-mon[32716]: pgmap v903: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 11 op/s 2024-04-23T18:54:24.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:22 smithi106 ceph-mon[32716]: pgmap v904: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-23T18:54:24.169 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:18 smithi122 ceph-mon[32546]: pgmap v902: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 170 B/s wr, 15 op/s 2024-04-23T18:54:24.176 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:20 smithi122 ceph-mon[32546]: pgmap v903: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 11 op/s 2024-04-23T18:54:24.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:22 smithi122 ceph-mon[32546]: pgmap v904: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-23T18:54:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- 172.21.15.33:0/1305917184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5481006c0 msgr2=0x7fc548100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 --2- 172.21.15.33:0/1305917184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5481006c0 0x7fc548100aa0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fc544009900 tx=0x7fc54402a410 comp rx=0 tx=0).stop 2024-04-23T18:54:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- 172.21.15.33:0/1305917184 shutdown_connections 2024-04-23T18:54:24.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 --2- 172.21.15.33:0/1305917184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc548107280 0x7fc54810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:24.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 --2- 172.21.15.33:0/1305917184 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc548101070 0x7fc548106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 --2- 172.21.15.33:0/1305917184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5481006c0 0x7fc548100aa0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:24.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- 172.21.15.33:0/1305917184 >> 172.21.15.33:0/1305917184 conn(0x7fc548077390 msgr2=0x7fc5480fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:24.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- 172.21.15.33:0/1305917184 shutdown_connections 2024-04-23T18:54:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- 172.21.15.33:0/1305917184 wait complete. 2024-04-23T18:54:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 Processor -- start 2024-04-23T18:54:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.857+0000 7fc54f410640 1 -- start start 2024-04-23T18:54:24.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5481006c0 0x7fc5481a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:24.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc54819a760 0x7fc54819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:24.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481121c0 con 0x7fc548107280 2024-04-23T18:54:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112040 con 0x7fc5481006c0 2024-04-23T18:54:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54f410640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc548112340 con 0x7fc54819a760 2024-04-23T18:54:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52050/0 (socket says 172.21.15.33:52050) 2024-04-23T18:54:25.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1762536829 learned_addr learned my addr 172.21.15.33:0/1762536829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:25.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5481006c0 0x7fc5481a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc54819a760 0x7fc54819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:25.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1762536829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc54819a760 msgr2=0x7fc54819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:25.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc54819a760 0x7fc54819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:25.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1762536829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5481006c0 msgr2=0x7fc5481a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5481006c0 0x7fc5481a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1762536829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54819b360 con 0x7fc548107280 2024-04-23T18:54:25.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc53800d770 tx=0x7fc53800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc53e7fc640 1 -- 172.21.15.33:0/1762536829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538017070 con 0x7fc548107280 2024-04-23T18:54:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.858+0000 7fc53e7fc640 1 -- 172.21.15.33:0/1762536829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc538009c30 con 0x7fc548107280 2024-04-23T18:54:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.859+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a73f0 con 0x7fc548107280 2024-04-23T18:54:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.859+0000 7fc53e7fc640 1 -- 172.21.15.33:0/1762536829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53800f620 con 0x7fc548107280 2024-04-23T18:54:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.859+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a77f0 con 0x7fc548107280 2024-04-23T18:54:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.861+0000 7fc53e7fc640 1 -- 172.21.15.33:0/1762536829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc53801a070 con 0x7fc548107280 2024-04-23T18:54:25.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.861+0000 7fc552204740 1 Processor -- start 2024-04-23T18:54:25.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.861+0000 7fc552204740 1 -- start start 2024-04-23T18:54:25.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb00fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:25.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb01a800 0x563ddb01cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:26.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ddaf74970 con 0x563ddb00ff40 2024-04-23T18:54:26.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ddaf114a0 con 0x563ddb00f620 2024-04-23T18:54:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc552204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ddaf70930 con 0x563ddb01a800 2024-04-23T18:54:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc53e7fc640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 0x7fc5340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:26.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52062/0 (socket says 172.21.15.33:52062) 2024-04-23T18:54:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 -- 172.21.15.33:0/1376045810 learned_addr learned my addr 172.21.15.33:0/1376045810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb00fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc53e7fc640 1 -- 172.21.15.33:0/1762536829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fc5380d0360 con 0x7fc548107280 2024-04-23T18:54:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb01a800 0x563ddb01cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:26.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 0x7fc5340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:26.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 -- 172.21.15.33:0/1376045810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb01a800 msgr2=0x563ddb01cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb01a800 0x563ddb01cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 -- 172.21.15.33:0/1376045810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 msgr2=0x563ddb00fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:26.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb00fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:26.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 -- 172.21.15.33:0/1376045810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddac87210 con 0x563ddb00ff40 2024-04-23T18:54:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc544009900 tx=0x7fc544032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2242c0dc8ffd3bae server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:27.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.862+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 0x7fc5340a6280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc540009fd0 tx=0x7fc540005bd0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:27.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc5337fe640 1 -- 172.21.15.33:0/1376045810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544040070 con 0x563ddb00ff40 2024-04-23T18:54:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc5337fe640 1 -- 172.21.15.33:0/1376045810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc544004030 con 0x563ddb00ff40 2024-04-23T18:54:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc5337fe640 1 -- 172.21.15.33:0/1376045810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544035040 con 0x563ddb00ff40 2024-04-23T18:54:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 -- 172.21.15.33:0/1376045810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 msgr2=0x563ddb01a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:27.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fc544009900 tx=0x7fc544032bd0 comp rx=0 tx=0).stop 2024-04-23T18:54:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 -- 172.21.15.33:0/1376045810 shutdown_connections 2024-04-23T18:54:27.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb01a800 0x563ddb01cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb01a2c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 --2- 172.21.15.33:0/1376045810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb00fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 -- 172.21.15.33:0/1376045810 >> 172.21.15.33:0/1376045810 conn(0x563ddb00bb30 msgr2=0x563ddb00bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:27.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 -- 172.21.15.33:0/1376045810 shutdown_connections 2024-04-23T18:54:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.863+0000 7fc552204740 1 -- 172.21.15.33:0/1376045810 wait complete. 2024-04-23T18:54:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 -- start start 2024-04-23T18:54:27.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 Processor -- start 2024-04-23T18:54:27.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00f620 0x563ddb01a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb0173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:27.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563ddb01d3d0 con 0x563ddb01a800 2024-04-23T18:54:28.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ddaf0f0b0 con 0x563ddb00ff40 2024-04-23T18:54:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ddb01d550 con 0x563ddb00f620 2024-04-23T18:54:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc552204740 10 client.0 _ll_register_callbacks cb 0x563ddaefcd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc54d9ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.864+0000 7fc54d9ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52072/0 (socket says 172.21.15.33:52072) 2024-04-23T18:54:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 -- 172.21.15.33:0/614385723 learned_addr learned my addr 172.21.15.33:0/614385723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00f620 0x563ddb01a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb0173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 -- 172.21.15.33:0/614385723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00f620 msgr2=0x563ddb01a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:28.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00f620 0x563ddb01a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 -- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 msgr2=0x563ddb0173c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:28.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb0173c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:28.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 -- 172.21.15.33:0/614385723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddad53620 con 0x563ddb01a800 2024-04-23T18:54:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb0173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc54000db80 tx=0x7fc540004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a3efaf310d9139 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540025030 con 0x563ddb01a800 2024-04-23T18:54:28.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fc54001b070 con 0x563ddb01a800 2024-04-23T18:54:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc544004030 con 0x563ddb01a800 2024-04-23T18:54:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc540021480 con 0x563ddb01a800 2024-04-23T18:54:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc552204740 10 client.18306 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:28.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563ddb029270 con 0x563ddb01a800 2024-04-23T18:54:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.865+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc544004030 con 0x563ddb01a800 2024-04-23T18:54:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fc54001e050 con 0x563ddb01a800 2024-04-23T18:54:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 10 client.18306 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:29.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 20 client.18306 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb02a730 0x563ddb02cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 4 client.18306 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:54:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ddaf82390 con 0x563ddb02a730 2024-04-23T18:54:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb02a730 0x563ddb02cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:29.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb02a730 0x563ddb02cb40 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d68cceea12bf718 server_cookie=b73ad3ba115c2ae7 in_seq=0 out_seq=0 2024-04-23T18:54:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.866+0000 7fc530ff9640 10 client.18306 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc530ff9640 1 -- 172.21.15.33:0/614385723 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ddaf82390 con 0x563ddb02a730 2024-04-23T18:54:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc530ff9640 10 client.18306 handle_command_reply: tid=0 2024-04-23T18:54:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 client.18306 shutdown 2024-04-23T18:54:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 msgr2=0x563ddb017900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fc54000db80 tx=0x7fc540004ab0 comp rx=0 tx=0).stop 2024-04-23T18:54:30.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 shutdown_connections 2024-04-23T18:54:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb02a730 0x563ddb02cb40 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:30.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb01a800 0x563ddb017900 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb0173c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 --2- 172.21.15.33:0/614385723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00f620 0x563ddb01a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc530ff9640 0 client.18306 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 >> 172.21.15.33:0/614385723 conn(0x563ddb00bb30 msgr2=0x563ddb0118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 shutdown_connections 2024-04-23T18:54:30.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.867+0000 7fc552204740 1 -- 172.21.15.33:0/614385723 wait complete. 2024-04-23T18:54:30.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.868+0000 7fc552204740 20 client.18306 trim_cache size 0 max 16384 2024-04-23T18:54:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 Processor -- start 2024-04-23T18:54:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 -- start start 2024-04-23T18:54:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb026290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:30.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb0128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:30.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481ae8e0 con 0x563ddb00ff40 2024-04-23T18:54:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b11c0 con 0x563ddb00f620 2024-04-23T18:54:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc552204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481aefb0 con 0x563ddb00bb30 2024-04-23T18:54:30.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc54d9ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:31.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc54d9ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52080/0 (socket says 172.21.15.33:52080) 2024-04-23T18:54:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc54d9ad640 1 -- 172.21.15.33:0/1338720302 learned_addr learned my addr 172.21.15.33:0/1338720302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb026290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:31.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.883+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb0128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 -- 172.21.15.33:0/1338720302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 msgr2=0x563ddb026290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:31.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb026290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:31.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 -- 172.21.15.33:0/1338720302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 msgr2=0x563ddb0128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:31.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb0128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 -- 172.21.15.33:0/1338720302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ddb00cb30 con 0x563ddb00ff40 2024-04-23T18:54:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc54000db50 tx=0x7fc540000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f605d0aa1ac9ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:31.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc532ffd640 1 -- 172.21.15.33:0/1338720302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540025030 con 0x563ddb00ff40 2024-04-23T18:54:31.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc532ffd640 1 -- 172.21.15.33:0/1338720302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540037070 con 0x563ddb00ff40 2024-04-23T18:54:31.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.884+0000 7fc532ffd640 1 -- 172.21.15.33:0/1338720302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540032b10 con 0x563ddb00ff40 2024-04-23T18:54:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- 172.21.15.33:0/1338720302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 msgr2=0x563ddb019c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc54000db50 tx=0x7fc540000f40 comp rx=0 tx=0).stop 2024-04-23T18:54:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- 172.21.15.33:0/1338720302 shutdown_connections 2024-04-23T18:54:31.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00ff40 0x563ddb019c00 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:31.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00f620 0x563ddb0128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:31.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 --2- 172.21.15.33:0/1338720302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb026290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- 172.21.15.33:0/1338720302 >> 172.21.15.33:0/1338720302 conn(0x563ddb017a60 msgr2=0x563ddb0121d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:32.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- 172.21.15.33:0/1338720302 shutdown_connections 2024-04-23T18:54:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- 172.21.15.33:0/1338720302 wait complete. 2024-04-23T18:54:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 -- start start 2024-04-23T18:54:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 Processor -- start 2024-04-23T18:54:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.885+0000 7fc552204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb01e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:32.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:32.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb08fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:32.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481af680 con 0x563ddb00f620 2024-04-23T18:54:32.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b0af0 con 0x563ddb00ff40 2024-04-23T18:54:32.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481afd50 con 0x563ddb00bb30 2024-04-23T18:54:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 10 client.0 _ll_register_callbacks cb 0x7fc540012f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52094/0 (socket says 172.21.15.33:52094) 2024-04-23T18:54:32.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1739641382 learned_addr learned my addr 172.21.15.33:0/1739641382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb01e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:33.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb08fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:33.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1739641382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 msgr2=0x563ddb01e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:33.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb01e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 msgr2=0x563ddb08fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb08fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 -- 172.21.15.33:0/1739641382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ddb01f2d0 con 0x563ddb00f620 2024-04-23T18:54:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54d9ad640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb08fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc54c9ab640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc538098cc0 tx=0x7fc53809bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa11e31bd0657416 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc530ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53808b060 con 0x563ddb00f620 2024-04-23T18:54:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 8542+0+0 (secure 0 0 0) 0x7fc5380a0170 con 0x563ddb00f620 2024-04-23T18:54:33.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.886+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc540037070 con 0x563ddb00f620 2024-04-23T18:54:33.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc5380b5070 con 0x563ddb00f620 2024-04-23T18:54:33.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 10 client.18312 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:33.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563ddb091390 con 0x563ddb00f620 2024-04-23T18:54:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc540037070 con 0x563ddb00f620 2024-04-23T18:54:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fc5380b8cd0 con 0x563ddb00f620 2024-04-23T18:54:34.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 10 client.18312 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 10 client.18312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 20 client.18312 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb01a800 0x563ddb093ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:34.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 4 client.18312 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:54:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ddaf7c200 con 0x563ddb01a800 2024-04-23T18:54:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.887+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb01a800 0x563ddb093ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.888+0000 7fc54d1ac640 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb01a800 0x563ddb093ab0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f8729068747930 server_cookie=3e663064544e2ddb in_seq=0 out_seq=0 2024-04-23T18:54:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.888+0000 7fc530ff9640 10 client.18312 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.894+0000 7fc530ff9640 1 -- 172.21.15.33:0/1739641382 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ddaf7c200 con 0x563ddb01a800 2024-04-23T18:54:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.894+0000 7fc530ff9640 10 client.18312 handle_command_reply: tid=0 2024-04-23T18:54:34.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.894+0000 7fc552204740 1 client.18312 shutdown 2024-04-23T18:54:34.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 msgr2=0x563ddb01ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:34.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc538098cc0 tx=0x7fc53809bef0 comp rx=0 tx=0).stop 2024-04-23T18:54:34.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 shutdown_connections 2024-04-23T18:54:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563ddb01a800 0x563ddb093ab0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ddb00ff40 0x563ddb08fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563ddb00f620 0x563ddb01ea30 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc530ff9640 0 client.18312 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:34.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 --2- 172.21.15.33:0/1739641382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ddb00bb30 0x563ddb01e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 >> 172.21.15.33:0/1739641382 conn(0x563ddb017a60 msgr2=0x563ddb0129f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:35.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 shutdown_connections 2024-04-23T18:54:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 1 -- 172.21.15.33:0/1739641382 wait complete. 2024-04-23T18:54:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc552204740 20 client.18312 trim_cache size 0 max 16384 2024-04-23T18:54:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:54:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:54:35.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:54:35.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:54:35.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 msgr2=0x7fc5340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:35.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 0x7fc5340a6280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc540009fd0 tx=0x7fc540005bd0 comp rx=0 tx=0).stop 2024-04-23T18:54:35.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 msgr2=0x7fc5481a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:35.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fc53800d770 tx=0x7fc53800dc30 comp rx=0 tx=0).stop 2024-04-23T18:54:35.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 shutdown_connections 2024-04-23T18:54:35.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc5340a3dc0 0x7fc5340a6280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:35.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc54819a760 0x7fc54819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548107280 0x7fc5481a0b90 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 --2- 172.21.15.33:0/1762536829 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5481006c0 0x7fc5481a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:35.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.895+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 >> 172.21.15.33:0/1762536829 conn(0x7fc548077390 msgr2=0x7fc548105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:35.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.896+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 shutdown_connections 2024-04-23T18:54:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:08.896+0000 7fc54f410640 1 -- 172.21.15.33:0/1762536829 wait complete. 2024-04-23T18:54:35.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:09 smithi033 ceph-mon[29274]: pgmap v861: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 64 KiB/s wr, 38 op/s 2024-04-23T18:54:36.024 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:10 smithi033 ceph-mon[29274]: osdmap e139: 12 total, 12 up, 12 in 2024-04-23T18:54:36.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:11 smithi033 ceph-mon[29274]: pgmap v863: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 20 KiB/s wr, 9 op/s 2024-04-23T18:54:36.227 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:13 smithi033 ceph-mon[29274]: pgmap v864: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 18 op/s 2024-04-23T18:54:36.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:15 smithi033 ceph-mon[29274]: pgmap v865: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-23T18:54:36.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:17 smithi033 ceph-mon[29274]: pgmap v866: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:54:36.384 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:17 smithi033 ceph-mon[29274]: osdmap e140: 12 total, 12 up, 12 in 2024-04-23T18:54:36.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:19 smithi033 ceph-mon[29274]: pgmap v868: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.0 KiB/s wr, 12 op/s 2024-04-23T18:54:36.399 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:21 smithi033 ceph-mon[29274]: pgmap v869: 97 pgs: 97 active+clean; 15 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 11 op/s 2024-04-23T18:54:36.409 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:24 smithi106 ceph-mon[32716]: pgmap v905: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:54:36.504 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:26 smithi106 ceph-mon[32716]: pgmap v906: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:54:36.512 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:28 smithi106 ceph-mon[32716]: pgmap v907: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 356 B/s wr, 13 op/s 2024-04-23T18:54:36.614 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:30 smithi106 ceph-mon[32716]: pgmap v908: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:36.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:32 smithi106 ceph-mon[32716]: pgmap v909: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:36.628 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:34 smithi106 ceph-mon[32716]: pgmap v910: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:54:36.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:24 smithi122 ceph-mon[32546]: pgmap v905: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:54:36.753 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:26 smithi122 ceph-mon[32546]: pgmap v906: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:54:36.815 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:28 smithi122 ceph-mon[32546]: pgmap v907: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 356 B/s wr, 13 op/s 2024-04-23T18:54:36.877 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:30 smithi122 ceph-mon[32546]: pgmap v908: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:36.908 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:32 smithi122 ceph-mon[32546]: pgmap v909: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:37.053 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:34 smithi122 ceph-mon[32546]: pgmap v910: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:54:37.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:36 smithi106 ceph-mon[32716]: pgmap v911: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:37.146 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:36 smithi122 ceph-mon[32546]: pgmap v911: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 -- 172.21.15.33:0/1361956232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc8106170 msgr2=0x7fbbc810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:37.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 --2- 172.21.15.33:0/1361956232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc8106170 0x7fbbc810dd40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc002a410 comp rx=0 tx=0).stop 2024-04-23T18:54:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 -- 172.21.15.33:0/1361956232 shutdown_connections 2024-04-23T18:54:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 --2- 172.21.15.33:0/1361956232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc8106170 0x7fbbc810dd40 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 --2- 172.21.15.33:0/1361956232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc8101aa0 0x7fbbc8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:37.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 --2- 172.21.15.33:0/1361956232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc81010f0 0x7fbbc81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:37.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 -- 172.21.15.33:0/1361956232 >> 172.21.15.33:0/1361956232 conn(0x7fbbc8077fd0 msgr2=0x7fbbc80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:37.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 -- 172.21.15.33:0/1361956232 shutdown_connections 2024-04-23T18:54:37.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.995+0000 7fbbd0659640 1 -- 172.21.15.33:0/1361956232 wait complete. 2024-04-23T18:54:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 Processor -- start 2024-04-23T18:54:37.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 -- start start 2024-04-23T18:54:37.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc81010f0 0x7fbbc81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc8106170 0x7fbbc8198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:37.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81121f0 con 0x7fbbc819cf40 2024-04-23T18:54:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8112070 con 0x7fbbc8106170 2024-04-23T18:54:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbd0659640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8112370 con 0x7fbbc81010f0 2024-04-23T18:54:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbcebf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbcebf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38654/0 (socket says 172.21.15.33:38654) 2024-04-23T18:54:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbcebf6640 1 -- 172.21.15.33:0/3640941551 learned_addr learned my addr 172.21.15.33:0/3640941551 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:38.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.996+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc8106170 0x7fbbc8198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc81010f0 0x7fbbc81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 -- 172.21.15.33:0/3640941551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc81010f0 msgr2=0x7fbbc81986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:38.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc81010f0 0x7fbbc81986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 -- 172.21.15.33:0/3640941551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc8106170 msgr2=0x7fbbc8198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc8106170 0x7fbbc8198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:38.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 -- 172.21.15.33:0/3640941551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc819da30 con 0x7fbbc819cf40 2024-04-23T18:54:38.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbcebf6640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0004770 tx=0x7fbbc0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:38.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbc77fe640 1 -- 172.21.15.33:0/3640941551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc003f030 con 0x7fbbc819cf40 2024-04-23T18:54:38.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc819dcc0 con 0x7fbbc819cf40 2024-04-23T18:54:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbc77fe640 1 -- 172.21.15.33:0/3640941551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0003aa0 con 0x7fbbc819cf40 2024-04-23T18:54:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbc77fe640 1 -- 172.21.15.33:0/3640941551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0035070 con 0x7fbbc819cf40 2024-04-23T18:54:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:21.997+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a5480 con 0x7fbbc819cf40 2024-04-23T18:54:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.000+0000 7fbbc77fe640 1 -- 172.21.15.33:0/3640941551 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbc0042030 con 0x7fbbc819cf40 2024-04-23T18:54:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.000+0000 7fbbd336f740 1 Processor -- start 2024-04-23T18:54:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 -- start start 2024-04-23T18:54:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f8f620 0x5594d0f8fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f9a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 0x5594d0f9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5594d0ef4970 con 0x5594d0f9a800 2024-04-23T18:54:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594d0e914a0 con 0x5594d0f8ff40 2024-04-23T18:54:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbd336f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5594d0ef0930 con 0x5594d0f8f620 2024-04-23T18:54:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbc77fe640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbcdbf4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 0x5594d0f9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:39.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbce3f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f9a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbcdbf4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 0x5594d0f9cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38656/0 (socket says 172.21.15.33:38656) 2024-04-23T18:54:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbcdbf4640 1 -- 172.21.15.33:0/79928167 learned_addr learned my addr 172.21.15.33:0/79928167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbcebf6640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f8f620 0x5594d0f8fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:39.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.001+0000 7fbbc77fe640 1 -- 172.21.15.33:0/3640941551 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fbbc00f0d60 con 0x7fbbc819cf40 2024-04-23T18:54:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 -- 172.21.15.33:0/79928167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f8f620 msgr2=0x5594d0f8fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:39.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f8f620 0x5594d0f8fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:39.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 -- 172.21.15.33:0/79928167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 msgr2=0x5594d0f9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 0x5594d0f9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:39.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 -- 172.21.15.33:0/79928167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d0c07210 con 0x5594d0f8ff40 2024-04-23T18:54:39.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbcebf6640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00bcaa0 tx=0x7fbbc002a410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f9a2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009b50 tx=0x7fbbb8003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f182855d9a00c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:40.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbc4bfe640 1 -- 172.21.15.33:0/79928167 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8009e40 con 0x5594d0f8ff40 2024-04-23T18:54:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbc4bfe640 1 -- 172.21.15.33:0/79928167 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004620 con 0x5594d0f8ff40 2024-04-23T18:54:40.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.002+0000 7fbbc4bfe640 1 -- 172.21.15.33:0/79928167 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8005480 con 0x5594d0f8ff40 2024-04-23T18:54:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- 172.21.15.33:0/79928167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 msgr2=0x5594d0f9a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f9a2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009b50 tx=0x7fbbb8003d40 comp rx=0 tx=0).stop 2024-04-23T18:54:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- 172.21.15.33:0/79928167 shutdown_connections 2024-04-23T18:54:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f9a800 0x5594d0f9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f9a2c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 --2- 172.21.15.33:0/79928167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f8f620 0x5594d0f8fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:40.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- 172.21.15.33:0/79928167 >> 172.21.15.33:0/79928167 conn(0x5594d0f8bb30 msgr2=0x5594d0f8bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:40.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- 172.21.15.33:0/79928167 shutdown_connections 2024-04-23T18:54:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- 172.21.15.33:0/79928167 wait complete. 2024-04-23T18:54:40.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.003+0000 7fbbd336f740 1 -- start start 2024-04-23T18:54:40.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 Processor -- start 2024-04-23T18:54:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f97370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:40.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0f978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:40.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5594d0f9d430 con 0x5594d0f8f620 2024-04-23T18:54:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594d0e8f0b0 con 0x5594d0f8ff40 2024-04-23T18:54:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5594d0f9d5b0 con 0x5594d0f9a800 2024-04-23T18:54:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbd336f740 10 client.0 _ll_register_callbacks cb 0x5594d0e7cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:41.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38670/0 (socket says 172.21.15.33:38670) 2024-04-23T18:54:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 -- 172.21.15.33:0/1289492661 learned_addr learned my addr 172.21.15.33:0/1289492661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f97370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:41.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 -- 172.21.15.33:0/1289492661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 msgr2=0x5594d0f978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:41.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0f978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:41.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 -- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 msgr2=0x5594d0f97370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:41.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f97370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 -- 172.21.15.33:0/1289492661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d0cd3620 con 0x5594d0f8f620 2024-04-23T18:54:41.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f97370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc005f60 tx=0x7fbbbc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09af31b77346cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.004+0000 7fbbadffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc012950 con 0x5594d0f8f620 2024-04-23T18:54:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fbbbc012af0 con 0x5594d0f8f620 2024-04-23T18:54:41.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb8004620 con 0x5594d0f8f620 2024-04-23T18:54:41.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbbbc012cf0 con 0x5594d0f8f620 2024-04-23T18:54:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 10 client.18324 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5594d0fa9310 con 0x5594d0f8f620 2024-04-23T18:54:41.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004620 con 0x5594d0f8f620 2024-04-23T18:54:41.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fbbbc01d030 con 0x5594d0f8f620 2024-04-23T18:54:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 10 client.18324 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 20 client.18324 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d0faa7d0 0x5594d0facc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 4 client.18324 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:54:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.005+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594d0f02390 con 0x5594d0faa7d0 2024-04-23T18:54:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.006+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d0faa7d0 0x5594d0facc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.006+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d0faa7d0 0x5594d0facc00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e88c1500ea02cd server_cookie=9729285041893f5e in_seq=0 out_seq=0 2024-04-23T18:54:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.006+0000 7fbbadffb640 10 client.18324 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbadffb640 1 -- 172.21.15.33:0/1289492661 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594d0f02390 con 0x5594d0faa7d0 2024-04-23T18:54:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbadffb640 10 client.18324 handle_command_reply: tid=0 2024-04-23T18:54:42.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 client.18324 shutdown 2024-04-23T18:54:42.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 msgr2=0x5594d0f9a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc005f60 tx=0x7fbbbc003a80 comp rx=0 tx=0).stop 2024-04-23T18:54:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 shutdown_connections 2024-04-23T18:54:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d0faa7d0 0x5594d0facc00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:42.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0f978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:42.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbadffb640 0 client.18324 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:42.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8ff40 0x5594d0f97370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:42.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 --2- 172.21.15.33:0/1289492661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8f620 0x5594d0f9a080 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:43.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 >> 172.21.15.33:0/1289492661 conn(0x5594d0f8bb30 msgr2=0x5594d0f91860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 shutdown_connections 2024-04-23T18:54:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.007+0000 7fbbd336f740 1 -- 172.21.15.33:0/1289492661 wait complete. 2024-04-23T18:54:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.008+0000 7fbbd336f740 20 client.18324 trim_cache size 0 max 16384 2024-04-23T18:54:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 Processor -- start 2024-04-23T18:54:43.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 -- start start 2024-04-23T18:54:43.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0f8aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fb2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:43.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81aa340 con 0x5594d0f8ff40 2024-04-23T18:54:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81acc20 con 0x5594d0f8f620 2024-04-23T18:54:43.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbd336f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81aaa10 con 0x5594d0f9a800 2024-04-23T18:54:43.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:43.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38674/0 (socket says 172.21.15.33:38674) 2024-04-23T18:54:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2826560491 learned_addr learned my addr 172.21.15.33:0/2826560491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:43.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0f8aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:43.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2826560491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 msgr2=0x5594d0fb2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:43.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fb2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:43.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2826560491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 msgr2=0x5594d0f8aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:43.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0f8aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:43.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2826560491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d0f958e0 con 0x5594d0f8ff40 2024-04-23T18:54:44.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0f8aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8004870 tx=0x7fbbb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f53e8f45ea98c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbaffff640 1 -- 172.21.15.33:0/2826560491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8035070 con 0x5594d0f8ff40 2024-04-23T18:54:44.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbaffff640 1 -- 172.21.15.33:0/2826560491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004e50 con 0x5594d0f8ff40 2024-04-23T18:54:44.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.023+0000 7fbbaffff640 1 -- 172.21.15.33:0/2826560491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8030570 con 0x5594d0f8ff40 2024-04-23T18:54:44.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- 172.21.15.33:0/2826560491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 msgr2=0x5594d0fa62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8004870 tx=0x7fbbb8005fd0 comp rx=0 tx=0).stop 2024-04-23T18:54:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- 172.21.15.33:0/2826560491 shutdown_connections 2024-04-23T18:54:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fb2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:44.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa62f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:44.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2826560491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0f8aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:44.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- 172.21.15.33:0/2826560491 >> 172.21.15.33:0/2826560491 conn(0x5594d0f8bb30 msgr2=0x5594d0f97490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- 172.21.15.33:0/2826560491 shutdown_connections 2024-04-23T18:54:44.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- 172.21.15.33:0/2826560491 wait complete. 2024-04-23T18:54:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 -- start start 2024-04-23T18:54:44.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.024+0000 7fbbd336f740 1 Processor -- start 2024-04-23T18:54:44.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0fa0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:44.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fa0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:44.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ab0e0 con 0x5594d0f8ff40 2024-04-23T18:54:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ac550 con 0x5594d0f8f620 2024-04-23T18:54:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81ab7b0 con 0x5594d0f9a800 2024-04-23T18:54:45.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 10 client.0 _ll_register_callbacks cb 0x7fbbbc004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:45.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38690/0 (socket says 172.21.15.33:38690) 2024-04-23T18:54:45.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2516263469 learned_addr learned my addr 172.21.15.33:0/2516263469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0fa0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2516263469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 msgr2=0x5594d0fa0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fa0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 msgr2=0x5594d0fa0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0fa0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:45.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 -- 172.21.15.33:0/2516263469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d100fd90 con 0x5594d0f8ff40 2024-04-23T18:54:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0fa0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:45.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbce3f5640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000f80 tx=0x7fbbb8017e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e956b430d27c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbadffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8035040 con 0x5594d0f8ff40 2024-04-23T18:54:45.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fbbb80049a0 con 0x5594d0f8ff40 2024-04-23T18:54:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.025+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbb8004e50 con 0x5594d0f8ff40 2024-04-23T18:54:45.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbbb8025040 con 0x5594d0f8ff40 2024-04-23T18:54:46.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 10 client.18330 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5594d1010fe0 con 0x5594d0f8ff40 2024-04-23T18:54:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbb8004e50 con 0x5594d0f8ff40 2024-04-23T18:54:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fbbb801f070 con 0x5594d0f8ff40 2024-04-23T18:54:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 10 client.18330 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 10 client.18330 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 20 client.18330 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d1011650 0x5594d1013a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 4 client.18330 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:54:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594d0efc200 con 0x5594d1011650 2024-04-23T18:54:46.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.026+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d1011650 0x5594d1013a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.027+0000 7fbbadffb640 10 client.18330 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.027+0000 7fbbcdbf4640 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d1011650 0x5594d1013a70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1a2e15c527554f server_cookie=cf3f666459bc6ba0 in_seq=0 out_seq=0 2024-04-23T18:54:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbadffb640 1 -- 172.21.15.33:0/2516263469 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594d0efc200 con 0x5594d1011650 2024-04-23T18:54:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbadffb640 10 client.18330 handle_command_reply: tid=0 2024-04-23T18:54:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 client.18330 shutdown 2024-04-23T18:54:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 msgr2=0x5594d0fa07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:46.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8000f80 tx=0x7fbbb8017e80 comp rx=0 tx=0).stop 2024-04-23T18:54:46.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 shutdown_connections 2024-04-23T18:54:46.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5594d1011650 0x5594d1013a70 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5594d0f9a800 0x5594d0fa0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5594d0f8ff40 0x5594d0fa07b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbadffb640 0 client.18330 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 --2- 172.21.15.33:0/2516263469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594d0f8f620 0x5594d0fa0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 >> 172.21.15.33:0/2516263469 conn(0x5594d0f8bb30 msgr2=0x5594d0fa52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 shutdown_connections 2024-04-23T18:54:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.033+0000 7fbbd336f740 1 -- 172.21.15.33:0/2516263469 wait complete. 2024-04-23T18:54:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd336f740 20 client.18330 trim_cache size 0 max 16384 2024-04-23T18:54:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:54:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:54:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:54:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:54:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 msgr2=0x7fbbb00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00bcaa0 tx=0x7fbbc002a410 comp rx=0 tx=0).stop 2024-04-23T18:54:47.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 msgr2=0x7fbbc819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:47.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0004770 tx=0x7fbbc0004270 comp rx=0 tx=0).stop 2024-04-23T18:54:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 shutdown_connections 2024-04-23T18:54:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbbb00a3dc0 0x7fbbb00a6280 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:48.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbc819cf40 0x7fbbc819d3f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbc8106170 0x7fbbc8198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 --2- 172.21.15.33:0/3640941551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbbc81010f0 0x7fbbc81986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:48.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 >> 172.21.15.33:0/3640941551 conn(0x7fbbc8077fd0 msgr2=0x7fbbc80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:48.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 shutdown_connections 2024-04-23T18:54:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:22.034+0000 7fbbd0659640 1 -- 172.21.15.33:0/3640941551 wait complete. 2024-04-23T18:54:48.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:23 smithi033 ceph-mon[29274]: pgmap v870: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:48.390 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:25 smithi033 ceph-mon[29274]: pgmap v871: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T18:54:48.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:27 smithi033 ceph-mon[29274]: pgmap v872: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:54:48.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:29 smithi033 ceph-mon[29274]: pgmap v873: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:48.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:31 smithi033 ceph-mon[29274]: pgmap v874: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:48.515 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:33 smithi033 ceph-mon[29274]: pgmap v875: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 341 B/s wr, 12 op/s 2024-04-23T18:54:48.525 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:38 smithi106 ceph-mon[32716]: pgmap v912: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:48.534 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:40 smithi106 ceph-mon[32716]: pgmap v913: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:48.599 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:42 smithi106 ceph-mon[32716]: pgmap v914: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:48.607 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:44 smithi106 ceph-mon[32716]: pgmap v915: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:48.660 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:46 smithi106 ceph-mon[32716]: pgmap v916: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:54:48.692 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:38 smithi122 ceph-mon[32546]: pgmap v912: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:48.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:40 smithi122 ceph-mon[32546]: pgmap v913: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:48.937 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:42 smithi122 ceph-mon[32546]: pgmap v914: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:49.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:44 smithi122 ceph-mon[32546]: pgmap v915: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:49.098 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:46 smithi122 ceph-mon[32546]: pgmap v916: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:54:49.106 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:48 smithi122 ceph-mon[32546]: pgmap v917: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:49.117 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:48 smithi106 ceph-mon[32716]: pgmap v917: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:49.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 -- 172.21.15.33:0/3339577886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5568107280 msgr2=0x7f556810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 --2- 172.21.15.33:0/3339577886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5568107280 0x7f556810ee50 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ca0 tx=0x7f556002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:54:49.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 -- 172.21.15.33:0/3339577886 shutdown_connections 2024-04-23T18:54:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 --2- 172.21.15.33:0/3339577886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5568107280 0x7f556810ee50 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:49.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 --2- 172.21.15.33:0/3339577886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5568101070 0x7f5568106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 --2- 172.21.15.33:0/3339577886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55681006c0 0x7f5568100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:49.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 -- 172.21.15.33:0/3339577886 >> 172.21.15.33:0/3339577886 conn(0x7f5568077390 msgr2=0x7f55680fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.134+0000 7f556d649640 1 -- 172.21.15.33:0/3339577886 shutdown_connections 2024-04-23T18:54:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 -- 172.21.15.33:0/3339577886 wait complete. 2024-04-23T18:54:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 Processor -- start 2024-04-23T18:54:49.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 -- start start 2024-04-23T18:54:49.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55681006c0 0x7f5568198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:49.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 0x7f5568198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:49.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:49.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55681121d0 con 0x7f556819cf30 2024-04-23T18:54:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5568112050 con 0x7f5568107280 2024-04-23T18:54:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.135+0000 7f556d649640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5568112350 con 0x7f55681006c0 2024-04-23T18:54:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57582/0 (socket says 172.21.15.33:57582) 2024-04-23T18:54:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 -- 172.21.15.33:0/75650060 learned_addr learned my addr 172.21.15.33:0/75650060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:49.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f5566ffd640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55681006c0 0x7f5568198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:49.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f555e7fc640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 0x7f5568198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:50.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 -- 172.21.15.33:0/75650060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55681006c0 msgr2=0x7f5568198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55681006c0 0x7f5568198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:50.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 -- 172.21.15.33:0/75650060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 msgr2=0x7f5568198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:50.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 0x7f5568198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 -- 172.21.15.33:0/75650060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819da20 con 0x7f556819cf30 2024-04-23T18:54:50.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f555e7fc640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 0x7f5568198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:50.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f55677fe640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f55600098d0 tx=0x7f55600317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f5564ff9640 1 -- 172.21.15.33:0/75650060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560003b70 con 0x7f556819cf30 2024-04-23T18:54:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819dcb0 con 0x7f556819cf30 2024-04-23T18:54:50.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f5564ff9640 1 -- 172.21.15.33:0/75650060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560003d10 con 0x7f556819cf30 2024-04-23T18:54:50.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.136+0000 7f5564ff9640 1 -- 172.21.15.33:0/75650060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556002fa20 con 0x7f556819cf30 2024-04-23T18:54:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.137+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a54d0 con 0x7f556819cf30 2024-04-23T18:54:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.139+0000 7f5564ff9640 1 -- 172.21.15.33:0/75650060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f556002fc40 con 0x7f556819cf30 2024-04-23T18:54:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.139+0000 7f5570433740 1 Processor -- start 2024-04-23T18:54:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.139+0000 7f5570433740 1 -- start start 2024-04-23T18:54:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 0x5585ec257a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257f40 0x5585ec2622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5564ff9640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 0x7f55540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:50.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:50.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5585ec1bc970 con 0x5585ec262800 2024-04-23T18:54:50.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585ec1594a0 con 0x5585ec257620 2024-04-23T18:54:50.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5570433740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585ec1b8930 con 0x5585ec257f40 2024-04-23T18:54:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57586/0 (socket says 172.21.15.33:57586) 2024-04-23T18:54:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 -- 172.21.15.33:0/2634637390 learned_addr learned my addr 172.21.15.33:0/2634637390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5566ffd640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 0x7f55540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5564ff9640 1 -- 172.21.15.33:0/75650060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7f55600f1bc0 con 0x7f556819cf30 2024-04-23T18:54:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f555e7fc640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 0x5585ec257a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f555e7fc640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257f40 0x5585ec2622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 -- 172.21.15.33:0/2634637390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257f40 msgr2=0x5585ec2622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257f40 0x5585ec2622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:51.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 -- 172.21.15.33:0/2634637390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 msgr2=0x5585ec257a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 0x5585ec257a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:51.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 -- 172.21.15.33:0/2634637390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ebecf210 con 0x5585ec262800 2024-04-23T18:54:51.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f555e7fc640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 0x5585ec257a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:51.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f5566ffd640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 0x7f55540a6280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5558009bd0 tx=0x7f5558005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:51.225 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.140+0000 7f55677fe640 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5560005cc0 tx=0x7f5560009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53d5a63a8ffc1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f555d7fa640 1 -- 172.21.15.33:0/2634637390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600bd5c0 con 0x5585ec262800 2024-04-23T18:54:51.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f555d7fa640 1 -- 172.21.15.33:0/2634637390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560036070 con 0x5585ec262800 2024-04-23T18:54:51.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f555d7fa640 1 -- 172.21.15.33:0/2634637390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55600bbda0 con 0x5585ec262800 2024-04-23T18:54:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 -- 172.21.15.33:0/2634637390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 msgr2=0x5585ec264bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5560005cc0 tx=0x7f5560009980 comp rx=0 tx=0).stop 2024-04-23T18:54:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 -- 172.21.15.33:0/2634637390 shutdown_connections 2024-04-23T18:54:51.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec262800 0x5585ec264bf0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:51.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257f40 0x5585ec2622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:51.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 --2- 172.21.15.33:0/2634637390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257620 0x5585ec257a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:51.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.141+0000 7f5570433740 1 -- 172.21.15.33:0/2634637390 >> 172.21.15.33:0/2634637390 conn(0x5585ec253b30 msgr2=0x5585ec253f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:51.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- 172.21.15.33:0/2634637390 shutdown_connections 2024-04-23T18:54:51.743 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- 172.21.15.33:0/2634637390 wait complete. 2024-04-23T18:54:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- start start 2024-04-23T18:54:51.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 Processor -- start 2024-04-23T18:54:51.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec262190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:51.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec25fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5585ec1bc970 con 0x5585ec257f40 2024-04-23T18:54:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585ec1570b0 con 0x5585ec262800 2024-04-23T18:54:52.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585ec2651f0 con 0x5585ec257620 2024-04-23T18:54:52.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5570433740 10 client.0 _ll_register_callbacks cb 0x5585ec144d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f555e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:52.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f555e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57594/0 (socket says 172.21.15.33:57594) 2024-04-23T18:54:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f555e7fc640 1 -- 172.21.15.33:0/3492609147 learned_addr learned my addr 172.21.15.33:0/3492609147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.142+0000 7f5566ffd640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec262190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f55677fe640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec25fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 -- 172.21.15.33:0/3492609147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 msgr2=0x5585ec262190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec262190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 -- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 msgr2=0x5585ec25fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec25fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 -- 172.21.15.33:0/3492609147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ebf9b620 con 0x5585ec257f40 2024-04-23T18:54:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f55677fe640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec25fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f5566ffd640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec262190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:54:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f555e7fc640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5550012790 tx=0x7f5550012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254821ca3024cbee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f554e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500119c0 con 0x5585ec257f40 2024-04-23T18:54:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7f5550011b60 con 0x5585ec257f40 2024-04-23T18:54:52.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5560036070 con 0x5585ec257f40 2024-04-23T18:54:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5550024050 con 0x5585ec257f40 2024-04-23T18:54:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f5570433740 10 client.18342 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:52.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5585ec270f50 con 0x5585ec257f40 2024-04-23T18:54:52.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.143+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560036070 con 0x5585ec257f40 2024-04-23T18:54:52.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f555000f420 con 0x5585ec257f40 2024-04-23T18:54:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 10 client.18342 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 10 client.18342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 20 client.18342 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec272480 0x5585ec274870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:53.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 4 client.18342 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:54:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ec1ca390 con 0x5585ec272480 2024-04-23T18:54:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5566ffd640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec272480 0x5585ec274870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:53.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f5566ffd640 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec272480 0x5585ec274870 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1cc402be9e3d449 server_cookie=279022bdb171af28 in_seq=0 out_seq=0 2024-04-23T18:54:53.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.144+0000 7f554e7fc640 10 client.18342 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f554e7fc640 1 -- 172.21.15.33:0/3492609147 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585ec1ca390 con 0x5585ec272480 2024-04-23T18:54:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f554e7fc640 10 client.18342 handle_command_reply: tid=0 2024-04-23T18:54:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 client.18342 shutdown 2024-04-23T18:54:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 msgr2=0x5585ec25f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5550012790 tx=0x7f5550012c50 comp rx=0 tx=0).stop 2024-04-23T18:54:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 shutdown_connections 2024-04-23T18:54:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec272480 0x5585ec274870 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec25fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:53.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec25f480 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:53.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 --2- 172.21.15.33:0/3492609147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec262190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:53.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f554e7fc640 0 client.18342 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 >> 172.21.15.33:0/3492609147 conn(0x5585ec253b30 msgr2=0x5585ec2598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 shutdown_connections 2024-04-23T18:54:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.145+0000 7f5570433740 1 -- 172.21.15.33:0/3492609147 wait complete. 2024-04-23T18:54:53.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.146+0000 7f5570433740 20 client.18342 trim_cache size 0 max 16384 2024-04-23T18:54:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.160+0000 7f5570433740 1 Processor -- start 2024-04-23T18:54:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 -- start start 2024-04-23T18:54:54.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec26de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec27a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55681ac640 con 0x5585ec257f40 2024-04-23T18:54:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55681aef20 con 0x5585ec262800 2024-04-23T18:54:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5570433740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55681acd10 con 0x5585ec257620 2024-04-23T18:54:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57610/0 (socket says 172.21.15.33:57610) 2024-04-23T18:54:54.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 -- 172.21.15.33:0/1665185862 learned_addr learned my addr 172.21.15.33:0/1665185862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:54.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f55677fe640 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec27a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:54.439 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f5566ffd640 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec26de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 -- 172.21.15.33:0/1665185862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 msgr2=0x5585ec26de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec26de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 -- 172.21.15.33:0/1665185862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 msgr2=0x5585ec27a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec27a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.161+0000 7f555e7fc640 1 -- 172.21.15.33:0/1665185862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ec25fd30 con 0x5585ec257f40 2024-04-23T18:54:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f555e7fc640 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f555000f9e0 tx=0x7f555000caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca7d70f8b9374cd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:54.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f555cff9640 1 -- 172.21.15.33:0/1665185862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5550004300 con 0x5585ec257f40 2024-04-23T18:54:54.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f555cff9640 1 -- 172.21.15.33:0/1665185862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550016070 con 0x5585ec257f40 2024-04-23T18:54:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f555cff9640 1 -- 172.21.15.33:0/1665185862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55500052b0 con 0x5585ec257f40 2024-04-23T18:54:55.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f5570433740 1 -- 172.21.15.33:0/1665185862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 msgr2=0x5585ec261cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f5570433740 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f555000f9e0 tx=0x7f555000caa0 comp rx=0 tx=0).stop 2024-04-23T18:54:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f5570433740 1 -- 172.21.15.33:0/1665185862 shutdown_connections 2024-04-23T18:54:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.162+0000 7f5570433740 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec262800 0x5585ec27a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257f40 0x5585ec261cc0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:55.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 --2- 172.21.15.33:0/1665185862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec257620 0x5585ec26de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- 172.21.15.33:0/1665185862 >> 172.21.15.33:0/1665185862 conn(0x5585ec253b30 msgr2=0x5585ec253f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- 172.21.15.33:0/1665185862 shutdown_connections 2024-04-23T18:54:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- 172.21.15.33:0/1665185862 wait complete. 2024-04-23T18:54:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- start start 2024-04-23T18:54:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 Processor -- start 2024-04-23T18:54:55.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:55.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 0x5585ec266540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec262800 0x5585ec266a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55681ad3e0 con 0x5585ec257620 2024-04-23T18:54:55.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55681ae850 con 0x5585ec257f40 2024-04-23T18:54:55.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55681adab0 con 0x5585ec262800 2024-04-23T18:54:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5570433740 10 client.0 _ll_register_callbacks cb 0x5585ec144d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:54:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57616/0 (socket says 172.21.15.33:57616) 2024-04-23T18:54:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.163+0000 7f5566ffd640 1 -- 172.21.15.33:0/510010625 learned_addr learned my addr 172.21.15.33:0/510010625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:54:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f555e7fc640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 0x5585ec266540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 -- 172.21.15.33:0/510010625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec262800 msgr2=0x5585ec266a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec262800 0x5585ec266a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 -- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 msgr2=0x5585ec266540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:54:56.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 0x5585ec266540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 -- 172.21.15.33:0/510010625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ec2d7c00 con 0x5585ec257620 2024-04-23T18:54:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f555e7fc640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 0x5585ec266540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:54:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5566ffd640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f555800e570 tx=0x7f5558004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405fc5c71de54927 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:54:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:54:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800e8b0 con 0x5585ec257620 2024-04-23T18:54:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7f555800ea50 con 0x5585ec257620 2024-04-23T18:54:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5550016070 con 0x5585ec257620 2024-04-23T18:54:56.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5558036050 con 0x5585ec257620 2024-04-23T18:54:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 10 client.18348 fetch_fsmap learned FSMap version 45 2024-04-23T18:54:56.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5585ec2d8f30 con 0x5585ec257620 2024-04-23T18:54:56.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5550016070 con 0x5585ec257620 2024-04-23T18:54:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7f5558032cd0 con 0x5585ec257620 2024-04-23T18:54:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 10 client.18348 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:54:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 10 client.18348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:54:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 20 client.18348 populate_metadata read hostname 'smithi033' 2024-04-23T18:54:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.164+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec2d9630 0x5585ec2dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:54:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.165+0000 7f5570433740 4 client.18348 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:54:56.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.165+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585ec1c4200 con 0x5585ec2d9630 2024-04-23T18:54:56.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.165+0000 7f555e7fc640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec2d9630 0x5585ec2dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:54:56.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.165+0000 7f554e7fc640 10 client.18348 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.165+0000 7f555e7fc640 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec2d9630 0x5585ec2dbab0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698c98cc74d68520 server_cookie=8497d10383d98910 in_seq=0 out_seq=0 2024-04-23T18:54:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f554e7fc640 1 -- 172.21.15.33:0/510010625 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585ec1c4200 con 0x5585ec2d9630 2024-04-23T18:54:57.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f554e7fc640 10 client.18348 handle_command_reply: tid=0 2024-04-23T18:54:57.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 client.18348 shutdown 2024-04-23T18:54:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 msgr2=0x5585ec266000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f555800e570 tx=0x7f5558004730 comp rx=0 tx=0).stop 2024-04-23T18:54:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 shutdown_connections 2024-04-23T18:54:57.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5585ec2d9630 0x5585ec2dbab0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:57.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585ec262800 0x5585ec266a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585ec257f40 0x5585ec266540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:57.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 --2- 172.21.15.33:0/510010625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5585ec257620 0x5585ec266000 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f554e7fc640 0 client.18348 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:54:57.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 >> 172.21.15.33:0/510010625 conn(0x5585ec253b30 msgr2=0x5585ec252420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:57.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 shutdown_connections 2024-04-23T18:54:57.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 1 -- 172.21.15.33:0/510010625 wait complete. 2024-04-23T18:54:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f5570433740 20 client.18348 trim_cache size 0 max 16384 2024-04-23T18:54:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:54:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:54:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:54:58.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:54:58.065 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:50 smithi106 ceph-mon[32716]: pgmap v918: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:52 smithi106 ceph-mon[32716]: pgmap v919: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.225 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:54 smithi106 ceph-mon[32716]: pgmap v920: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:58.233 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:56 smithi106 ceph-mon[32716]: pgmap v921: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:50 smithi122 ceph-mon[32546]: pgmap v918: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.251 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:52 smithi122 ceph-mon[32546]: pgmap v919: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.350 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:54 smithi122 ceph-mon[32546]: pgmap v920: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:58.360 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:56 smithi122 ceph-mon[32546]: pgmap v921: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:58.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 msgr2=0x7f55540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 0x7f55540a6280 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5558009bd0 tx=0x7f5558005f90 comp rx=0 tx=0).stop 2024-04-23T18:54:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 msgr2=0x7f556819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.170+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f55600098d0 tx=0x7f55600317b0 comp rx=0 tx=0).stop 2024-04-23T18:54:58.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 shutdown_connections 2024-04-23T18:54:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f55540a3dc0 0x7f55540a6280 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:58.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f556819cf30 0x7f556819d3e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5568107280 0x7f5568198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 --2- 172.21.15.33:0/75650060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55681006c0 0x7f5568198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 >> 172.21.15.33:0/75650060 conn(0x7f5568077390 msgr2=0x7f5568105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:58.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 shutdown_connections 2024-04-23T18:54:59.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:35.171+0000 7f556d649640 1 -- 172.21.15.33:0/75650060 wait complete. 2024-04-23T18:54:59.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:35 smithi033 ceph-mon[29274]: pgmap v876: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:54:59.085 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:37 smithi033 ceph-mon[29274]: pgmap v877: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:54:59.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:39 smithi033 ceph-mon[29274]: pgmap v878: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:59.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:41 smithi033 ceph-mon[29274]: pgmap v879: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:59.173 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:43 smithi033 ceph-mon[29274]: pgmap v880: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:54:59.231 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:45 smithi033 ceph-mon[29274]: pgmap v881: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:54:59.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:47 smithi033 ceph-mon[29274]: pgmap v882: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:54:59.294 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:54:58 smithi106 ceph-mon[32716]: pgmap v922: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:59.305 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:54:58 smithi122 ceph-mon[32546]: pgmap v922: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:54:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- 172.21.15.33:0/3125185665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0341031c0 msgr2=0x7fd0341035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:54:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 --2- 172.21.15.33:0/3125185665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0341031c0 0x7fd0341035a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd024009900 tx=0x7fd02402a480 comp rx=0 tx=0).stop 2024-04-23T18:54:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- 172.21.15.33:0/3125185665 shutdown_connections 2024-04-23T18:54:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 --2- 172.21.15.33:0/3125185665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd03410c590 0x7fd03410e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:59.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 --2- 172.21.15.33:0/3125185665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd034103ae0 0x7fd03410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:59.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 --2- 172.21.15.33:0/3125185665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0341031c0 0x7fd0341035a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:54:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- 172.21.15.33:0/3125185665 >> 172.21.15.33:0/3125185665 conn(0x7fd034077fd0 msgr2=0x7fd0340783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:54:59.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- 172.21.15.33:0/3125185665 shutdown_connections 2024-04-23T18:54:59.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- 172.21.15.33:0/3125185665 wait complete. 2024-04-23T18:54:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 Processor -- start 2024-04-23T18:54:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.272+0000 7fd03a38e640 1 -- start start 2024-04-23T18:54:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 0x7fd034198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 0x7fd034198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:00.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:00.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341121c0 con 0x7fd03410c590 2024-04-23T18:55:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd034112040 con 0x7fd0341031c0 2024-04-23T18:55:00.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03a38e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd034112340 con 0x7fd034103ae0 2024-04-23T18:55:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51128/0 (socket says 172.21.15.33:51128) 2024-04-23T18:55:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 -- 172.21.15.33:0/2168884693 learned_addr learned my addr 172.21.15.33:0/2168884693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd033fff640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 0x7fd034198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd0337fe640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 0x7fd034198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 -- 172.21.15.33:0/2168884693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 msgr2=0x7fd034198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 0x7fd034198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 -- 172.21.15.33:0/2168884693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 msgr2=0x7fd034198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 0x7fd034198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 -- 172.21.15.33:0/2168884693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419d630 con 0x7fd03410c590 2024-04-23T18:55:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd033fff640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 0x7fd034198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd0337fe640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 0x7fd034198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.273+0000 7fd03892b640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00cce0 tx=0x7fd02c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:00.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.274+0000 7fd0317fa640 1 -- 172.21.15.33:0/2168884693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00f450 con 0x7fd03410c590 2024-04-23T18:55:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.274+0000 7fd0317fa640 1 -- 172.21.15.33:0/2168884693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c00faa0 con 0x7fd03410c590 2024-04-23T18:55:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.274+0000 7fd0317fa640 1 -- 172.21.15.33:0/2168884693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c018b70 con 0x7fd03410c590 2024-04-23T18:55:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.274+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03419d920 con 0x7fd03410c590 2024-04-23T18:55:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.274+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a51b0 con 0x7fd03410c590 2024-04-23T18:55:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.276+0000 7fd0317fa640 1 -- 172.21.15.33:0/2168884693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd02c018d10 con 0x7fd03410c590 2024-04-23T18:55:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.276+0000 7fd03d180740 1 Processor -- start 2024-04-23T18:55:00.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.276+0000 7fd03d180740 1 -- start start 2024-04-23T18:55:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.276+0000 7fd03d180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03d180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03d180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8712000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:01.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03d180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cdc8665970 con 0x55cdc8700520 2024-04-23T18:55:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03d180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cdc86024a0 con 0x55cdc870fc10 2024-04-23T18:55:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03d180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdc8661930 con 0x55cdc8700e40 2024-04-23T18:55:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0317fa640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 0x7fd0200a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:01.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51140/0 (socket says 172.21.15.33:51140) 2024-04-23T18:55:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 -- 172.21.15.33:0/1131196354 learned_addr learned my addr 172.21.15.33:0/1131196354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd033fff640 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:01.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03892b640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 0x7fd0200a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8712000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0317fa640 1 -- 172.21.15.33:0/2168884693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fd02c0d0ca0 con 0x7fd03410c590 2024-04-23T18:55:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 -- 172.21.15.33:0/1131196354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 msgr2=0x55cdc870f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:01.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 -- 172.21.15.33:0/1131196354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 msgr2=0x55cdc8712000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8712000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:01.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 -- 172.21.15.33:0/1131196354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc8378210 con 0x55cdc8700520 2024-04-23T18:55:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0155e0 tx=0x7fd01c009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704b4c16fdb57f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd03892b640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 0x7fd0200a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd02c099090 tx=0x7fd02c06e670 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:02.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.277+0000 7fd02a7fc640 1 -- 172.21.15.33:0/1131196354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0134c0 con 0x55cdc8700520 2024-04-23T18:55:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd02a7fc640 1 -- 172.21.15.33:0/1131196354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c01b070 con 0x55cdc8700520 2024-04-23T18:55:02.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd02a7fc640 1 -- 172.21.15.33:0/1131196354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c012680 con 0x55cdc8700520 2024-04-23T18:55:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 -- 172.21.15.33:0/1131196354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 msgr2=0x55cdc8700900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:02.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0155e0 tx=0x7fd01c009630 comp rx=0 tx=0).stop 2024-04-23T18:55:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 -- 172.21.15.33:0/1131196354 shutdown_connections 2024-04-23T18:55:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8712000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 --2- 172.21.15.33:0/1131196354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc8700900 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 -- 172.21.15.33:0/1131196354 >> 172.21.15.33:0/1131196354 conn(0x55cdc86fcb30 msgr2=0x55cdc86fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 -- 172.21.15.33:0/1131196354 shutdown_connections 2024-04-23T18:55:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.278+0000 7fd03d180740 1 -- 172.21.15.33:0/1131196354 wait complete. 2024-04-23T18:55:02.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 -- start start 2024-04-23T18:55:02.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 Processor -- start 2024-04-23T18:55:03.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:03.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:03.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc870cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cdc8712840 con 0x55cdc8700520 2024-04-23T18:55:03.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cdc86000b0 con 0x55cdc870fc10 2024-04-23T18:55:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdc87129c0 con 0x55cdc8700e40 2024-04-23T18:55:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd03d180740 10 client.0 _ll_register_callbacks cb 0x55cdc85edd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:03.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:03.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd0337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51142/0 (socket says 172.21.15.33:51142) 2024-04-23T18:55:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd0337fe640 1 -- 172.21.15.33:0/1566737434 learned_addr learned my addr 172.21.15.33:0/1566737434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.279+0000 7fd033fff640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:03.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 -- 172.21.15.33:0/1566737434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 msgr2=0x55cdc870c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 -- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 msgr2=0x55cdc870cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc870cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 -- 172.21.15.33:0/1566737434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc8444620 con 0x55cdc8700520 2024-04-23T18:55:03.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005940 tx=0x7fd01c015320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d0a702a9c9e898 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0052f0 con 0x55cdc8700520 2024-04-23T18:55:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fd01c005490 con 0x55cdc8700520 2024-04-23T18:55:04.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd01c01b070 con 0x55cdc8700520 2024-04-23T18:55:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd01c0136d0 con 0x55cdc8700520 2024-04-23T18:55:04.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd03d180740 10 client.18360 fetch_fsmap learned FSMap version 45 2024-04-23T18:55:04.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cdc871e740 con 0x55cdc8700520 2024-04-23T18:55:04.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd01c01b070 con 0x55cdc8700520 2024-04-23T18:55:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.280+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fd01c00c070 con 0x55cdc8700520 2024-04-23T18:55:04.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 10 client.18360 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:55:04.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:04.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 20 client.18360 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:04.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc871fb50 0x55cdc8721f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 4 client.18360 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:55:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc8673390 con 0x55cdc871fb50 2024-04-23T18:55:04.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd033fff640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc871fb50 0x55cdc8721f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:04.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd0137fe640 10 client.18360 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.281+0000 7fd033fff640 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc871fb50 0x55cdc8721f40 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8b490172a14900 server_cookie=9b6a1f95bad94e52 in_seq=0 out_seq=0 2024-04-23T18:55:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd0137fe640 1 -- 172.21.15.33:0/1566737434 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdc8673390 con 0x55cdc871fb50 2024-04-23T18:55:04.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd0137fe640 10 client.18360 handle_command_reply: tid=0 2024-04-23T18:55:04.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 client.18360 shutdown 2024-04-23T18:55:04.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 msgr2=0x55cdc870f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:04.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005940 tx=0x7fd01c015320 comp rx=0 tx=0).stop 2024-04-23T18:55:04.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 shutdown_connections 2024-04-23T18:55:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc871fb50 0x55cdc8721f40 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:05.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc870cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700e40 0x55cdc870c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 --2- 172.21.15.33:0/1566737434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700520 0x55cdc870f4b0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:05.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 >> 172.21.15.33:0/1566737434 conn(0x55cdc86fcb30 msgr2=0x55cdc87029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd0137fe640 0 client.18360 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 shutdown_connections 2024-04-23T18:55:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 1 -- 172.21.15.33:0/1566737434 wait complete. 2024-04-23T18:55:05.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.282+0000 7fd03d180740 20 client.18360 trim_cache size 0 max 16384 2024-04-23T18:55:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 Processor -- start 2024-04-23T18:55:05.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 -- start start 2024-04-23T18:55:05.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc870c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc87277c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341aa430 con 0x55cdc8700e40 2024-04-23T18:55:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341acd10 con 0x55cdc870fc10 2024-04-23T18:55:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd03d180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341aab00 con 0x55cdc8700520 2024-04-23T18:55:05.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51154/0 (socket says 172.21.15.33:51154) 2024-04-23T18:55:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 -- 172.21.15.33:0/1011254874 learned_addr learned my addr 172.21.15.33:0/1011254874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:05.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc870c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc87277c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 -- 172.21.15.33:0/1011254874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 msgr2=0x55cdc870c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc870c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 -- 172.21.15.33:0/1011254874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 msgr2=0x55cdc87277c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:06.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc87277c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 -- 172.21.15.33:0/1011254874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc8701380 con 0x55cdc8700e40 2024-04-23T18:55:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd0337fe640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc870c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.298+0000 7fd033fff640 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd024005940 tx=0x7fd0240058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b090050abf9a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd029ffb640 1 -- 172.21.15.33:0/1011254874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024049070 con 0x55cdc8700e40 2024-04-23T18:55:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd029ffb640 1 -- 172.21.15.33:0/1011254874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0240054f0 con 0x55cdc8700e40 2024-04-23T18:55:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd029ffb640 1 -- 172.21.15.33:0/1011254874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024035040 con 0x55cdc8700e40 2024-04-23T18:55:06.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 -- 172.21.15.33:0/1011254874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 msgr2=0x55cdc870efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fd024005940 tx=0x7fd0240058b0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 -- 172.21.15.33:0/1011254874 shutdown_connections 2024-04-23T18:55:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc87277c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc870efe0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 --2- 172.21.15.33:0/1011254874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc870c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:06.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 -- 172.21.15.33:0/1011254874 >> 172.21.15.33:0/1011254874 conn(0x55cdc86fcb30 msgr2=0x55cdc8702900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 -- 172.21.15.33:0/1011254874 shutdown_connections 2024-04-23T18:55:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.299+0000 7fd03d180740 1 -- 172.21.15.33:0/1011254874 wait complete. 2024-04-23T18:55:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 -- start start 2024-04-23T18:55:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 Processor -- start 2024-04-23T18:55:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc8717c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:07.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8708b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:07.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341ab1d0 con 0x55cdc8700e40 2024-04-23T18:55:07.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341ac640 con 0x55cdc870fc10 2024-04-23T18:55:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0341ab8a0 con 0x55cdc8700520 2024-04-23T18:55:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd03d180740 10 client.0 _ll_register_callbacks cb 0x7fd01c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51158/0 (socket says 172.21.15.33:51158) 2024-04-23T18:55:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 -- 172.21.15.33:0/765440059 learned_addr learned my addr 172.21.15.33:0/765440059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:07.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd0337fe640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8708b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 -- 172.21.15.33:0/765440059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 msgr2=0x55cdc8717c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc8717c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 -- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 msgr2=0x55cdc8708b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8708b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.300+0000 7fd033fff640 1 -- 172.21.15.33:0/765440059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc8709180 con 0x55cdc8700e40 2024-04-23T18:55:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd033fff640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fd024046000 tx=0x7fd0240442e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f0167b95112fe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024049040 con 0x55cdc8700e40 2024-04-23T18:55:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fd0240037f0 con 0x55cdc8700e40 2024-04-23T18:55:08.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0240054f0 con 0x55cdc8700e40 2024-04-23T18:55:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fd024035040 con 0x55cdc8700e40 2024-04-23T18:55:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 10 client.18366 fetch_fsmap learned FSMap version 45 2024-04-23T18:55:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cdc8785980 con 0x55cdc8700e40 2024-04-23T18:55:08.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0240054f0 con 0x55cdc8700e40 2024-04-23T18:55:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fd024055030 con 0x55cdc8700e40 2024-04-23T18:55:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 10 client.18366 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:55:08.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 10 client.18366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 20 client.18366 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc8786070 0x55cdc8788490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 4 client.18366 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:55:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.301+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdc866d200 con 0x55cdc8786070 2024-04-23T18:55:08.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.302+0000 7fd0337fe640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc8786070 0x55cdc8788490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.302+0000 7fd0337fe640 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc8786070 0x55cdc8788490 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=102a08b6ae9c3a7c server_cookie=e6590661a4037b19 in_seq=0 out_seq=0 2024-04-23T18:55:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.302+0000 7fd0137fe640 10 client.18366 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:08.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd0137fe640 1 -- 172.21.15.33:0/765440059 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdc866d200 con 0x55cdc8786070 2024-04-23T18:55:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd0137fe640 10 client.18366 handle_command_reply: tid=0 2024-04-23T18:55:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd03d180740 1 client.18366 shutdown 2024-04-23T18:55:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 msgr2=0x55cdc8718150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:09.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fd024046000 tx=0x7fd0240442e0 comp rx=0 tx=0).stop 2024-04-23T18:55:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.306+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 shutdown_connections 2024-04-23T18:55:09.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55cdc8786070 0x55cdc8788490 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cdc870fc10 0x55cdc8708b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cdc8700e40 0x55cdc8718150 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd0137fe640 0 client.18366 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:09.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 --2- 172.21.15.33:0/765440059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdc8700520 0x55cdc8717c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 >> 172.21.15.33:0/765440059 conn(0x55cdc86fcb30 msgr2=0x55cdc86fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 shutdown_connections 2024-04-23T18:55:09.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 1 -- 172.21.15.33:0/765440059 wait complete. 2024-04-23T18:55:09.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03d180740 20 client.18366 trim_cache size 0 max 16384 2024-04-23T18:55:09.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:55:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:55:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:55:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:55:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 msgr2=0x7fd0200a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 0x7fd0200a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd02c099090 tx=0x7fd02c06e670 comp rx=0 tx=0).stop 2024-04-23T18:55:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 msgr2=0x7fd03419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00cce0 tx=0x7fd02c00a620 comp rx=0 tx=0).stop 2024-04-23T18:55:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 shutdown_connections 2024-04-23T18:55:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd0200a3dc0 0x7fd0200a6280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd03410c590 0x7fd03419cf00 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:10.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd034103ae0 0x7fd034198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:10.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 --2- 172.21.15.33:0/2168884693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd0341031c0 0x7fd034198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 >> 172.21.15.33:0/2168884693 conn(0x7fd034077fd0 msgr2=0x7fd0340fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 shutdown_connections 2024-04-23T18:55:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:53:48.307+0000 7fd03a38e640 1 -- 172.21.15.33:0/2168884693 wait complete. 2024-04-23T18:55:10.240 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:00 smithi106 ceph-mon[32716]: pgmap v923: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:10.249 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:02 smithi106 ceph-mon[32716]: pgmap v924: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:10.335 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:04 smithi106 ceph-mon[32716]: pgmap v925: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:10.532 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:06 smithi106 ceph-mon[32716]: pgmap v926: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:10.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:08 smithi106 ceph-mon[32716]: pgmap v927: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:55:10.691 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:09 smithi106 ceph-mon[32716]: osdmap e144: 12 total, 12 up, 12 in 2024-04-23T18:55:10.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:00 smithi122 ceph-mon[32546]: pgmap v923: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:10.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:02 smithi122 ceph-mon[32546]: pgmap v924: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:55:10.809 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:04 smithi122 ceph-mon[32546]: pgmap v925: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:10.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:06 smithi122 ceph-mon[32546]: pgmap v926: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:10.824 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:08 smithi122 ceph-mon[32546]: pgmap v927: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:55:10.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:09 smithi122 ceph-mon[32546]: osdmap e144: 12 total, 12 up, 12 in 2024-04-23T18:55:10.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:49 smithi033 ceph-mon[29274]: pgmap v883: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:55:10.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:51 smithi033 ceph-mon[29274]: pgmap v884: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T18:55:10.974 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:53 smithi033 ceph-mon[29274]: pgmap v885: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-23T18:55:10.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:55 smithi033 ceph-mon[29274]: pgmap v886: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:10.990 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:57 smithi033 ceph-mon[29274]: pgmap v887: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:10.998 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:53:59 smithi033 ceph-mon[29274]: pgmap v888: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:11.070 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:10 smithi106 ceph-mon[32716]: pgmap v929: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:55:11.137 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:10 smithi122 ceph-mon[32546]: pgmap v929: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:55:11.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 -- 172.21.15.33:0/701059905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c06ad00 msgr2=0x7fde2c06b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 --2- 172.21.15.33:0/701059905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c06ad00 0x7fde2c06b180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009860 tx=0x7fde1c02a340 comp rx=0 tx=0).stop 2024-04-23T18:55:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 -- 172.21.15.33:0/701059905 shutdown_connections 2024-04-23T18:55:11.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 --2- 172.21.15.33:0/701059905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c10e810 0x7fde2c110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 --2- 172.21.15.33:0/701059905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c06ad00 0x7fde2c06b180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 --2- 172.21.15.33:0/701059905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c102e90 0x7fde2c06a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:11.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.407+0000 7fde334cf640 1 -- 172.21.15.33:0/701059905 >> 172.21.15.33:0/701059905 conn(0x7fde2c076c50 msgr2=0x7fde2c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:11.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- 172.21.15.33:0/701059905 shutdown_connections 2024-04-23T18:55:11.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- 172.21.15.33:0/701059905 wait complete. 2024-04-23T18:55:11.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 Processor -- start 2024-04-23T18:55:11.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- start start 2024-04-23T18:55:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c102e90 0x7fde2c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:11.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 0x7fde2c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1143e0 con 0x7fde2c06ad00 2024-04-23T18:55:11.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c114260 con 0x7fde2c102e90 2024-04-23T18:55:11.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.408+0000 7fde334cf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c114560 con 0x7fde2c10e810 2024-04-23T18:55:12.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46586/0 (socket says 172.21.15.33:46586) 2024-04-23T18:55:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 -- 172.21.15.33:0/1374258675 learned_addr learned my addr 172.21.15.33:0/1374258675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 0x7fde2c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c102e90 0x7fde2c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 -- 172.21.15.33:0/1374258675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 msgr2=0x7fde2c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 0x7fde2c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:12.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 -- 172.21.15.33:0/1374258675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c102e90 msgr2=0x7fde2c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c102e90 0x7fde2c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 -- 172.21.15.33:0/1374258675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c19afe0 con 0x7fde2c06ad00 2024-04-23T18:55:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 0x7fde2c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:55:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde3126b640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fde2800e9c0 tx=0x7fde2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde227fc640 1 -- 172.21.15.33:0/1374258675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2800cde0 con 0x7fde2c06ad00 2024-04-23T18:55:12.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c19b2d0 con 0x7fde2c06ad00 2024-04-23T18:55:12.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde227fc640 1 -- 172.21.15.33:0/1374258675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde280035f0 con 0x7fde2c06ad00 2024-04-23T18:55:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde227fc640 1 -- 172.21.15.33:0/1374258675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde28010690 con 0x7fde2c06ad00 2024-04-23T18:55:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.409+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c1a75e0 con 0x7fde2c06ad00 2024-04-23T18:55:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.412+0000 7fde227fc640 1 -- 172.21.15.33:0/1374258675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde28004000 con 0x7fde2c06ad00 2024-04-23T18:55:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.412+0000 7fde361d8740 1 Processor -- start 2024-04-23T18:55:13.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.412+0000 7fde361d8740 1 -- start start 2024-04-23T18:55:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb5f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5581fb555970 con 0x5581fb5fb800 2024-04-23T18:55:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5581fb4f24a0 con 0x5581fb5f0f40 2024-04-23T18:55:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde361d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5581fb551930 con 0x5581fb5f0620 2024-04-23T18:55:13.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde227fc640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 0x7fde180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46588/0 (socket says 172.21.15.33:46588) 2024-04-23T18:55:13.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 -- 172.21.15.33:0/1397651817 learned_addr learned my addr 172.21.15.33:0/1397651817 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:13.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 0x7fde180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:13.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb5f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:13.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde227fc640 1 -- 172.21.15.33:0/1374258675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fde280d0b10 con 0x7fde2c06ad00 2024-04-23T18:55:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde3126b640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 -- 172.21.15.33:0/1397651817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 msgr2=0x5581fb5f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb5f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:14.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 -- 172.21.15.33:0/1397651817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 msgr2=0x5581fb5fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:14.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 -- 172.21.15.33:0/1397651817 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fb268210 con 0x5581fb5fb800 2024-04-23T18:55:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb5f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009860 tx=0x7fde1c005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6fa15681d64320 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 0x7fde180a6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde24009f40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:14.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.413+0000 7fde177fe640 1 -- 172.21.15.33:0/1397651817 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c037040 con 0x5581fb5fb800 2024-04-23T18:55:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde177fe640 1 -- 172.21.15.33:0/1397651817 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1c030450 con 0x5581fb5fb800 2024-04-23T18:55:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde177fe640 1 -- 172.21.15.33:0/1397651817 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c003970 con 0x5581fb5fb800 2024-04-23T18:55:14.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 -- 172.21.15.33:0/1397651817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 msgr2=0x5581fb5fdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:14.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009860 tx=0x7fde1c005b20 comp rx=0 tx=0).stop 2024-04-23T18:55:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 -- 172.21.15.33:0/1397651817 shutdown_connections 2024-04-23T18:55:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb5fdbf0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:14.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 --2- 172.21.15.33:0/1397651817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb5f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:14.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 -- 172.21.15.33:0/1397651817 >> 172.21.15.33:0/1397651817 conn(0x5581fb5ecb30 msgr2=0x5581fb5ecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:14.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 -- 172.21.15.33:0/1397651817 shutdown_connections 2024-04-23T18:55:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.414+0000 7fde361d8740 1 -- 172.21.15.33:0/1397651817 wait complete. 2024-04-23T18:55:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 -- start start 2024-04-23T18:55:14.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 Processor -- start 2024-04-23T18:55:14.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:14.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5f8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:14.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 0x5581fb5f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5581fb5fe3d0 con 0x5581fb5f0620 2024-04-23T18:55:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5581fb4f00b0 con 0x5581fb5f0f40 2024-04-23T18:55:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5581fb5fe550 con 0x5581fb5fb800 2024-04-23T18:55:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde361d8740 10 client.0 _ll_register_callbacks cb 0x5581fb4ddd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:15.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde3126b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:15.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde3126b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46604/0 (socket says 172.21.15.33:46604) 2024-04-23T18:55:15.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.415+0000 7fde3126b640 1 -- 172.21.15.33:0/357939968 learned_addr learned my addr 172.21.15.33:0/357939968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:15.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde31a6c640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 0x5581fb5f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:15.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde30a6a640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:15.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 -- 172.21.15.33:0/357939968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 msgr2=0x5581fb5f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 0x5581fb5f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 -- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 msgr2=0x5581fb5f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:15.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 -- 172.21.15.33:0/357939968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fb334620 con 0x5581fb5f0620 2024-04-23T18:55:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde30a6a640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde31a6c640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 0x5581fb5f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde3126b640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fde28098f00 tx=0x7fde2809d1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6190bb3d43cf3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde280a0e50 con 0x5581fb5f0620 2024-04-23T18:55:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fde28099110 con 0x5581fb5f0620 2024-04-23T18:55:15.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c030450 con 0x5581fb5f0620 2024-04-23T18:55:15.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fde280b6020 con 0x5581fb5f0620 2024-04-23T18:55:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde361d8740 10 client.18378 fetch_fsmap learned FSMap version 45 2024-04-23T18:55:15.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5581fb60a220 con 0x5581fb5f0620 2024-04-23T18:55:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.416+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1c030450 con 0x5581fb5f0620 2024-04-23T18:55:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fde280987f0 con 0x5581fb5f0620 2024-04-23T18:55:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 10 client.18378 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:55:16.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 10 client.18378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:16.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 20 client.18378 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb60b6e0 0x5581fb60dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 4 client.18378 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:55:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581fb563390 con 0x5581fb60b6e0 2024-04-23T18:55:16.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde30a6a640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb60b6e0 0x5581fb60dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde14ff9640 10 client.18378 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:16.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.417+0000 7fde30a6a640 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb60b6e0 0x5581fb60dad0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=263ceae3b9cefac8 server_cookie=f371f06bacfa4d6c in_seq=0 out_seq=0 2024-04-23T18:55:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde14ff9640 1 -- 172.21.15.33:0/357939968 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581fb563390 con 0x5581fb60b6e0 2024-04-23T18:55:16.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde14ff9640 10 client.18378 handle_command_reply: tid=0 2024-04-23T18:55:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 client.18378 shutdown 2024-04-23T18:55:16.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 msgr2=0x5581fb5fb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fde28098f00 tx=0x7fde2809d1a0 comp rx=0 tx=0).stop 2024-04-23T18:55:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 shutdown_connections 2024-04-23T18:55:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb60b6e0 0x5581fb60dad0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5fb800 0x5581fb5f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:16.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb5f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 --2- 172.21.15.33:0/357939968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0620 0x5581fb5fb120 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:17.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde14ff9640 0 client.18378 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.418+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 >> 172.21.15.33:0/357939968 conn(0x5581fb5ecb30 msgr2=0x5581fb5f2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.419+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 shutdown_connections 2024-04-23T18:55:17.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.419+0000 7fde361d8740 1 -- 172.21.15.33:0/357939968 wait complete. 2024-04-23T18:55:17.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.419+0000 7fde361d8740 20 client.18378 trim_cache size 0 max 16384 2024-04-23T18:55:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 Processor -- start 2024-04-23T18:55:17.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 -- start start 2024-04-23T18:55:17.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb607290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:17.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5fb800 0x5581fb6133b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1ae8d0 con 0x5581fb5f0f40 2024-04-23T18:55:17.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1b11b0 con 0x5581fb5fb800 2024-04-23T18:55:17.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde361d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1aefa0 con 0x5581fb5f0620 2024-04-23T18:55:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:17.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46606/0 (socket says 172.21.15.33:46606) 2024-04-23T18:55:17.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 -- 172.21.15.33:0/2868843250 learned_addr learned my addr 172.21.15.33:0/2868843250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 -- 172.21.15.33:0/2868843250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 msgr2=0x5581fb607290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb607290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:17.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 -- 172.21.15.33:0/2868843250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5fb800 msgr2=0x5581fb6133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5fb800 0x5581fb6133b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 -- 172.21.15.33:0/2868843250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581fb5f3fe0 con 0x5581fb5f0f40 2024-04-23T18:55:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde30a6a640 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fde1c02a860 tx=0x7fde1c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100ef10049a53dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:18.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.434+0000 7fde16ffd640 1 -- 172.21.15.33:0/2868843250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c052070 con 0x5581fb5f0f40 2024-04-23T18:55:18.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde16ffd640 1 -- 172.21.15.33:0/2868843250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1c04a070 con 0x5581fb5f0f40 2024-04-23T18:55:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde16ffd640 1 -- 172.21.15.33:0/2868843250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c037040 con 0x5581fb5f0f40 2024-04-23T18:55:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- 172.21.15.33:0/2868843250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 msgr2=0x5581fb5ebef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fde1c02a860 tx=0x7fde1c005760 comp rx=0 tx=0).stop 2024-04-23T18:55:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- 172.21.15.33:0/2868843250 shutdown_connections 2024-04-23T18:55:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5fb800 0x5581fb6133b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5f0f40 0x5581fb5ebef0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:18.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 --2- 172.21.15.33:0/2868843250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb607290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:18.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- 172.21.15.33:0/2868843250 >> 172.21.15.33:0/2868843250 conn(0x5581fb5ecb30 msgr2=0x5581fb5f8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:18.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- 172.21.15.33:0/2868843250 shutdown_connections 2024-04-23T18:55:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- 172.21.15.33:0/2868843250 wait complete. 2024-04-23T18:55:18.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 -- start start 2024-04-23T18:55:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.435+0000 7fde361d8740 1 Processor -- start 2024-04-23T18:55:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb603d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb6010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:19.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1af670 con 0x5581fb5fb800 2024-04-23T18:55:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1b0ae0 con 0x5581fb5f0f40 2024-04-23T18:55:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1afd40 con 0x5581fb5f0620 2024-04-23T18:55:19.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde361d8740 10 client.0 _ll_register_callbacks cb 0x7fde2809c400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:19.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46610/0 (socket says 172.21.15.33:46610) 2024-04-23T18:55:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 -- 172.21.15.33:0/1523992545 learned_addr learned my addr 172.21.15.33:0/1523992545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:19.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb6010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:19.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde3126b640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb603d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:19.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 -- 172.21.15.33:0/1523992545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 msgr2=0x5581fb603d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb603d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 -- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 msgr2=0x5581fb6010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb6010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:19.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde31a6c640 1 -- 172.21.15.33:0/1523992545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581fb670c30 con 0x5581fb5fb800 2024-04-23T18:55:19.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.436+0000 7fde30a6a640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb6010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde31a6c640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fde2400fe00 tx=0x7fde24011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cedb4afc069501 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240045d0 con 0x5581fb5fb800 2024-04-23T18:55:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 8542+0+0 (secure 0 0 0) 0x7fde24004770 con 0x5581fb5fb800 2024-04-23T18:55:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c04a070 con 0x5581fb5fb800 2024-04-23T18:55:19.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fde24029440 con 0x5581fb5fb800 2024-04-23T18:55:19.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 10 client.18384 fetch_fsmap learned FSMap version 45 2024-04-23T18:55:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5581fb671e40 con 0x5581fb5fb800 2024-04-23T18:55:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde1c04a070 con 0x5581fb5fb800 2024-04-23T18:55:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 45) ==== 7144+0+0 (secure 0 0 0) 0x7fde24027020 con 0x5581fb5fb800 2024-04-23T18:55:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 10 client.18384 fetch_fsmap finished waiting for FSMap version 45 2024-04-23T18:55:20.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 20 client.18384 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb6725f0 0x5581fb674a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 4 client.18384 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:55:20.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.437+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581fb55d200 con 0x5581fb6725f0 2024-04-23T18:55:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.438+0000 7fde3126b640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb6725f0 0x5581fb674a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.438+0000 7fde14ff9640 10 client.18384 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:01.438+0000 7fde3126b640 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb6725f0 0x5581fb674a10 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cc7d96b76093786 server_cookie=e1da85bb9b9d5481 in_seq=0 out_seq=0 2024-04-23T18:55:20.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:01 smithi033 ceph-mon[29274]: pgmap v889: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde14ff9640 1 -- 172.21.15.33:0/1523992545 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581fb55d200 con 0x5581fb6725f0 2024-04-23T18:55:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde14ff9640 10 client.18384 handle_command_reply: tid=0 2024-04-23T18:55:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 client.18384 shutdown 2024-04-23T18:55:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 msgr2=0x5581fb601790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fde2400fe00 tx=0x7fde24011690 comp rx=0 tx=0).stop 2024-04-23T18:55:20.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 shutdown_connections 2024-04-23T18:55:20.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5581fb6725f0 0x5581fb674a10 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5581fb5fb800 0x5581fb601790 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581fb5f0f40 0x5581fb6010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:20.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde14ff9640 0 client.18384 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 --2- 172.21.15.33:0/1523992545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581fb5f0620 0x5581fb603d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:21.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.199+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 >> 172.21.15.33:0/1523992545 conn(0x5581fb5ecb30 msgr2=0x5581fb606280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 shutdown_connections 2024-04-23T18:55:21.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde361d8740 1 -- 172.21.15.33:0/1523992545 wait complete. 2024-04-23T18:55:21.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde361d8740 20 client.18384 trim_cache size 0 max 16384 2024-04-23T18:55:21.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:55:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:55:21.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:55:21.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:55:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 msgr2=0x7fde180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 0x7fde180a6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde24009f40 comp rx=0 tx=0).stop 2024-04-23T18:55:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 msgr2=0x7fde2c1a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.200+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fde2800e9c0 tx=0x7fde2800ee80 comp rx=0 tx=0).stop 2024-04-23T18:55:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 shutdown_connections 2024-04-23T18:55:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fde180a3dc0 0x7fde180a6280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:21.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2c10e810 0x7fde2c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:21.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde2c102e90 0x7fde2c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 --2- 172.21.15.33:0/1374258675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde2c06ad00 0x7fde2c1a0660 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 >> 172.21.15.33:0/1374258675 conn(0x7fde2c076c50 msgr2=0x7fde2c10f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 shutdown_connections 2024-04-23T18:55:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:03.201+0000 7fde334cf640 1 -- 172.21.15.33:0/1374258675 wait complete. 2024-04-23T18:55:21.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-mon[29274]: pgmap v890: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-04-23T18:55:22.004 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:11 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:22.077 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:12 smithi122 ceph-mon[32546]: pgmap v930: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:55:22.113 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:12 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2871715025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:55:22.164 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:12 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:22.289 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:12 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:22.298 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:12 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:22.307 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:13 smithi122 ceph-mon[32546]: pgmap v931: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-04-23T18:55:22.367 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:13 smithi122 ceph-mon[32546]: pgmap v932: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:55:22.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:15 smithi122 ceph-mon[32546]: pgmap v933: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:55:22.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:17 smithi122 ceph-mon[32546]: pgmap v934: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:55:22.529 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:17 smithi122 ceph-mon[32546]: osdmap e145: 12 total, 12 up, 12 in 2024-04-23T18:55:22.573 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:19 smithi122 ceph-mon[32546]: pgmap v936: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-04-23T18:55:22.635 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:21 smithi122 ceph-mon[32546]: pgmap v937: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T18:55:22.695 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:11 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:22.705 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:12 smithi106 ceph-mon[32716]: pgmap v930: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:55:22.802 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:12 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2871715025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:55:22.815 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:12 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:22.894 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:12 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:22.978 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:12 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:23.020 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:13 smithi106 ceph-mon[32716]: pgmap v931: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-04-23T18:55:23.071 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:13 smithi106 ceph-mon[32716]: pgmap v932: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:55:23.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:15 smithi106 ceph-mon[32716]: pgmap v933: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:55:23.165 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:17 smithi106 ceph-mon[32716]: pgmap v934: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:55:23.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:17 smithi106 ceph-mon[32716]: osdmap e145: 12 total, 12 up, 12 in 2024-04-23T18:55:23.239 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:19 smithi106 ceph-mon[32716]: pgmap v936: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-04-23T18:55:23.295 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:21 smithi106 ceph-mon[32716]: pgmap v937: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T18:55:23.390 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:55:23.462 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: in thread 7f27979bb700 thread_name:md_log_replay 2024-04-23T18:55:23.492 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:55:23.534 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f27a8d7cd20] 2024-04-23T18:55:23.562 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55563454938f] 2024-04-23T18:55:23.571 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55563454cb23] 2024-04-23T18:55:23.655 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 4: (MDLog::_replay_thread()+0x617) [0x5556345a7647] 2024-04-23T18:55:23.661 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5556345b0d31] 2024-04-23T18:55:23.760 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f27a8d721ca] 2024-04-23T18:55:23.768 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 7: clone() 2024-04-23T18:55:23.802 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:54:03.228+0000 7f27979bb700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:55:23.873 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: in thread 7f27979bb700 thread_name:md_log_replay 2024-04-23T18:55:23.922 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:23.990 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:55:24.064 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f27a8d7cd20] 2024-04-23T18:55:24.073 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55563454938f] 2024-04-23T18:55:24.083 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55563454cb23] 2024-04-23T18:55:24.131 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 4: (MDLog::_replay_thread()+0x617) [0x5556345a7647] 2024-04-23T18:55:24.208 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5556345b0d31] 2024-04-23T18:55:24.218 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f27a8d721ca] 2024-04-23T18:55:24.275 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 7: clone() 2024-04-23T18:55:24.287 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:55:24.349 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:24.358 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 0> 2024-04-23T18:54:03.228+0000 7f27979bb700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:55:24.368 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: in thread 7f27979bb700 thread_name:md_log_replay 2024-04-23T18:55:24.446 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:24.486 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:55:24.541 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f27a8d7cd20] 2024-04-23T18:55:24.583 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55563454938f] 2024-04-23T18:55:24.698 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55563454cb23] 2024-04-23T18:55:24.807 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 4: (MDLog::_replay_thread()+0x617) [0x5556345a7647] 2024-04-23T18:55:24.820 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5556345b0d31] 2024-04-23T18:55:24.830 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f27a8d721ca] 2024-04-23T18:55:24.887 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 7: clone() 2024-04-23T18:55:24.950 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:55:25.030 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:25.042 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 0> 2024-04-23T18:54:03.228+0000 7f27979bb700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:55:25.112 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: in thread 7f27979bb700 thread_name:md_log_replay 2024-04-23T18:55:25.174 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:25.183 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:55:25.192 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f27a8d7cd20] 2024-04-23T18:55:25.204 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2: /usr/bin/ceph-mds(+0x54938f) [0x55563454938f] 2024-04-23T18:55:25.288 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55563454cb23] 2024-04-23T18:55:25.360 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 4: (MDLog::_replay_thread()+0x617) [0x5556345a7647] 2024-04-23T18:55:25.424 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5556345b0d31] 2024-04-23T18:55:25.573 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f27a8d721ca] 2024-04-23T18:55:25.650 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 7: clone() 2024-04-23T18:55:25.722 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:55:25.731 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[69527]: 2024-04-23T18:55:25.742 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:03 smithi033 podman[100507]: 2024-04-23 18:54:03.292220107 +0000 UTC m=+0.024009932 container died 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T18:55:25.751 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:04 smithi033 podman[100507]: 2024-04-23 18:54:04.021324525 +0000 UTC m=+0.753114377 container remove 66762eb185a3e683aa45b7cdda1560298a54e6c04be30c2bbca282173a85c57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-23T18:55:25.814 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:55:25.877 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-23T18:55:25.887 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.264s CPU time. 2024-04-23T18:55:25.940 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:23 smithi106 ceph-mon[32716]: pgmap v938: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:25.997 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:05 smithi033 ceph-mon[29274]: pgmap v891: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-23T18:55:26.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:05 smithi033 ceph-mon[29274]: osdmap e141: 12 total, 12 up, 12 in 2024-04-23T18:55:26.061 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:07 smithi033 ceph-mon[29274]: pgmap v893: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:55:26.070 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:07 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:55:26.151 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:09 smithi033 ceph-mon[29274]: pgmap v894: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:55:26.184 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: pgmap v895: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:55:26.243 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:26.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:26.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:26.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:26.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:55:26.489 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:55:26.500 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:55:26.511 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:11 smithi033 ceph-mon[29274]: osdmap e142: 12 total, 12 up, 12 in 2024-04-23T18:55:26.572 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:12 smithi033 ceph-mon[29274]: pgmap v896: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-23T18:55:26.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:12 smithi033 ceph-mon[29274]: pgmap v897: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 33 op/s 2024-04-23T18:55:26.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:14 smithi033 ceph-mon[29274]: pgmap v899: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:55:26.706 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 1. 2024-04-23T18:55:26.797 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 systemd[1]: Stopped Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:55:26.805 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Consumed 1.264s CPU time. 2024-04-23T18:55:26.905 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 systemd[1]: Starting Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:55:26.987 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 podman[100730]: 2024-04-23 18:54:14.642788825 +0000 UTC m=+0.019682058 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:55:27.082 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:14 smithi033 podman[100730]: 2024-04-23 18:54:14.747133114 +0000 UTC m=+0.124026364 container create ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T18:55:27.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:15 smithi033 ceph-mon[29274]: daemon mds.j restarted 2024-04-23T18:55:27.211 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 podman[100730]: 2024-04-23 18:54:15.0089574 +0000 UTC m=+0.385850634 container init ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416) 2024-04-23T18:55:27.306 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 podman[100730]: 2024-04-23 18:54:15.012344497 +0000 UTC m=+0.389237706 container start ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-23T18:55:27.354 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 ceph-mds[100749]: -- 172.21.15.33:0/1624844824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c6cddda40 con 0x556c6cdb1000 2024-04-23T18:55:27.367 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: starting mds.j at 2024-04-23T18:55:27.377 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 bash[100730]: ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba 2024-04-23T18:55:27.441 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:54:15 smithi033 systemd[1]: Started Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:55:27.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:25 smithi106 ceph-mon[32716]: pgmap v939: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:27.598 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:27 smithi106 ceph-mon[32716]: pgmap v940: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:27.683 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:23 smithi122 ceph-mon[32546]: pgmap v938: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:27.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:25 smithi122 ceph-mon[32546]: pgmap v939: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:27.795 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:27 smithi122 ceph-mon[32546]: pgmap v940: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:55:27.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 -- 172.21.15.33:0/3212223282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c106170 msgr2=0x7f817c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 --2- 172.21.15.33:0/3212223282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c106170 0x7f817c10dd40 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8174009ca0 tx=0x7f817402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:55:27.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 -- 172.21.15.33:0/3212223282 shutdown_connections 2024-04-23T18:55:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 --2- 172.21.15.33:0/3212223282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c106170 0x7f817c10dd40 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 --2- 172.21.15.33:0/3212223282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c101aa0 0x7f817c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 --2- 172.21.15.33:0/3212223282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 -- 172.21.15.33:0/3212223282 >> 172.21.15.33:0/3212223282 conn(0x7f817c077fd0 msgr2=0x7f817c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.294+0000 7f8181d68640 1 -- 172.21.15.33:0/3212223282 shutdown_connections 2024-04-23T18:55:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.295+0000 7f8181d68640 1 -- 172.21.15.33:0/3212223282 wait complete. 2024-04-23T18:55:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.295+0000 7f8181d68640 1 Processor -- start 2024-04-23T18:55:28.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.295+0000 7f8181d68640 1 -- start start 2024-04-23T18:55:28.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c106170 0x7f817c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:28.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:28.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c111110 con 0x7f817c19bdc0 2024-04-23T18:55:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c110f90 con 0x7f817c106170 2024-04-23T18:55:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f8181d68640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f817c111290 con 0x7f817c1010f0 2024-04-23T18:55:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35462/0 (socket says 172.21.15.33:35462) 2024-04-23T18:55:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 -- 172.21.15.33:0/1796707080 learned_addr learned my addr 172.21.15.33:0/1796707080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817affd640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c106170 0x7f817c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:28.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817b7fe640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:28.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 -- 172.21.15.33:0/1796707080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 msgr2=0x7f817c197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 -- 172.21.15.33:0/1796707080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c106170 msgr2=0x7f817c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c106170 0x7f817c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817bfff640 1 -- 172.21.15.33:0/1796707080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817c19c8b0 con 0x7f817c19bdc0 2024-04-23T18:55:28.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.296+0000 7f817b7fe640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f817bfff640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8174009430 tx=0x7f81740319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f8178ff9640 1 -- 172.21.15.33:0/1796707080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174003c30 con 0x7f817c19bdc0 2024-04-23T18:55:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817c19cb40 con 0x7f817c19bdc0 2024-04-23T18:55:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f8178ff9640 1 -- 172.21.15.33:0/1796707080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8174003dd0 con 0x7f817c19bdc0 2024-04-23T18:55:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f8178ff9640 1 -- 172.21.15.33:0/1796707080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817402faa0 con 0x7f817c19bdc0 2024-04-23T18:55:29.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.297+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817c1a4360 con 0x7f817c19bdc0 2024-04-23T18:55:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.299+0000 7f8178ff9640 1 -- 172.21.15.33:0/1796707080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f817402fc40 con 0x7f817c19bdc0 2024-04-23T18:55:29.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.299+0000 7f8184a84740 1 Processor -- start 2024-04-23T18:55:29.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.299+0000 7f8184a84740 1 -- start start 2024-04-23T18:55:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829de9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:29.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:29.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e829d4e970 con 0x55e829de9f40 2024-04-23T18:55:29.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e829ceb4a0 con 0x55e829df4800 2024-04-23T18:55:29.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8184a84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e829d4a930 con 0x55e829de9620 2024-04-23T18:55:29.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:29.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35474/0 (socket says 172.21.15.33:35474) 2024-04-23T18:55:29.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 -- 172.21.15.33:0/4136453763 learned_addr learned my addr 172.21.15.33:0/4136453763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:29.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817affd640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8178ff9640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 0x7f81640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817affd640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829de9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 -- 172.21.15.33:0/4136453763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 msgr2=0x55e829de9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829de9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 -- 172.21.15.33:0/4136453763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 msgr2=0x55e829df6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 -- 172.21.15.33:0/4136453763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e829a61210 con 0x55e829de9f40 2024-04-23T18:55:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817affd640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:30.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817bfff640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 0x7f81640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f8178ff9640 1 -- 172.21.15.33:0/1796707080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f81740f1cd0 con 0x7f817c19bdc0 2024-04-23T18:55:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.300+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f816c008c60 tx=0x7f816c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b7d3432e42171c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:30.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8161ffb640 1 -- 172.21.15.33:0/4136453763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c014070 con 0x55e829de9f40 2024-04-23T18:55:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f817bfff640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 0x7f81640a6280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81740ba700 tx=0x7f817408d610 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:30.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8161ffb640 1 -- 172.21.15.33:0/4136453763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c0047a0 con 0x55e829de9f40 2024-04-23T18:55:30.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8161ffb640 1 -- 172.21.15.33:0/4136453763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c010690 con 0x55e829de9f40 2024-04-23T18:55:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 -- 172.21.15.33:0/4136453763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 msgr2=0x55e829df42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f816c008c60 tx=0x7f816c00e720 comp rx=0 tx=0).stop 2024-04-23T18:55:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 -- 172.21.15.33:0/4136453763 shutdown_connections 2024-04-23T18:55:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:30.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9f40 0x55e829df42c0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:30.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 --2- 172.21.15.33:0/4136453763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829de9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 -- 172.21.15.33:0/4136453763 >> 172.21.15.33:0/4136453763 conn(0x55e829de5b30 msgr2=0x55e829de5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 -- 172.21.15.33:0/4136453763 shutdown_connections 2024-04-23T18:55:30.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.301+0000 7f8184a84740 1 -- 172.21.15.33:0/4136453763 wait complete. 2024-04-23T18:55:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 -- start start 2024-04-23T18:55:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 Processor -- start 2024-04-23T18:55:31.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:31.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9f40 0x55e829df3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:31.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:31.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e829df7430 con 0x55e829de9620 2024-04-23T18:55:31.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e829ce90b0 con 0x55e829df4800 2024-04-23T18:55:31.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e829df75b0 con 0x55e829de9f40 2024-04-23T18:55:31.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f8184a84740 10 client.0 _ll_register_callbacks cb 0x55e829cd6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:31.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35490/0 (socket says 172.21.15.33:35490) 2024-04-23T18:55:31.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f817affd640 1 -- 172.21.15.33:0/2152622116 learned_addr learned my addr 172.21.15.33:0/2152622116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:31.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.302+0000 7f817affd640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9f40 0x55e829df3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 -- 172.21.15.33:0/2152622116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9f40 msgr2=0x55e829df3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9f40 0x55e829df3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 -- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 msgr2=0x55e829df3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 -- 172.21.15.33:0/2152622116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e829b2d620 con 0x55e829de9620 2024-04-23T18:55:31.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f817affd640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8170005fd0 tx=0x7f81700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d63a8aa5e205d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:31.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170004220 con 0x55e829de9620 2024-04-23T18:55:31.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f81700043c0 con 0x55e829de9620 2024-04-23T18:55:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816c0047a0 con 0x55e829de9620 2024-04-23T18:55:31.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8170005020 con 0x55e829de9620 2024-04-23T18:55:31.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f8184a84740 10 client.18402 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:32.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e829e07870 con 0x55e829de9620 2024-04-23T18:55:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f816c0047a0 con 0x55e829de9620 2024-04-23T18:55:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.303+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f817001e030 con 0x55e829de9620 2024-04-23T18:55:32.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 10 client.18402 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:32.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 10 client.18402 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 20 client.18402 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e08ce0 0x55e829e0b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:32.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 4 client.18402 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:55:32.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e829d5c390 con 0x55e829e08ce0 2024-04-23T18:55:32.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e08ce0 0x55e829e0b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f817b7fe640 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e08ce0 0x55e829e0b0f0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1d1cd0b22c1383 server_cookie=abb8084dcf74cc3b in_seq=0 out_seq=0 2024-04-23T18:55:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.304+0000 7f815affd640 10 client.18402 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f815affd640 1 -- 172.21.15.33:0/2152622116 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e829d5c390 con 0x55e829e08ce0 2024-04-23T18:55:32.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f815affd640 10 client.18402 handle_command_reply: tid=0 2024-04-23T18:55:32.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 client.18402 shutdown 2024-04-23T18:55:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 msgr2=0x55e829df2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8170005fd0 tx=0x7f81700046c0 comp rx=0 tx=0).stop 2024-04-23T18:55:32.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 shutdown_connections 2024-04-23T18:55:32.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e08ce0 0x55e829e0b0f0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829df4800 0x55e829df3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9f40 0x55e829df3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:32.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f815affd640 0 client.18402 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:32.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 --2- 172.21.15.33:0/2152622116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829de9620 0x55e829df2ea0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 >> 172.21.15.33:0/2152622116 conn(0x55e829de5b30 msgr2=0x55e829deb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 shutdown_connections 2024-04-23T18:55:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 1 -- 172.21.15.33:0/2152622116 wait complete. 2024-04-23T18:55:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.305+0000 7f8184a84740 20 client.18402 trim_cache size 0 max 16384 2024-04-23T18:55:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 Processor -- start 2024-04-23T18:55:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 -- start start 2024-04-23T18:55:33.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829e10a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:33.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829e10f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1aa2c0 con 0x55e829df4800 2024-04-23T18:55:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1acba0 con 0x55e829de9f40 2024-04-23T18:55:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f8184a84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1aa990 con 0x55e829de9620 2024-04-23T18:55:33.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:33.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35506/0 (socket says 172.21.15.33:35506) 2024-04-23T18:55:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.320+0000 7f817affd640 1 -- 172.21.15.33:0/4178701849 learned_addr learned my addr 172.21.15.33:0/4178701849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829e10f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829e10a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:33.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 -- 172.21.15.33:0/4178701849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 msgr2=0x55e829e10a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:33.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829e10a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 -- 172.21.15.33:0/4178701849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 msgr2=0x55e829e10f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:34.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829e10f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 -- 172.21.15.33:0/4178701849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e829df1230 con 0x55e829df4800 2024-04-23T18:55:34.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f817affd640 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8170001eb0 tx=0x7f817000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e895329b942001f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:34.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f81617fa640 1 -- 172.21.15.33:0/4178701849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817002e020 con 0x55e829df4800 2024-04-23T18:55:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f81617fa640 1 -- 172.21.15.33:0/4178701849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81700120d0 con 0x55e829df4800 2024-04-23T18:55:34.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.321+0000 7f81617fa640 1 -- 172.21.15.33:0/4178701849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8170004960 con 0x55e829df4800 2024-04-23T18:55:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- 172.21.15.33:0/4178701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 msgr2=0x55e829e00c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8170001eb0 tx=0x7f817000b410 comp rx=0 tx=0).stop 2024-04-23T18:55:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- 172.21.15.33:0/4178701849 shutdown_connections 2024-04-23T18:55:34.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e00c80 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829e10f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 --2- 172.21.15.33:0/4178701849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829e10a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- 172.21.15.33:0/4178701849 >> 172.21.15.33:0/4178701849 conn(0x55e829de5b30 msgr2=0x55e829df2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- 172.21.15.33:0/4178701849 shutdown_connections 2024-04-23T18:55:34.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- 172.21.15.33:0/4178701849 wait complete. 2024-04-23T18:55:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 -- start start 2024-04-23T18:55:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.322+0000 7f8184a84740 1 Processor -- start 2024-04-23T18:55:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829dfde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:35.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829dfe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:35.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1ab060 con 0x55e829df4800 2024-04-23T18:55:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1ac4d0 con 0x55e829de9f40 2024-04-23T18:55:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f817c1ab730 con 0x55e829de9620 2024-04-23T18:55:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 10 client.0 _ll_register_callbacks cb 0x7f8170012550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:35.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35518/0 (socket says 172.21.15.33:35518) 2024-04-23T18:55:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 -- 172.21.15.33:0/4135588598 learned_addr learned my addr 172.21.15.33:0/4135588598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829dfe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:35.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829dfde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 -- 172.21.15.33:0/4135588598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 msgr2=0x55e829dfde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829dfde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 -- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 msgr2=0x55e829dfe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829dfe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 -- 172.21.15.33:0/4135588598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e829e6d800 con 0x55e829df4800 2024-04-23T18:55:36.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f817affd640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8170012700 tx=0x7f8170005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75867eb1cbe03c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:36.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f815affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:36.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817002e030 con 0x55e829df4800 2024-04-23T18:55:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f8170005440 con 0x55e829df4800 2024-04-23T18:55:36.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.323+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81700120d0 con 0x55e829df4800 2024-04-23T18:55:36.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8170005640 con 0x55e829df4800 2024-04-23T18:55:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 10 client.18408 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55e829e6eb10 con 0x55e829df4800 2024-04-23T18:55:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81700120d0 con 0x55e829df4800 2024-04-23T18:55:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f817001e040 con 0x55e829df4800 2024-04-23T18:55:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 10 client.18408 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:36.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 10 client.18408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:36.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 20 client.18408 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:36.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e6f110 0x55e829e71500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:36.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 4 client.18408 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:55:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e829d56200 con 0x55e829e6f110 2024-04-23T18:55:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.324+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e6f110 0x55e829e71500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.325+0000 7f815affd640 10 client.18408 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:36.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.325+0000 7f817b7fe640 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e6f110 0x55e829e71500 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3885ce1a4f25e070 server_cookie=928fcef5747bd558 in_seq=0 out_seq=0 2024-04-23T18:55:36.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f815affd640 1 -- 172.21.15.33:0/4135588598 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e829d56200 con 0x55e829e6f110 2024-04-23T18:55:36.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f815affd640 10 client.18408 handle_command_reply: tid=0 2024-04-23T18:55:36.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 client.18408 shutdown 2024-04-23T18:55:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 msgr2=0x55e829e6d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:36.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8170012700 tx=0x7f8170005300 comp rx=0 tx=0).stop 2024-04-23T18:55:36.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 shutdown_connections 2024-04-23T18:55:36.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55e829e6f110 0x55e829e71500 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e829df4800 0x55e829e6d270 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e829de9f40 0x55e829dfe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f815affd640 0 client.18408 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 --2- 172.21.15.33:0/4135588598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e829de9620 0x55e829dfde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 >> 172.21.15.33:0/4135588598 conn(0x55e829de5b30 msgr2=0x55e829defcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.331+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 shutdown_connections 2024-04-23T18:55:37.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8184a84740 1 -- 172.21.15.33:0/4135588598 wait complete. 2024-04-23T18:55:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8184a84740 20 client.18408 trim_cache size 0 max 16384 2024-04-23T18:55:37.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:55:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:55:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:55:37.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:55:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 msgr2=0x7f81640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:37.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 0x7f81640a6280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f81740ba700 tx=0x7f817408d610 comp rx=0 tx=0).stop 2024-04-23T18:55:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 msgr2=0x7f817c19c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8174009430 tx=0x7f81740319f0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 shutdown_connections 2024-04-23T18:55:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f81640a3dc0 0x7f81640a6280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f817c19bdc0 0x7f817c19c270 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f817c106170 0x7f817c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:37.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 --2- 172.21.15.33:0/1796707080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f817c1010f0 0x7f817c197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:38.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 >> 172.21.15.33:0/1796707080 conn(0x7f817c077fd0 msgr2=0x7f817c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 shutdown_connections 2024-04-23T18:55:38.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:16.332+0000 7f8181d68640 1 -- 172.21.15.33:0/1796707080 wait complete. 2024-04-23T18:55:38.120 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: pgmap v900: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 155 B/s wr, 6 op/s 2024-04-23T18:55:38.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: osdmap e143: 12 total, 12 up, 12 in 2024-04-23T18:55:38.266 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6834/2623838743,v1:172.21.15.33:6835/2623838743] up:boot 2024-04-23T18:55:38.275 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:55:38.364 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:55:38.373 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:55:38.381 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:18 smithi033 ceph-mon[29274]: pgmap v902: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 170 B/s wr, 15 op/s 2024-04-23T18:55:38.390 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:20 smithi033 ceph-mon[29274]: pgmap v903: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 127 B/s wr, 11 op/s 2024-04-23T18:55:38.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:22 smithi033 ceph-mon[29274]: pgmap v904: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-23T18:55:38.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:24 smithi033 ceph-mon[29274]: pgmap v905: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:55:38.506 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:26 smithi033 ceph-mon[29274]: pgmap v906: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:55:38.514 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:28 smithi033 ceph-mon[29274]: pgmap v907: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 356 B/s wr, 13 op/s 2024-04-23T18:55:38.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:29 smithi122 ceph-mon[32546]: pgmap v941: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 268 B/s wr, 9 op/s 2024-04-23T18:55:38.663 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:31 smithi122 ceph-mon[32546]: pgmap v942: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:38.715 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:33 smithi122 ceph-mon[32546]: pgmap v943: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:38.792 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:35 smithi122 ceph-mon[32546]: pgmap v944: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:38.828 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:37 smithi122 ceph-mon[32546]: pgmap v945: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:38.910 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:29 smithi106 ceph-mon[32716]: pgmap v941: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 268 B/s wr, 9 op/s 2024-04-23T18:55:38.923 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:31 smithi106 ceph-mon[32716]: pgmap v942: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:38.984 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:33 smithi106 ceph-mon[32716]: pgmap v943: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:39.070 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:35 smithi106 ceph-mon[32716]: pgmap v944: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:39.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:37 smithi106 ceph-mon[32716]: pgmap v945: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:39.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 -- 172.21.15.33:0/2575164393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 msgr2=0x7f04b010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:39.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 --2- 172.21.15.33:0/2575164393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b010ee60 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009ca0 tx=0x7f04a802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:55:39.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 -- 172.21.15.33:0/2575164393 shutdown_connections 2024-04-23T18:55:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 --2- 172.21.15.33:0/2575164393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b010ee60 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 --2- 172.21.15.33:0/2575164393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b0102b70 0x7f04b0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 --2- 172.21.15.33:0/2575164393 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b01021c0 0x7f04b01025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.432+0000 7f04b7e07640 1 -- 172.21.15.33:0/2575164393 >> 172.21.15.33:0/2575164393 conn(0x7f04b00fda20 msgr2=0x7f04b00ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- 172.21.15.33:0/2575164393 shutdown_connections 2024-04-23T18:55:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- 172.21.15.33:0/2575164393 wait complete. 2024-04-23T18:55:39.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 Processor -- start 2024-04-23T18:55:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- start start 2024-04-23T18:55:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 0x7f04b0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 0x7f04b019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0112230 con 0x7f04b0107290 2024-04-23T18:55:39.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01120b0 con 0x7f04b019cf10 2024-04-23T18:55:39.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.433+0000 7f04b7e07640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01123b0 con 0x7f04b01021c0 2024-04-23T18:55:39.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:39.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 0x7f04b0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:39.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38010/0 (socket says 172.21.15.33:38010) 2024-04-23T18:55:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 -- 172.21.15.33:0/894988030 learned_addr learned my addr 172.21.15.33:0/894988030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:40.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 0x7f04b0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:56316/0 (socket says 172.21.15.33:56316) 2024-04-23T18:55:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b63a4640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 0x7f04b019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 -- 172.21.15.33:0/894988030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 msgr2=0x7f04b0198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 0x7f04b0198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 -- 172.21.15.33:0/894988030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 msgr2=0x7f04b019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 0x7f04b019d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 -- 172.21.15.33:0/894988030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b019da00 con 0x7f04b0107290 2024-04-23T18:55:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b63a4640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 0x7f04b019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b53a2640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f04a000d770 tx=0x7f04a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04a6ffd640 1 -- 172.21.15.33:0/894988030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0017070 con 0x7f04b0107290 2024-04-23T18:55:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04a6ffd640 1 -- 172.21.15.33:0/894988030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a0009c30 con 0x7f04b0107290 2024-04-23T18:55:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b019dcf0 con 0x7f04b0107290 2024-04-23T18:55:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04a6ffd640 1 -- 172.21.15.33:0/894988030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000f620 con 0x7f04b0107290 2024-04-23T18:55:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.434+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01a54e0 con 0x7f04b0107290 2024-04-23T18:55:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04a6ffd640 1 -- 172.21.15.33:0/894988030 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f04a001a070 con 0x7f04b0107290 2024-04-23T18:55:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 Processor -- start 2024-04-23T18:55:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 -- start start 2024-04-23T18:55:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2bd620 0x55c0bc2bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:40.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc222970 con 0x55c0bc2bdf40 2024-04-23T18:55:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc1bf4a0 con 0x55c0bc2bd620 2024-04-23T18:55:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04babf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc21e930 con 0x55c0bc2c8800 2024-04-23T18:55:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:41.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38018/0 (socket says 172.21.15.33:38018) 2024-04-23T18:55:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2714648253 learned_addr learned my addr 172.21.15.33:0/2714648253 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04a6ffd640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 0x7f049c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.438+0000 7f04b53a2640 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:41.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b63a4640 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2bd620 0x55c0bc2bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:41.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b63a4640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 0x7f049c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2714648253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2c8800 msgr2=0x55c0bc2cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:41.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:41.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2714648253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2bd620 msgr2=0x55c0bc2bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2bd620 0x55c0bc2bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:41.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2714648253 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bbf35210 con 0x55c0bc2bdf40 2024-04-23T18:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04a6ffd640 1 -- 172.21.15.33:0/894988030 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f04a00d02d0 con 0x7f04b0107290 2024-04-23T18:55:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00bcd0 tx=0x7f04ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18904bbaf0ab81e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f049bfff640 1 -- 172.21.15.33:0/2714648253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac005220 con 0x55c0bc2bdf40 2024-04-23T18:55:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f04b63a4640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 0x7f049c0a6280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009a60 tx=0x7f04a80095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f049bfff640 1 -- 172.21.15.33:0/2714648253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac004c50 con 0x55c0bc2bdf40 2024-04-23T18:55:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.439+0000 7f049bfff640 1 -- 172.21.15.33:0/2714648253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac011730 con 0x55c0bc2bdf40 2024-04-23T18:55:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- 172.21.15.33:0/2714648253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 msgr2=0x55c0bc2c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00bcd0 tx=0x7f04ac0107a0 comp rx=0 tx=0).stop 2024-04-23T18:55:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- 172.21.15.33:0/2714648253 shutdown_connections 2024-04-23T18:55:42.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:42.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c82c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 --2- 172.21.15.33:0/2714648253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2bd620 0x55c0bc2bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:42.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- 172.21.15.33:0/2714648253 >> 172.21.15.33:0/2714648253 conn(0x55c0bc2b9b30 msgr2=0x55c0bc2b9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- 172.21.15.33:0/2714648253 shutdown_connections 2024-04-23T18:55:42.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- 172.21.15.33:0/2714648253 wait complete. 2024-04-23T18:55:42.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.440+0000 7f04babf1740 1 -- start start 2024-04-23T18:55:42.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 Processor -- start 2024-04-23T18:55:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bd620 0x55c0bc2c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:42.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc2cb430 con 0x55c0bc2bdf40 2024-04-23T18:55:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc1bd0b0 con 0x55c0bc2c8800 2024-04-23T18:55:42.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c0bc2cb5b0 con 0x55c0bc2bd620 2024-04-23T18:55:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04babf1740 10 client.0 _ll_register_callbacks cb 0x55c0bc1aad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:42.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04b53a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04b53a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38022/0 (socket says 172.21.15.33:38022) 2024-04-23T18:55:42.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04b53a2640 1 -- 172.21.15.33:0/3360837293 learned_addr learned my addr 172.21.15.33:0/3360837293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:42.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bd620 0x55c0bc2c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.441+0000 7f04b63a4640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 -- 172.21.15.33:0/3360837293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bd620 msgr2=0x55c0bc2c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:42.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bd620 0x55c0bc2c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:42.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 -- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 msgr2=0x55c0bc2c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 -- 172.21.15.33:0/3360837293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0bc001620 con 0x55c0bc2bdf40 2024-04-23T18:55:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b63a4640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04b53a2640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f04a0098c50 tx=0x7f04a0010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f275f3547429ad server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:43.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0021040 con 0x55c0bc2bdf40 2024-04-23T18:55:43.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f04a00a4a60 con 0x55c0bc2bdf40 2024-04-23T18:55:43.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac004c50 con 0x55c0bc2bdf40 2024-04-23T18:55:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04a0009de0 con 0x55c0bc2bdf40 2024-04-23T18:55:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 10 client.18420 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:43.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0bc2d7350 con 0x55c0bc2bdf40 2024-04-23T18:55:43.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac004c50 con 0x55c0bc2bdf40 2024-04-23T18:55:43.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f04a0097fd0 con 0x55c0bc2bdf40 2024-04-23T18:55:43.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 10 client.18420 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:43.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 10 client.18420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:43.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.442+0000 7f04babf1740 20 client.18420 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:43.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc2d8920 0x55c0bc2dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:43.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04babf1740 4 client.18420 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:55:43.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0bc230390 con 0x55c0bc2d8920 2024-04-23T18:55:44.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc2d8920 0x55c0bc2dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc2d8920 0x55c0bc2dad30 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797bd4fbfbca49b3 server_cookie=9b96c92322863d4 in_seq=0 out_seq=0 2024-04-23T18:55:44.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.443+0000 7f04997fa640 10 client.18420 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:44.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04997fa640 1 -- 172.21.15.33:0/3360837293 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0bc230390 con 0x55c0bc2d8920 2024-04-23T18:55:44.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04997fa640 10 client.18420 handle_command_reply: tid=0 2024-04-23T18:55:44.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 client.18420 shutdown 2024-04-23T18:55:44.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 msgr2=0x55c0bc2c53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f04a0098c50 tx=0x7f04a0010c00 comp rx=0 tx=0).stop 2024-04-23T18:55:44.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 shutdown_connections 2024-04-23T18:55:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc2d8920 0x55c0bc2dad30 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2c53e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 --2- 172.21.15.33:0/3360837293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bd620 0x55c0bc2c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:44.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04997fa640 0 client.18420 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:44.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 >> 172.21.15.33:0/3360837293 conn(0x55c0bc2b9b30 msgr2=0x55c0bc2bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:44.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 shutdown_connections 2024-04-23T18:55:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 1 -- 172.21.15.33:0/3360837293 wait complete. 2024-04-23T18:55:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.444+0000 7f04babf1740 20 client.18420 trim_cache size 0 max 16384 2024-04-23T18:55:44.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.459+0000 7f04babf1740 1 Processor -- start 2024-04-23T18:55:44.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.459+0000 7f04babf1740 1 -- start start 2024-04-23T18:55:44.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:44.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2e0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:45.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01ac570 con 0x55c0bc2bd620 2024-04-23T18:55:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01aee50 con 0x55c0bc2c8800 2024-04-23T18:55:45.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04babf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01acc40 con 0x55c0bc2bdf40 2024-04-23T18:55:45.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38034/0 (socket says 172.21.15.33:38034) 2024-04-23T18:55:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2386149936 learned_addr learned my addr 172.21.15.33:0/2386149936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b53a2640 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2e0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:45.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2386149936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 msgr2=0x55c0bc2e0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:45.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2e0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:45.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2386149936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 msgr2=0x55c0bc2e0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:45.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 -- 172.21.15.33:0/2386149936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0bc2bdd70 con 0x55c0bc2bd620 2024-04-23T18:55:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b53a2640 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005640 tx=0x7f04ac020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aafb7dc01e20ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f049b7fe640 1 -- 172.21.15.33:0/2386149936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac004220 con 0x55c0bc2bd620 2024-04-23T18:55:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f049b7fe640 1 -- 172.21.15.33:0/2386149936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac0043c0 con 0x55c0bc2bd620 2024-04-23T18:55:46.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.460+0000 7f049b7fe640 1 -- 172.21.15.33:0/2386149936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0292d0 con 0x55c0bc2bd620 2024-04-23T18:55:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- 172.21.15.33:0/2386149936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 msgr2=0x55c0bc2d42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f04ac005640 tx=0x7f04ac020040 comp rx=0 tx=0).stop 2024-04-23T18:55:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- 172.21.15.33:0/2386149936 shutdown_connections 2024-04-23T18:55:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:46.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2e0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 --2- 172.21.15.33:0/2386149936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d42f0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- 172.21.15.33:0/2386149936 >> 172.21.15.33:0/2386149936 conn(0x55c0bc2b9b30 msgr2=0x55c0bc2b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- 172.21.15.33:0/2386149936 shutdown_connections 2024-04-23T18:55:46.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- 172.21.15.33:0/2386149936 wait complete. 2024-04-23T18:55:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 -- start start 2024-04-23T18:55:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.461+0000 7f04babf1740 1 Processor -- start 2024-04-23T18:55:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:46.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:46.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01ad310 con 0x55c0bc2bd620 2024-04-23T18:55:46.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01ae780 con 0x55c0bc2c8800 2024-04-23T18:55:46.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04b01ad9e0 con 0x55c0bc2bdf40 2024-04-23T18:55:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04babf1740 10 client.0 _ll_register_callbacks cb 0x7f04a009bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38038/0 (socket says 172.21.15.33:38038) 2024-04-23T18:55:47.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 -- 172.21.15.33:0/1328140047 learned_addr learned my addr 172.21.15.33:0/1328140047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b63a4640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b53a2640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 -- 172.21.15.33:0/1328140047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 msgr2=0x55c0bc2d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:47.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 -- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 msgr2=0x55c0bc2cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:47.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 -- 172.21.15.33:0/1328140047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0bc2ce7b0 con 0x55c0bc2bd620 2024-04-23T18:55:47.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04b5ba3640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00bbf0 tx=0x7f04ac010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818f0009a05eb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.462+0000 7f04997fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:47.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac0035f0 con 0x55c0bc2bd620 2024-04-23T18:55:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f04ac003790 con 0x55c0bc2bd620 2024-04-23T18:55:47.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04ac0043c0 con 0x55c0bc2bd620 2024-04-23T18:55:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f04ac003990 con 0x55c0bc2bd620 2024-04-23T18:55:47.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 10 client.18426 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0bc33ea70 con 0x55c0bc2bd620 2024-04-23T18:55:47.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04ac0043c0 con 0x55c0bc2bd620 2024-04-23T18:55:47.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f04ac011440 con 0x55c0bc2bd620 2024-04-23T18:55:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 10 client.18426 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 10 client.18426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 20 client.18426 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc33f190 0x55c0bc3415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 4 client.18426 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:55:48.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0bc2c5cb0 con 0x55c0bc33f190 2024-04-23T18:55:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.463+0000 7f04b53a2640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc33f190 0x55c0bc3415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:48.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.464+0000 7f04b53a2640 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc33f190 0x55c0bc3415b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ac1f3c19c075fd server_cookie=6731fbd3e851f961 in_seq=0 out_seq=0 2024-04-23T18:55:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.464+0000 7f04997fa640 10 client.18426 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04997fa640 1 -- 172.21.15.33:0/1328140047 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0bc2c5cb0 con 0x55c0bc33f190 2024-04-23T18:55:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04997fa640 10 client.18426 handle_command_reply: tid=0 2024-04-23T18:55:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 client.18426 shutdown 2024-04-23T18:55:48.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 msgr2=0x55c0bc2d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00bbf0 tx=0x7f04ac010fd0 comp rx=0 tx=0).stop 2024-04-23T18:55:48.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 shutdown_connections 2024-04-23T18:55:48.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c0bc33f190 0x55c0bc3415b0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:48.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0bc2c8800 0x55c0bc2cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:48.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c0bc2bdf40 0x55c0bc2d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:48.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 --2- 172.21.15.33:0/1328140047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c0bc2bd620 0x55c0bc2d0830 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:48.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04997fa640 0 client.18426 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:48.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 >> 172.21.15.33:0/1328140047 conn(0x55c0bc2b9b30 msgr2=0x55c0bc2b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:48.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 shutdown_connections 2024-04-23T18:55:49.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 1 -- 172.21.15.33:0/1328140047 wait complete. 2024-04-23T18:55:49.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.468+0000 7f04babf1740 20 client.18426 trim_cache size 0 max 16384 2024-04-23T18:55:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:55:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:55:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:55:49.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:55:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 msgr2=0x7f049c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 0x7f049c0a6280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009a60 tx=0x7f04a80095c0 comp rx=0 tx=0).stop 2024-04-23T18:55:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 msgr2=0x7f04b0198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f04a000d770 tx=0x7f04a000dc30 comp rx=0 tx=0).stop 2024-04-23T18:55:49.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 shutdown_connections 2024-04-23T18:55:49.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f049c0a3dc0 0x7f049c0a6280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:49.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04b019cf10 0x7f04b019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04b0107290 0x7f04b0198b90 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 --2- 172.21.15.33:0/894988030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04b01021c0 0x7f04b0198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:49.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 >> 172.21.15.33:0/894988030 conn(0x7f04b00fda20 msgr2=0x7f04b00fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:49.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 shutdown_connections 2024-04-23T18:55:49.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:29.469+0000 7f04b7e07640 1 -- 172.21.15.33:0/894988030 wait complete. 2024-04-23T18:55:49.765 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:30 smithi033 ceph-mon[29274]: pgmap v908: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:55:49.845 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:32 smithi033 ceph-mon[29274]: pgmap v909: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:55:49.855 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:34 smithi033 ceph-mon[29274]: pgmap v910: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:55:49.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:36 smithi033 ceph-mon[29274]: pgmap v911: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:38 smithi033 ceph-mon[29274]: pgmap v912: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:50.061 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:40 smithi033 ceph-mon[29274]: pgmap v913: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:42 smithi033 ceph-mon[29274]: pgmap v914: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.172 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:39 smithi122 ceph-mon[32546]: pgmap v946: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:55:50.225 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:41 smithi122 ceph-mon[32546]: pgmap v947: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.239 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:43 smithi122 ceph-mon[32546]: pgmap v948: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:50.305 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:45 smithi122 ceph-mon[32546]: pgmap v949: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.319 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:47 smithi122 ceph-mon[32546]: pgmap v950: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.329 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:49 smithi122 ceph-mon[32546]: pgmap v951: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:50.410 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:39 smithi106 ceph-mon[32716]: pgmap v946: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:55:50.497 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:41 smithi106 ceph-mon[32716]: pgmap v947: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.506 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:43 smithi106 ceph-mon[32716]: pgmap v948: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:50.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:45 smithi106 ceph-mon[32716]: pgmap v949: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.598 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:47 smithi106 ceph-mon[32716]: pgmap v950: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:55:50.656 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:49 smithi106 ceph-mon[32716]: pgmap v951: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:55:50.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- 172.21.15.33:0/887484995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 msgr2=0x7f3d5c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:50.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/887484995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1025a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009900 tx=0x7f3d5802a410 comp rx=0 tx=0).stop 2024-04-23T18:55:50.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- 172.21.15.33:0/887484995 shutdown_connections 2024-04-23T18:55:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/887484995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 0x7f3d5c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:50.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/887484995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c102b70 0x7f3d5c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/887484995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1025a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- 172.21.15.33:0/887484995 >> 172.21.15.33:0/887484995 conn(0x7f3d5c0fda20 msgr2=0x7f3d5c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:51.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- 172.21.15.33:0/887484995 shutdown_connections 2024-04-23T18:55:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- 172.21.15.33:0/887484995 wait complete. 2024-04-23T18:55:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 Processor -- start 2024-04-23T18:55:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.569+0000 7f3d63bfe640 1 -- start start 2024-04-23T18:55:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 0x7f3d5c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:51.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c19cf60 0x7f3d5c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1121d0 con 0x7f3d5c1021c0 2024-04-23T18:55:51.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c112050 con 0x7f3d5c19cf60 2024-04-23T18:55:51.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d63bfe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c112350 con 0x7f3d5c107290 2024-04-23T18:55:51.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:51.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54240/0 (socket says 172.21.15.33:54240) 2024-04-23T18:55:51.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 -- 172.21.15.33:0/2073112732 learned_addr learned my addr 172.21.15.33:0/2073112732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6219b640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c19cf60 0x7f3d5c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d61199640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 0x7f3d5c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 -- 172.21.15.33:0/2073112732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 msgr2=0x7f3d5c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:51.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 0x7f3d5c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 -- 172.21.15.33:0/2073112732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c19cf60 msgr2=0x7f3d5c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:51.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c19cf60 0x7f3d5c19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 -- 172.21.15.33:0/2073112732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c19da50 con 0x7f3d5c1021c0 2024-04-23T18:55:51.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d6199a640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005940 tx=0x7f3d580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:51.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.570+0000 7f3d52ffd640 1 -- 172.21.15.33:0/2073112732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5803f030 con 0x7f3d5c1021c0 2024-04-23T18:55:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.571+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c19dce0 con 0x7f3d5c1021c0 2024-04-23T18:55:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.571+0000 7f3d52ffd640 1 -- 172.21.15.33:0/2073112732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58003cb0 con 0x7f3d5c1021c0 2024-04-23T18:55:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.571+0000 7f3d52ffd640 1 -- 172.21.15.33:0/2073112732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58035070 con 0x7f3d5c1021c0 2024-04-23T18:55:52.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.571+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c1a5500 con 0x7f3d5c1021c0 2024-04-23T18:55:52.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.573+0000 7f3d52ffd640 1 -- 172.21.15.33:0/2073112732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3d58042030 con 0x7f3d5c1021c0 2024-04-23T18:55:52.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 Processor -- start 2024-04-23T18:55:52.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 -- start start 2024-04-23T18:55:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879aca900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559879a2f970 con 0x559879ad9c10 2024-04-23T18:55:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5598799cc4a0 con 0x559879acae40 2024-04-23T18:55:52.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d669ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559879a2b930 con 0x559879aca520 2024-04-23T18:55:52.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d52ffd640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 0x7f3d480a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d61199640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d6219b640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 0x7f3d480a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:53.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d61199640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54242/0 (socket says 172.21.15.33:54242) 2024-04-23T18:55:53.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d61199640 1 -- 172.21.15.33:0/2662572552 learned_addr learned my addr 172.21.15.33:0/2662572552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.574+0000 7f3d6199a640 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:53.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d52ffd640 1 -- 172.21.15.33:0/2073112732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f3d580f1020 con 0x7f3d5c1021c0 2024-04-23T18:55:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d6219b640 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879aca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 -- 172.21.15.33:0/2662572552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 msgr2=0x559879aca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879aca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:53.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 -- 172.21.15.33:0/2662572552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 msgr2=0x559879ad96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:53.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 -- 172.21.15.33:0/2662572552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559879742210 con 0x559879ad9c10 2024-04-23T18:55:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d61199640 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00bc70 tx=0x7f3d4c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d0e926f5b41b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d6219b640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 0x7f3d480a6280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098b0 tx=0x7f3d54009af0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d47fff640 1 -- 172.21.15.33:0/2662572552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c005020 con 0x559879ad9c10 2024-04-23T18:55:53.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d47fff640 1 -- 172.21.15.33:0/2662572552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c0047a0 con 0x559879ad9c10 2024-04-23T18:55:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.575+0000 7f3d47fff640 1 -- 172.21.15.33:0/2662572552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0116a0 con 0x559879ad9c10 2024-04-23T18:55:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- 172.21.15.33:0/2662572552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 msgr2=0x559879adc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:53.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00bc70 tx=0x7f3d4c0107a0 comp rx=0 tx=0).stop 2024-04-23T18:55:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- 172.21.15.33:0/2662572552 shutdown_connections 2024-04-23T18:55:54.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879adc000 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 --2- 172.21.15.33:0/2662572552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879aca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- 172.21.15.33:0/2662572552 >> 172.21.15.33:0/2662572552 conn(0x559879ac6b30 msgr2=0x559879ac6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- 172.21.15.33:0/2662572552 shutdown_connections 2024-04-23T18:55:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- 172.21.15.33:0/2662572552 wait complete. 2024-04-23T18:55:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.576+0000 7f3d669ed740 1 -- start start 2024-04-23T18:55:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 Processor -- start 2024-04-23T18:55:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ad9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559879adc840 con 0x559879ad9c10 2024-04-23T18:55:54.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5598799ca0b0 con 0x559879acae40 2024-04-23T18:55:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559879adc9c0 con 0x559879aca520 2024-04-23T18:55:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d669ed740 10 client.0 _ll_register_callbacks cb 0x5598799b7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:54.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54250/0 (socket says 172.21.15.33:54250) 2024-04-23T18:55:54.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 -- 172.21.15.33:0/415409846 learned_addr learned my addr 172.21.15.33:0/415409846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:54.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d61199640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:54.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6199a640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ad9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 -- 172.21.15.33:0/415409846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 msgr2=0x559879ad9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ad9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 -- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 msgr2=0x559879ad6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.577+0000 7f3d6219b640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:55.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d6219b640 1 -- 172.21.15.33:0/415409846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987980e620 con 0x559879ad9c10 2024-04-23T18:55:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d6219b640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003d60 tx=0x7f3d54009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f72b578ac71c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004450 con 0x559879ad9c10 2024-04-23T18:55:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d4c0047a0 con 0x559879ad9c10 2024-04-23T18:55:55.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f3d540045f0 con 0x559879ad9c10 2024-04-23T18:55:55.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3d540047f0 con 0x559879ad9c10 2024-04-23T18:55:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d669ed740 10 client.18438 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559879ae87f0 con 0x559879ad9c10 2024-04-23T18:55:55.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.578+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c0047a0 con 0x559879ad9c10 2024-04-23T18:55:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f3d5400f040 con 0x559879ad9c10 2024-04-23T18:55:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 10 client.18438 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 10 client.18438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 20 client.18438 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879ae9c60 0x559879aec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 4 client.18438 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:55:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559879a3d390 con 0x559879ae9c60 2024-04-23T18:55:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d6199a640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879ae9c60 0x559879aec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:55.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d457fa640 10 client.18438 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:55.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.579+0000 7f3d6199a640 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879ae9c60 0x559879aec070 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb946b792bea11d1 server_cookie=87715b47a87c4fa4 in_seq=0 out_seq=0 2024-04-23T18:55:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d457fa640 1 -- 172.21.15.33:0/415409846 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559879a3d390 con 0x559879ae9c60 2024-04-23T18:55:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d457fa640 10 client.18438 handle_command_reply: tid=0 2024-04-23T18:55:55.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 client.18438 shutdown 2024-04-23T18:55:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 msgr2=0x559879ad6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3d54003d60 tx=0x7f3d54009600 comp rx=0 tx=0).stop 2024-04-23T18:55:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 shutdown_connections 2024-04-23T18:55:56.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879ae9c60 0x559879aec070 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad6db0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 --2- 172.21.15.33:0/415409846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ad9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d457fa640 0 client.18438 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:55:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 >> 172.21.15.33:0/415409846 conn(0x559879ac6b30 msgr2=0x559879acc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 shutdown_connections 2024-04-23T18:55:56.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.580+0000 7f3d669ed740 1 -- 172.21.15.33:0/415409846 wait complete. 2024-04-23T18:55:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.581+0000 7f3d669ed740 20 client.18438 trim_cache size 0 max 16384 2024-04-23T18:55:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 Processor -- start 2024-04-23T18:55:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 -- start start 2024-04-23T18:55:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ae5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:56.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:56.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:56.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1aa300 con 0x559879ad9c10 2024-04-23T18:55:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1acbe0 con 0x559879acae40 2024-04-23T18:55:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d669ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1aa9d0 con 0x559879aca520 2024-04-23T18:55:57.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54266/0 (socket says 172.21.15.33:54266) 2024-04-23T18:55:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 -- 172.21.15.33:0/420346957 learned_addr learned my addr 172.21.15.33:0/420346957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:57.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d61199640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:57.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6199a640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ae5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 -- 172.21.15.33:0/420346957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 msgr2=0x559879ae5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ae5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 -- 172.21.15.33:0/420346957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 msgr2=0x559879ad1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:57.564 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 -- 172.21.15.33:0/420346957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559879ad7120 con 0x559879ad9c10 2024-04-23T18:55:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6199a640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ae5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d61199640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d6219b640 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3d5402a7b0 tx=0x7f3d54010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baac161205291c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d477fe640 1 -- 172.21.15.33:0/420346957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5402a3e0 con 0x559879ad9c10 2024-04-23T18:55:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d477fe640 1 -- 172.21.15.33:0/420346957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5402a830 con 0x559879ad9c10 2024-04-23T18:55:57.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.596+0000 7f3d477fe640 1 -- 172.21.15.33:0/420346957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54004640 con 0x559879ad9c10 2024-04-23T18:55:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- 172.21.15.33:0/420346957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 msgr2=0x559879ad9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:55:57.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3d5402a7b0 tx=0x7f3d54010f20 comp rx=0 tx=0).stop 2024-04-23T18:55:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- 172.21.15.33:0/420346957 shutdown_connections 2024-04-23T18:55:57.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879ad9c10 0x559879ad9090 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:57.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879acae40 0x559879ad1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 --2- 172.21.15.33:0/420346957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879aca520 0x559879ae5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- 172.21.15.33:0/420346957 >> 172.21.15.33:0/420346957 conn(0x559879ac6b30 msgr2=0x559879ad1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:55:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- 172.21.15.33:0/420346957 shutdown_connections 2024-04-23T18:55:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- 172.21.15.33:0/420346957 wait complete. 2024-04-23T18:55:58.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 -- start start 2024-04-23T18:55:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.597+0000 7f3d669ed740 1 Processor -- start 2024-04-23T18:55:58.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 0x559879add310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 0x559879add850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:58.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1ab0a0 con 0x559879aca520 2024-04-23T18:55:58.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1ac510 con 0x559879ad9c10 2024-04-23T18:55:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1ab770 con 0x559879acae40 2024-04-23T18:55:58.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d669ed740 10 client.0 _ll_register_callbacks cb 0x7f3d54003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:55:58.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54278/0 (socket says 172.21.15.33:54278) 2024-04-23T18:55:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 -- 172.21.15.33:0/3411169375 learned_addr learned my addr 172.21.15.33:0/3411169375 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:55:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d61199640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 0x559879add310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6219b640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 0x559879add850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:55:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 -- 172.21.15.33:0/3411169375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 msgr2=0x559879add310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:58.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 0x559879add310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 -- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 msgr2=0x559879add850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:55:58.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 0x559879add850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:55:58.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 -- 172.21.15.33:0/3411169375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559879ade050 con 0x559879aca520 2024-04-23T18:55:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6219b640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 0x559879add850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:55:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d61199640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 0x559879add310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:55:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d6199a640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3d580b99f0 tx=0x7f3d58032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e11acb21fc6c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:55:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.598+0000 7f3d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:55:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58049040 con 0x559879aca520 2024-04-23T18:55:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7f3d580a6260 con 0x559879aca520 2024-04-23T18:55:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5402a830 con 0x559879aca520 2024-04-23T18:55:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3d580c1320 con 0x559879aca520 2024-04-23T18:55:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 10 client.18444 fetch_fsmap learned FSMap version 47 2024-04-23T18:55:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x559879b50120 con 0x559879aca520 2024-04-23T18:55:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5402a830 con 0x559879aca520 2024-04-23T18:55:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f3d580d6290 con 0x559879aca520 2024-04-23T18:55:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 10 client.18444 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:55:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 10 client.18444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:55:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 20 client.18444 populate_metadata read hostname 'smithi033' 2024-04-23T18:55:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879b507f0 0x559879b52c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:55:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 4 client.18444 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:56:00.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559879a37200 con 0x559879b507f0 2024-04-23T18:56:00.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.599+0000 7f3d61199640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879b507f0 0x559879b52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:00.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.600+0000 7f3d457fa640 10 client.18444 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:00.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.600+0000 7f3d61199640 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879b507f0 0x559879b52c10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c88819579c64ad1 server_cookie=16126021c8845a5 in_seq=0 out_seq=0 2024-04-23T18:56:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d457fa640 1 -- 172.21.15.33:0/3411169375 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559879a37200 con 0x559879b507f0 2024-04-23T18:56:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d457fa640 10 client.18444 handle_command_reply: tid=0 2024-04-23T18:56:00.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 client.18444 shutdown 2024-04-23T18:56:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 msgr2=0x559879ae1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3d580b99f0 tx=0x7f3d58032f60 comp rx=0 tx=0).stop 2024-04-23T18:56:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 shutdown_connections 2024-04-23T18:56:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559879b507f0 0x559879b52c10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559879ad9c10 0x559879add850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559879acae40 0x559879add310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d457fa640 0 client.18444 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:01.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 --2- 172.21.15.33:0/3411169375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559879aca520 0x559879ae1f20 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 >> 172.21.15.33:0/3411169375 conn(0x559879ac6b30 msgr2=0x559879ac5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:01.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 shutdown_connections 2024-04-23T18:56:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 1 -- 172.21.15.33:0/3411169375 wait complete. 2024-04-23T18:56:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.604+0000 7f3d669ed740 20 client.18444 trim_cache size 0 max 16384 2024-04-23T18:56:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:56:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:56:01.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:56:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:56:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 msgr2=0x7f3d480a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 0x7f3d480a6280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3d540098b0 tx=0x7f3d54009af0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 msgr2=0x7f3d5c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:01.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005940 tx=0x7f3d580042f0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 shutdown_connections 2024-04-23T18:56:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3d480a3dc0 0x7f3d480a6280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d5c19cf60 0x7f3d5c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d5c107290 0x7f3d5c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 --2- 172.21.15.33:0/2073112732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5c1021c0 0x7f3d5c1986f0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:02.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 >> 172.21.15.33:0/2073112732 conn(0x7f3d5c0fda20 msgr2=0x7f3d5c0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 shutdown_connections 2024-04-23T18:56:02.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:42.605+0000 7f3d63bfe640 1 -- 172.21.15.33:0/2073112732 wait complete. 2024-04-23T18:56:02.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:44 smithi033 ceph-mon[29274]: pgmap v915: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:02.247 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:46 smithi033 ceph-mon[29274]: pgmap v916: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-23T18:56:02.354 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:48 smithi033 ceph-mon[29274]: pgmap v917: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:02.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:50 smithi033 ceph-mon[29274]: pgmap v918: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:02.538 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:52 smithi033 ceph-mon[29274]: pgmap v919: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:02.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:54 smithi033 ceph-mon[29274]: pgmap v920: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:02.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:51 smithi122 ceph-mon[32546]: pgmap v952: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:02.815 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:53 smithi122 ceph-mon[32546]: pgmap v953: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:02.860 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:55 smithi122 ceph-mon[32546]: pgmap v954: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:02.944 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:57 smithi122 ceph-mon[32546]: pgmap v955: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:03.021 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:55:59 smithi122 ceph-mon[32546]: pgmap v956: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:03.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:01 smithi122 ceph-mon[32546]: pgmap v957: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:03.192 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:51 smithi106 ceph-mon[32716]: pgmap v952: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:03.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:53 smithi106 ceph-mon[32716]: pgmap v953: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:03.346 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:55 smithi106 ceph-mon[32716]: pgmap v954: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:03.363 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:57 smithi106 ceph-mon[32716]: pgmap v955: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:03.425 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:55:59 smithi106 ceph-mon[32716]: pgmap v956: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:03.435 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:01 smithi106 ceph-mon[32716]: pgmap v957: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:03.499 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:03 smithi122 ceph-mon[32546]: pgmap v958: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:03.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:03 smithi106 ceph-mon[32716]: pgmap v958: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 -- 172.21.15.33:0/3552599910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c102b70 msgr2=0x7fb87c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 --2- 172.21.15.33:0/3552599910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c102b70 0x7fb87c106bb0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb864009900 tx=0x7fb86402a410 comp rx=0 tx=0).stop 2024-04-23T18:56:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 -- 172.21.15.33:0/3552599910 shutdown_connections 2024-04-23T18:56:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 --2- 172.21.15.33:0/3552599910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c107290 0x7fb87c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 --2- 172.21.15.33:0/3552599910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c102b70 0x7fb87c106bb0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 --2- 172.21.15.33:0/3552599910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c1021c0 0x7fb87c1025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:03.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 -- 172.21.15.33:0/3552599910 >> 172.21.15.33:0/3552599910 conn(0x7fb87c0fda20 msgr2=0x7fb87c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.705+0000 7fb88139b640 1 -- 172.21.15.33:0/3552599910 shutdown_connections 2024-04-23T18:56:03.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.706+0000 7fb88139b640 1 -- 172.21.15.33:0/3552599910 wait complete. 2024-04-23T18:56:03.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.706+0000 7fb88139b640 1 Processor -- start 2024-04-23T18:56:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.706+0000 7fb88139b640 1 -- start start 2024-04-23T18:56:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c1021c0 0x7fb87c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:04.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:04.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c19cf30 0x7fb87c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:04.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1121d0 con 0x7fb87c107290 2024-04-23T18:56:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c112050 con 0x7fb87c19cf30 2024-04-23T18:56:04.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb88139b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c112350 con 0x7fb87c1021c0 2024-04-23T18:56:04.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51578/0 (socket says 172.21.15.33:51578) 2024-04-23T18:56:04.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1150859875 learned_addr learned my addr 172.21.15.33:0/1150859875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:04.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87affd640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c1021c0 0x7fb87c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c19cf30 0x7fb87c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1150859875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c1021c0 msgr2=0x7fb87c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:04.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c1021c0 0x7fb87c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:04.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1150859875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c19cf30 msgr2=0x7fb87c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:04.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c19cf30 0x7fb87c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:04.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1150859875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c19da20 con 0x7fb87c107290 2024-04-23T18:56:04.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.707+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb864006010 tx=0x7fb864004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.708+0000 7fb873fff640 1 -- 172.21.15.33:0/1150859875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86403f030 con 0x7fb87c107290 2024-04-23T18:56:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.708+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c19dcb0 con 0x7fb87c107290 2024-04-23T18:56:04.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.708+0000 7fb873fff640 1 -- 172.21.15.33:0/1150859875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb864003aa0 con 0x7fb87c107290 2024-04-23T18:56:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.708+0000 7fb873fff640 1 -- 172.21.15.33:0/1150859875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864035070 con 0x7fb87c107290 2024-04-23T18:56:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.708+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1a54d0 con 0x7fb87c107290 2024-04-23T18:56:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.710+0000 7fb873fff640 1 -- 172.21.15.33:0/1150859875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb864042030 con 0x7fb87c107290 2024-04-23T18:56:04.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.710+0000 7fb884192740 1 Processor -- start 2024-04-23T18:56:04.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 -- start start 2024-04-23T18:56:04.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:04.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc14f970 con 0x55bbcc1ea620 2024-04-23T18:56:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc0ec4a0 con 0x55bbcc1f5800 2024-04-23T18:56:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb884192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc14b930 con 0x55bbcc1eaf40 2024-04-23T18:56:05.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb873fff640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 0x7fb8680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51586/0 (socket says 172.21.15.33:51586) 2024-04-23T18:56:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87b7fe640 1 -- 172.21.15.33:0/338499405 learned_addr learned my addr 172.21.15.33:0/338499405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87affd640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:05.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.711+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 0x7fb8680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb873fff640 1 -- 172.21.15.33:0/1150859875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7fb8640f0bd0 con 0x7fb87c107290 2024-04-23T18:56:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 -- 172.21.15.33:0/338499405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 msgr2=0x55bbcc1f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 -- 172.21.15.33:0/338499405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 msgr2=0x55bbcc1f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 -- 172.21.15.33:0/338499405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcbe62210 con 0x55bbcc1ea620 2024-04-23T18:56:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87affd640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:56:05.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:56:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb86c012d00 tx=0x7fb86c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597decb6ce89744b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 0x7fb8680a6280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb86c010c40 tx=0x7fb86c00c000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb8717fa640 1 -- 172.21.15.33:0/338499405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c024070 con 0x55bbcc1ea620 2024-04-23T18:56:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb8717fa640 1 -- 172.21.15.33:0/338499405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb86c009d20 con 0x55bbcc1ea620 2024-04-23T18:56:06.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.712+0000 7fb8717fa640 1 -- 172.21.15.33:0/338499405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c005280 con 0x55bbcc1ea620 2024-04-23T18:56:06.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- 172.21.15.33:0/338499405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 msgr2=0x55bbcc1eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:06.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fb86c012d00 tx=0x7fb86c010740 comp rx=0 tx=0).stop 2024-04-23T18:56:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- 172.21.15.33:0/338499405 shutdown_connections 2024-04-23T18:56:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 --2- 172.21.15.33:0/338499405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc1eaa00 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- 172.21.15.33:0/338499405 >> 172.21.15.33:0/338499405 conn(0x55bbcc1e6b30 msgr2=0x55bbcc1e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- 172.21.15.33:0/338499405 shutdown_connections 2024-04-23T18:56:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- 172.21.15.33:0/338499405 wait complete. 2024-04-23T18:56:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.713+0000 7fb884192740 1 -- start start 2024-04-23T18:56:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 Processor -- start 2024-04-23T18:56:06.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 0x55bbcc1f5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc1f83d0 con 0x55bbcc1eaf40 2024-04-23T18:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc0ea0b0 con 0x55bbcc1f5800 2024-04-23T18:56:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbcc1f8550 con 0x55bbcc1ea620 2024-04-23T18:56:07.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb884192740 10 client.0 _ll_register_callbacks cb 0x55bbcc0d7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51596/0 (socket says 172.21.15.33:51596) 2024-04-23T18:56:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 -- 172.21.15.33:0/2576605506 learned_addr learned my addr 172.21.15.33:0/2576605506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87affd640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 0x55bbcc1f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 -- 172.21.15.33:0/2576605506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 msgr2=0x55bbcc1f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:07.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 0x55bbcc1f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 -- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 msgr2=0x55bbcc1f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:07.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:07.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 -- 172.21.15.33:0/2576605506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcbf2e620 con 0x55bbcc1eaf40 2024-04-23T18:56:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87affd640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 0x55bbcc1f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:07.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb86403e040 tx=0x7fb864003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01b947044f1dffa server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.714+0000 7fb8627fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:07.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864049040 con 0x55bbcc1eaf40 2024-04-23T18:56:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7fb864003c50 con 0x55bbcc1eaf40 2024-04-23T18:56:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbcc15d390 con 0x55bbcc1eaf40 2024-04-23T18:56:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb8640c0ed0 con 0x55bbcc1eaf40 2024-04-23T18:56:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 10 client.18456 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bbcc204380 con 0x55bbcc1eaf40 2024-04-23T18:56:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bbcc15d390 con 0x55bbcc1eaf40 2024-04-23T18:56:07.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7fb8640d8050 con 0x55bbcc1eaf40 2024-04-23T18:56:07.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 10 client.18456 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 20 client.18456 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc205790 0x55bbcc207c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 4 client.18456 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:56:08.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.715+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbcc157200 con 0x55bbcc205790 2024-04-23T18:56:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.716+0000 7fb87affd640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc205790 0x55bbcc207c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:08.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.716+0000 7fb8627fc640 10 client.18456 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.716+0000 7fb87affd640 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc205790 0x55bbcc207c00 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f435cbd221014 server_cookie=9eb6cc9ceb8deaba in_seq=0 out_seq=0 2024-04-23T18:56:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb8627fc640 1 -- 172.21.15.33:0/2576605506 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbcc157200 con 0x55bbcc205790 2024-04-23T18:56:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb8627fc640 10 client.18456 handle_command_reply: tid=0 2024-04-23T18:56:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 client.18456 shutdown 2024-04-23T18:56:08.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 msgr2=0x55bbcc1f24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:08.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fb86403e040 tx=0x7fb864003f50 comp rx=0 tx=0).stop 2024-04-23T18:56:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 shutdown_connections 2024-04-23T18:56:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc205790 0x55bbcc207c00 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f24f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb8627fc640 0 client.18456 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 --2- 172.21.15.33:0/2576605506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1ea620 0x55bbcc1f5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 >> 172.21.15.33:0/2576605506 conn(0x55bbcc1e6b30 msgr2=0x55bbcc1ec860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:09.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 shutdown_connections 2024-04-23T18:56:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 1 -- 172.21.15.33:0/2576605506 wait complete. 2024-04-23T18:56:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.717+0000 7fb884192740 20 client.18456 trim_cache size 0 max 16384 2024-04-23T18:56:09.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 Processor -- start 2024-04-23T18:56:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 -- start start 2024-04-23T18:56:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc20d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ac520 con 0x55bbcc1ea620 2024-04-23T18:56:09.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1aee00 con 0x55bbcc1f5800 2024-04-23T18:56:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.732+0000 7fb884192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1acbf0 con 0x55bbcc1eaf40 2024-04-23T18:56:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:09.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51602/0 (socket says 172.21.15.33:51602) 2024-04-23T18:56:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 -- 172.21.15.33:0/422081015 learned_addr learned my addr 172.21.15.33:0/422081015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc20d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 -- 172.21.15.33:0/422081015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 msgr2=0x55bbcc1f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 -- 172.21.15.33:0/422081015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 msgr2=0x55bbcc20d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc20d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:09.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 -- 172.21.15.33:0/422081015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbcc208250 con 0x55bbcc1ea620 2024-04-23T18:56:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb87affd640 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb874009c40 tx=0x7fb874009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98943115eff6da69 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb870ff9640 1 -- 172.21.15.33:0/422081015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb874005020 con 0x55bbcc1ea620 2024-04-23T18:56:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb870ff9640 1 -- 172.21.15.33:0/422081015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8740047a0 con 0x55bbcc1ea620 2024-04-23T18:56:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.733+0000 7fb870ff9640 1 -- 172.21.15.33:0/422081015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87401f350 con 0x55bbcc1ea620 2024-04-23T18:56:10.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- 172.21.15.33:0/422081015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 msgr2=0x55bbcc201290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:10.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fb874009c40 tx=0x7fb874009420 comp rx=0 tx=0).stop 2024-04-23T18:56:10.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- 172.21.15.33:0/422081015 shutdown_connections 2024-04-23T18:56:10.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1f5800 0x55bbcc20d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- 172.21.15.33:0/422081015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1ea620 0x55bbcc201290 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:10.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- 172.21.15.33:0/422081015 >> 172.21.15.33:0/422081015 conn(0x55bbcc1e6b30 msgr2=0x55bbcc1e5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:10.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- 172.21.15.33:0/422081015 shutdown_connections 2024-04-23T18:56:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- 172.21.15.33:0/422081015 wait complete. 2024-04-23T18:56:10.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- start start 2024-04-23T18:56:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 Processor -- start 2024-04-23T18:56:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1ea620 0x55bbcc1fa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1f5800 0x55bbcc1fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ad2c0 con 0x55bbcc1eaf40 2024-04-23T18:56:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ae730 con 0x55bbcc1ea620 2024-04-23T18:56:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.734+0000 7fb884192740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1ad990 con 0x55bbcc1f5800 2024-04-23T18:56:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb884192740 10 client.0 _ll_register_callbacks cb 0x7fb8640bc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:11.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51606/0 (socket says 172.21.15.33:51606) 2024-04-23T18:56:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1713369232 learned_addr learned my addr 172.21.15.33:0/1713369232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:11.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87affd640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1ea620 0x55bbcc1fa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87b7fe640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1f5800 0x55bbcc1fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:11.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1713369232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1f5800 msgr2=0x55bbcc1fb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:11.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1f5800 0x55bbcc1fb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1ea620 msgr2=0x55bbcc1fa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1ea620 0x55bbcc1fa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 -- 172.21.15.33:0/1713369232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbcc1e1090 con 0x55bbcc1eaf40 2024-04-23T18:56:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb87a7fc640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb8640b95a0 tx=0x7fb864009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f7d76956e04ebe server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:11.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb8627fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864049020 con 0x55bbcc1eaf40 2024-04-23T18:56:12.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(143..143 src has 1..143) ==== 8498+0+0 (secure 0 0 0) 0x7fb864003c50 con 0x55bbcc1eaf40 2024-04-23T18:56:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8740047a0 con 0x55bbcc1eaf40 2024-04-23T18:56:12.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb8640c5790 con 0x55bbcc1eaf40 2024-04-23T18:56:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb884192740 10 client.18462 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.735+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bbcc26cde0 con 0x55bbcc1eaf40 2024-04-23T18:56:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8740047a0 con 0x55bbcc1eaf40 2024-04-23T18:56:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7fb8640d8070 con 0x55bbcc1eaf40 2024-04-23T18:56:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 10 client.18462 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 10 client.18462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:12.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 20 client.18462 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc26d370 0x55bbcc26f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 4 client.18462 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:56:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbcc0d35f0 con 0x55bbcc26d370 2024-04-23T18:56:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb87affd640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc26d370 0x55bbcc26f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb8627fc640 10 client.18462 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:12.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.736+0000 7fb87affd640 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc26d370 0x55bbcc26f790 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c31f575370c20fa0 server_cookie=e661e734b1a07120 in_seq=0 out_seq=0 2024-04-23T18:56:12.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.740+0000 7fb8627fc640 1 -- 172.21.15.33:0/1713369232 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbcc0d35f0 con 0x55bbcc26d370 2024-04-23T18:56:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.740+0000 7fb8627fc640 10 client.18462 handle_command_reply: tid=0 2024-04-23T18:56:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.740+0000 7fb884192740 1 client.18462 shutdown 2024-04-23T18:56:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.740+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 msgr2=0x55bbcc1facb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.740+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb8640b95a0 tx=0x7fb864009260 comp rx=0 tx=0).stop 2024-04-23T18:56:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 shutdown_connections 2024-04-23T18:56:13.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55bbcc26d370 0x55bbcc26f790 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbcc1f5800 0x55bbcc1fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbcc1eaf40 0x55bbcc1facb0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb8627fc640 0 client.18462 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 --2- 172.21.15.33:0/1713369232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bbcc1ea620 0x55bbcc1fa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 >> 172.21.15.33:0/1713369232 conn(0x55bbcc1e6b30 msgr2=0x55bbcc1e5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 shutdown_connections 2024-04-23T18:56:13.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 1 -- 172.21.15.33:0/1713369232 wait complete. 2024-04-23T18:56:13.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb884192740 20 client.18462 trim_cache size 0 max 16384 2024-04-23T18:56:13.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:56:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:56:13.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:56:13.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:56:13.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 msgr2=0x7fb8680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 0x7fb8680a6280 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb86c010c40 tx=0x7fb86c00c000 comp rx=0 tx=0).stop 2024-04-23T18:56:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 msgr2=0x7fb87c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb864006010 tx=0x7fb864004060 comp rx=0 tx=0).stop 2024-04-23T18:56:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 shutdown_connections 2024-04-23T18:56:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb8680a3dc0 0x7fb8680a6280 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb87c19cf30 0x7fb87c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb87c107290 0x7fb87c198bb0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 --2- 172.21.15.33:0/1150859875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb87c1021c0 0x7fb87c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 >> 172.21.15.33:0/1150859875 conn(0x7fb87c0fda20 msgr2=0x7fb87c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 shutdown_connections 2024-04-23T18:56:14.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:54:55.741+0000 7fb88139b640 1 -- 172.21.15.33:0/1150859875 wait complete. 2024-04-23T18:56:14.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:56 smithi033 ceph-mon[29274]: pgmap v921: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:14.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:54:58 smithi033 ceph-mon[29274]: pgmap v922: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:14.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:00 smithi033 ceph-mon[29274]: pgmap v923: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:14.501 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:02 smithi033 ceph-mon[29274]: pgmap v924: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:04 smithi033 ceph-mon[29274]: pgmap v925: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:14.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:06 smithi033 ceph-mon[29274]: pgmap v926: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:14.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:08 smithi033 ceph-mon[29274]: pgmap v927: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:56:14.718 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:05 smithi106 ceph-mon[32716]: pgmap v959: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:14.727 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:05 smithi106 ceph-mon[32716]: osdmap e146: 12 total, 12 up, 12 in 2024-04-23T18:56:14.818 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:07 smithi106 ceph-mon[32716]: pgmap v961: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:56:14.874 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:09 smithi106 ceph-mon[32716]: pgmap v962: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:14.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:11 smithi106 ceph-mon[32716]: pgmap v963: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:14.890 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:11 smithi106 ceph-mon[32716]: osdmap e147: 12 total, 12 up, 12 in 2024-04-23T18:56:14.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:12 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:56:14.944 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: pgmap v965: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:56:14.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:56:14.989 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:56:15.075 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: pgmap v966: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-04-23T18:56:15.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: pgmap v967: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 44 op/s 2024-04-23T18:56:15.092 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:13 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:56:15.194 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:05 smithi122 ceph-mon[32546]: pgmap v959: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:15.247 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:05 smithi122 ceph-mon[32546]: osdmap e146: 12 total, 12 up, 12 in 2024-04-23T18:56:15.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:07 smithi122 ceph-mon[32546]: pgmap v961: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:56:15.385 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:09 smithi122 ceph-mon[32546]: pgmap v962: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:15.513 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:11 smithi122 ceph-mon[32546]: pgmap v963: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:56:15.521 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:11 smithi122 ceph-mon[32546]: osdmap e147: 12 total, 12 up, 12 in 2024-04-23T18:56:15.589 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:12 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:56:15.597 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: pgmap v965: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:56:15.605 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:56:15.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:56:15.664 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: pgmap v966: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-04-23T18:56:15.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: pgmap v967: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 44 op/s 2024-04-23T18:56:15.786 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:13 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:56:15.820 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:15 smithi106 ceph-mon[32716]: pgmap v968: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 150 B/s wr, 6 op/s 2024-04-23T18:56:15.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:15 smithi106 ceph-mon[32716]: daemon mds.j restarted 2024-04-23T18:56:15.901 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:15 smithi122 ceph-mon[32546]: pgmap v968: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 150 B/s wr, 6 op/s 2024-04-23T18:56:15.908 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:15 smithi122 ceph-mon[32546]: daemon mds.j restarted 2024-04-23T18:56:16.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2049008689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 msgr2=0x7ffa6810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:16.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2049008689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa6810c000 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009900 tx=0x7ffa5802a480 comp rx=0 tx=0).stop 2024-04-23T18:56:16.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2049008689 shutdown_connections 2024-04-23T18:56:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2049008689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa6810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2049008689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa6810c000 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2049008689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 0x7ffa681035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:16.285 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2049008689 >> 172.21.15.33:0/2049008689 conn(0x7ffa680771d0 msgr2=0x7ffa680775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:16.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2049008689 shutdown_connections 2024-04-23T18:56:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.841+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2049008689 wait complete. 2024-04-23T18:56:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 Processor -- start 2024-04-23T18:56:16.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 -- start start 2024-04-23T18:56:16.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 0x7ffa68100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa681018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:16.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68112210 con 0x7ffa68103b20 2024-04-23T18:56:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68112090 con 0x7ffa6810c5d0 2024-04-23T18:56:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6f3c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa68112390 con 0x7ffa68103200 2024-04-23T18:56:16.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6c95f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:16.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6c95f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49390/0 (socket says 172.21.15.33:49390) 2024-04-23T18:56:16.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6c95f640 1 -- 172.21.15.33:0/2887051760 learned_addr learned my addr 172.21.15.33:0/2887051760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:17.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.842+0000 7ffa6d160640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 0x7ffa68100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6d961640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa681018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 -- 172.21.15.33:0/2887051760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 msgr2=0x7ffa68100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:17.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 0x7ffa68100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 -- 172.21.15.33:0/2887051760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 msgr2=0x7ffa681018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa681018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 -- 172.21.15.33:0/2887051760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa681a6c70 con 0x7ffa68103b20 2024-04-23T18:56:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6d961640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa681018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005940 tx=0x7ffa580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:17.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa5e7fc640 1 -- 172.21.15.33:0/2887051760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5803f030 con 0x7ffa68103b20 2024-04-23T18:56:17.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa5e7fc640 1 -- 172.21.15.33:0/2887051760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa58003cb0 con 0x7ffa68103b20 2024-04-23T18:56:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa681a6f00 con 0x7ffa68103b20 2024-04-23T18:56:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa5e7fc640 1 -- 172.21.15.33:0/2887051760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58035070 con 0x7ffa68103b20 2024-04-23T18:56:17.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.843+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa681a7360 con 0x7ffa68103b20 2024-04-23T18:56:17.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.846+0000 7ffa5e7fc640 1 -- 172.21.15.33:0/2887051760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa58042030 con 0x7ffa68103b20 2024-04-23T18:56:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.846+0000 7ffa721b0740 1 Processor -- start 2024-04-23T18:56:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 -- start start 2024-04-23T18:56:17.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e90a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:17.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:17.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437e9dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:17.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b437df5970 con 0x55b437e90f40 2024-04-23T18:56:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b437d924a0 con 0x55b437e90620 2024-04-23T18:56:17.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa721b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b437df1930 con 0x55b437e9b800 2024-04-23T18:56:17.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49404/0 (socket says 172.21.15.33:49404) 2024-04-23T18:56:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 -- 172.21.15.33:0/1350850047 learned_addr learned my addr 172.21.15.33:0/1350850047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:17.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d961640 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e90a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437e9dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:17.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa5e7fc640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 0x7ffa540a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:17.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 -- 172.21.15.33:0/1350850047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 msgr2=0x55b437e9dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:18.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437e9dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 -- 172.21.15.33:0/1350850047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 msgr2=0x55b437e90a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e90a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d160640 1 -- 172.21.15.33:0/1350850047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b437b08210 con 0x55b437e90f40 2024-04-23T18:56:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.847+0000 7ffa6d961640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 0x7ffa540a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.848+0000 7ffa5e7fc640 1 -- 172.21.15.33:0/2887051760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 8586+0+0 (secure 0 0 0) 0x7ffa580f0d10 con 0x7ffa68103b20 2024-04-23T18:56:18.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.848+0000 7ffa6d160640 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400b760 tx=0x7ffa6400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe948dd09b86c8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:18.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.848+0000 7ffa537fe640 1 -- 172.21.15.33:0/1350850047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64003dd0 con 0x55b437e90f40 2024-04-23T18:56:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.848+0000 7ffa537fe640 1 -- 172.21.15.33:0/1350850047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa64015070 con 0x55b437e90f40 2024-04-23T18:56:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.848+0000 7ffa537fe640 1 -- 172.21.15.33:0/1350850047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64004d90 con 0x55b437e90f40 2024-04-23T18:56:18.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- 172.21.15.33:0/1350850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 msgr2=0x55b437e9b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa6d961640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 0x7ffa540a62b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009370 tx=0x7ffa60004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400b760 tx=0x7ffa6400bc20 comp rx=0 tx=0).stop 2024-04-23T18:56:18.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- 172.21.15.33:0/1350850047 shutdown_connections 2024-04-23T18:56:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437e9dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:18.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9b2c0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 --2- 172.21.15.33:0/1350850047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e90a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- 172.21.15.33:0/1350850047 >> 172.21.15.33:0/1350850047 conn(0x55b437e8cb30 msgr2=0x55b437e8cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- 172.21.15.33:0/1350850047 shutdown_connections 2024-04-23T18:56:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- 172.21.15.33:0/1350850047 wait complete. 2024-04-23T18:56:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 -- start start 2024-04-23T18:56:18.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.849+0000 7ffa721b0740 1 Processor -- start 2024-04-23T18:56:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 0x55b437e983c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:19.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 0x55b437e98900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b437e9e430 con 0x55b437e90620 2024-04-23T18:56:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b437d900b0 con 0x55b437e9b800 2024-04-23T18:56:19.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b437e9e5b0 con 0x55b437e90f40 2024-04-23T18:56:19.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa721b0740 10 client.0 _ll_register_callbacks cb 0x55b437d7dd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:19.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49412/0 (socket says 172.21.15.33:49412) 2024-04-23T18:56:19.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 -- 172.21.15.33:0/906530746 learned_addr learned my addr 172.21.15.33:0/906530746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:19.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d961640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 0x55b437e98900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:19.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 0x55b437e983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 -- 172.21.15.33:0/906530746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 msgr2=0x55b437e983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:19.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 0x55b437e983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:19.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 -- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 msgr2=0x55b437e98900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 0x55b437e98900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 -- 172.21.15.33:0/906530746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b437bd4620 con 0x55b437e90620 2024-04-23T18:56:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 0x55b437e983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d961640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 0x55b437e98900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:19.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa6d160640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ffa640039c0 tx=0x7ffa64003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59c99af81ebf24c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.850+0000 7ffa50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64028060 con 0x55b437e90620 2024-04-23T18:56:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 8586+0+0 (secure 0 0 0) 0x7ffa64011850 con 0x55b437e90620 2024-04-23T18:56:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa64015070 con 0x55b437e90620 2024-04-23T18:56:20.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffa640046c0 con 0x55b437e90620 2024-04-23T18:56:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 10 client.18474 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:20.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b437eaa2d0 con 0x55b437e90620 2024-04-23T18:56:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa64015070 con 0x55b437e90620 2024-04-23T18:56:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7ffa64022070 con 0x55b437e90620 2024-04-23T18:56:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 10 client.18474 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 20 client.18474 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437eab680 0x55b437eada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 4 client.18474 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:56:20.524 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.851+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b437e03390 con 0x55b437eab680 2024-04-23T18:56:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.852+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437eab680 0x55b437eada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.852+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437eab680 0x55b437eada70 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7fc2f389b25850 server_cookie=48e1fca7359dcea1 in_seq=0 out_seq=0 2024-04-23T18:56:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.852+0000 7ffa50ff9640 10 client.18474 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa50ff9640 1 -- 172.21.15.33:0/906530746 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b437e03390 con 0x55b437eab680 2024-04-23T18:56:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa50ff9640 10 client.18474 handle_command_reply: tid=0 2024-04-23T18:56:20.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 client.18474 shutdown 2024-04-23T18:56:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 msgr2=0x55b437e9b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7ffa640039c0 tx=0x7ffa64003e00 comp rx=0 tx=0).stop 2024-04-23T18:56:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 shutdown_connections 2024-04-23T18:56:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437eab680 0x55b437eada70 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e9b800 0x55b437e98900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:21.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa50ff9640 0 client.18474 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e90f40 0x55b437e983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 --2- 172.21.15.33:0/906530746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437e9b0d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:21.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 >> 172.21.15.33:0/906530746 conn(0x55b437e8cb30 msgr2=0x55b437e928b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:21.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 shutdown_connections 2024-04-23T18:56:21.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 1 -- 172.21.15.33:0/906530746 wait complete. 2024-04-23T18:56:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.853+0000 7ffa721b0740 20 client.18474 trim_cache size 0 max 16384 2024-04-23T18:56:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.868+0000 7ffa721b0740 1 Processor -- start 2024-04-23T18:56:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.868+0000 7ffa721b0740 1 -- start start 2024-04-23T18:56:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90f40 0x55b437e9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437eb3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681ae940 con 0x55b437e90620 2024-04-23T18:56:21.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681b1220 con 0x55b437e90f40 2024-04-23T18:56:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa721b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681af010 con 0x55b437e9b800 2024-04-23T18:56:21.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:21.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49420/0 (socket says 172.21.15.33:49420) 2024-04-23T18:56:21.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 -- 172.21.15.33:0/165257454 learned_addr learned my addr 172.21.15.33:0/165257454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 -- 172.21.15.33:0/165257454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 msgr2=0x55b437eb3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437eb3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 -- 172.21.15.33:0/165257454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90f40 msgr2=0x55b437e9ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:22.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90f40 0x55b437e9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 -- 172.21.15.33:0/165257454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b437e8db30 con 0x55b437e90620 2024-04-23T18:56:22.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa6d160640 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ffa640044d0 tx=0x7ffa640041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc6ddf4bf119c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:22.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa52ffd640 1 -- 172.21.15.33:0/165257454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64028070 con 0x55b437e90620 2024-04-23T18:56:22.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa52ffd640 1 -- 172.21.15.33:0/165257454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa64015030 con 0x55b437e90620 2024-04-23T18:56:22.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.869+0000 7ffa52ffd640 1 -- 172.21.15.33:0/165257454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64019680 con 0x55b437e90620 2024-04-23T18:56:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- 172.21.15.33:0/165257454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 msgr2=0x55b437ea72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:22.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7ffa640044d0 tx=0x7ffa640041a0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- 172.21.15.33:0/165257454 shutdown_connections 2024-04-23T18:56:22.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437eb3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90f40 0x55b437e9ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 --2- 172.21.15.33:0/165257454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90620 0x55b437ea72f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- 172.21.15.33:0/165257454 >> 172.21.15.33:0/165257454 conn(0x55b437e8cb30 msgr2=0x55b437e93a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- 172.21.15.33:0/165257454 shutdown_connections 2024-04-23T18:56:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- 172.21.15.33:0/165257454 wait complete. 2024-04-23T18:56:22.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 -- start start 2024-04-23T18:56:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.870+0000 7ffa721b0740 1 Processor -- start 2024-04-23T18:56:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e9e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437f0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681af6e0 con 0x55b437e90f40 2024-04-23T18:56:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681b0b50 con 0x55b437e90620 2024-04-23T18:56:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa681afdb0 con 0x55b437e9b800 2024-04-23T18:56:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa721b0740 10 client.0 _ll_register_callbacks cb 0x7ffa6400be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:23.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49436/0 (socket says 172.21.15.33:49436) 2024-04-23T18:56:23.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6d160640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e9e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:23.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 -- 172.21.15.33:0/70319811 learned_addr learned my addr 172.21.15.33:0/70319811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:23.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6d961640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437f0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 -- 172.21.15.33:0/70319811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 msgr2=0x55b437f0fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:23.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437f0fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:23.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 -- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 msgr2=0x55b437e9e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:23.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e9e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 -- 172.21.15.33:0/70319811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b437f100b0 con 0x55b437e90f40 2024-04-23T18:56:23.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6d961640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437f0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:23.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa6c95f640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ffa58031440 tx=0x7ffa5802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f672a1c1e5f8613 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:23.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.871+0000 7ffa50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:23.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58049040 con 0x55b437e90f40 2024-04-23T18:56:23.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 8586+0+0 (secure 0 0 0) 0x7ffa580ee6a0 con 0x55b437e90f40 2024-04-23T18:56:23.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa64015030 con 0x55b437e90f40 2024-04-23T18:56:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ffa58003e60 con 0x55b437e90f40 2024-04-23T18:56:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 10 client.18480 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55b437f113f0 con 0x55b437e90f40 2024-04-23T18:56:23.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa64015030 con 0x55b437e90f40 2024-04-23T18:56:23.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7ffa580b92e0 con 0x55b437e90f40 2024-04-23T18:56:23.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 10 client.18480 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 10 client.18480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:24.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 20 client.18480 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437f11b20 0x55b437f13f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 4 client.18480 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:56:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b437dfd200 con 0x55b437f11b20 2024-04-23T18:56:24.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.872+0000 7ffa6d160640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437f11b20 0x55b437f13f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.873+0000 7ffa6d160640 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437f11b20 0x55b437f13f40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172ffc63c2067a58 server_cookie=9131ca3df2e49066 in_seq=0 out_seq=0 2024-04-23T18:56:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.873+0000 7ffa50ff9640 10 client.18480 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:24.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.879+0000 7ffa50ff9640 1 -- 172.21.15.33:0/70319811 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b437dfd200 con 0x55b437f11b20 2024-04-23T18:56:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.879+0000 7ffa50ff9640 10 client.18480 handle_command_reply: tid=0 2024-04-23T18:56:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.879+0000 7ffa721b0740 1 client.18480 shutdown 2024-04-23T18:56:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 msgr2=0x55b437e9eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:24.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ffa58031440 tx=0x7ffa5802a480 comp rx=0 tx=0).stop 2024-04-23T18:56:24.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 shutdown_connections 2024-04-23T18:56:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55b437f11b20 0x55b437f13f40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b437e9b800 0x55b437f0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b437e90f40 0x55b437e9eef0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 --2- 172.21.15.33:0/70319811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b437e90620 0x55b437e9e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa50ff9640 0 client.18480 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:24.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 >> 172.21.15.33:0/70319811 conn(0x55b437e8cb30 msgr2=0x55b437e91fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 shutdown_connections 2024-04-23T18:56:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 1 -- 172.21.15.33:0/70319811 wait complete. 2024-04-23T18:56:25.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa721b0740 20 client.18480 trim_cache size 0 max 16384 2024-04-23T18:56:25.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:56:25.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:56:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:56:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:56:25.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 msgr2=0x7ffa540a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 0x7ffa540a62b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffa60009370 tx=0x7ffa60004a40 comp rx=0 tx=0).stop 2024-04-23T18:56:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 msgr2=0x7ffa68101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:25.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005940 tx=0x7ffa580042f0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 shutdown_connections 2024-04-23T18:56:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ffa540a3df0 0x7ffa540a62b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa6810c5d0 0x7ffa681018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa68103b20 0x7ffa68101390 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 --2- 172.21.15.33:0/2887051760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa68103200 0x7ffa68100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.880+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 >> 172.21.15.33:0/2887051760 conn(0x7ffa680771d0 msgr2=0x7ffa680ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.881+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 shutdown_connections 2024-04-23T18:56:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:08.881+0000 7ffa6f3c4640 1 -- 172.21.15.33:0/2887051760 wait complete. 2024-04-23T18:56:25.718 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:09 smithi033 ceph-mon[29274]: osdmap e144: 12 total, 12 up, 12 in 2024-04-23T18:56:25.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:10 smithi033 ceph-mon[29274]: pgmap v929: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:56:25.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:16 smithi122 ceph-mon[32546]: osdmap e148: 12 total, 12 up, 12 in 2024-04-23T18:56:25.820 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6834/395009281,v1:172.21.15.33:6835/395009281] up:boot 2024-04-23T18:56:25.831 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:56:25.838 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:56:25.964 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:56:26.044 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:17 smithi122 ceph-mon[32546]: pgmap v970: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-23T18:56:26.053 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:19 smithi122 ceph-mon[32546]: pgmap v971: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-04-23T18:56:26.145 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:21 smithi122 ceph-mon[32546]: pgmap v972: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T18:56:26.153 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:23 smithi122 ceph-mon[32546]: pgmap v973: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:56:26.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:25 smithi122 ceph-mon[32546]: pgmap v974: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:56:26.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:16 smithi106 ceph-mon[32716]: osdmap e148: 12 total, 12 up, 12 in 2024-04-23T18:56:26.325 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6834/395009281,v1:172.21.15.33:6835/395009281] up:boot 2024-04-23T18:56:26.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:56:26.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:56:26.454 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:56:26.461 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:17 smithi106 ceph-mon[32716]: pgmap v970: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-23T18:56:26.577 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:19 smithi106 ceph-mon[32716]: pgmap v971: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-04-23T18:56:26.733 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:21 smithi106 ceph-mon[32716]: pgmap v972: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T18:56:26.808 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:23 smithi106 ceph-mon[32716]: pgmap v973: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:56:26.816 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:25 smithi106 ceph-mon[32716]: pgmap v974: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:56:26.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 -- 172.21.15.33:0/3146897090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab4106170 msgr2=0x7f1ab410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:26.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 --2- 172.21.15.33:0/3146897090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab4106170 0x7f1ab410dd40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ca0 tx=0x7f1aac02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:56:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 -- 172.21.15.33:0/3146897090 shutdown_connections 2024-04-23T18:56:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 --2- 172.21.15.33:0/3146897090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab4106170 0x7f1ab410dd40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 --2- 172.21.15.33:0/3146897090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab4101a50 0x7f1ab4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 --2- 172.21.15.33:0/3146897090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab41010a0 0x7f1ab4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 -- 172.21.15.33:0/3146897090 >> 172.21.15.33:0/3146897090 conn(0x7f1ab4077f50 msgr2=0x7f1ab4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:27.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 -- 172.21.15.33:0/3146897090 shutdown_connections 2024-04-23T18:56:27.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.006+0000 7f1abc443640 1 -- 172.21.15.33:0/3146897090 wait complete. 2024-04-23T18:56:27.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 Processor -- start 2024-04-23T18:56:27.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 -- start start 2024-04-23T18:56:27.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab4106170 0x7f1ab4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab419bee0 0x7f1ab419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:27.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41110c0 con 0x7f1ab41010a0 2024-04-23T18:56:27.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4110f40 con 0x7f1ab419bee0 2024-04-23T18:56:27.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1abc443640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4111240 con 0x7f1ab4106170 2024-04-23T18:56:27.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1aba1df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1aba1df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49440/0 (socket says 172.21.15.33:49440) 2024-04-23T18:56:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1aba1df640 1 -- 172.21.15.33:0/2871715025 learned_addr learned my addr 172.21.15.33:0/2871715025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:27.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1ab99de640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab4106170 0x7f1ab4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.007+0000 7f1aba9e0640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab419bee0 0x7f1ab419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:27.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 -- 172.21.15.33:0/2871715025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab4106170 msgr2=0x7f1ab4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab4106170 0x7f1ab4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:27.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 -- 172.21.15.33:0/2871715025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab419bee0 msgr2=0x7f1ab419c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:27.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab419bee0 0x7f1ab419c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:28.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 -- 172.21.15.33:0/2871715025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab419c9d0 con 0x7f1ab41010a0 2024-04-23T18:56:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aba1df640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9f0 tx=0x7f1ab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000cd80 con 0x7f1ab41010a0 2024-04-23T18:56:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab00108e0 con 0x7f1ab41010a0 2024-04-23T18:56:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab419ccc0 con 0x7f1ab41010a0 2024-04-23T18:56:28.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0018980 con 0x7f1ab41010a0 2024-04-23T18:56:28.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.008+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a44b0 con 0x7f1ab41010a0 2024-04-23T18:56:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.009+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab4068d70 con 0x7f1ab41010a0 2024-04-23T18:56:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.010+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ab0010430 con 0x7f1ab41010a0 2024-04-23T18:56:28.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.011+0000 7f1aab7fe640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 0x7f1aa00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:28.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.011+0000 7f1ab99de640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 0x7f1aa00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.011+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 8586+0+0 (secure 0 0 0) 0x7f1ab00d1080 con 0x7f1ab41010a0 2024-04-23T18:56:28.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.011+0000 7f1ab99de640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 0x7f1aa00a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4100970 tx=0x7f1aa400b410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:28.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.019+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ab009d030 con 0x7f1ab41010a0 2024-04-23T18:56:28.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.247+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ab4198970 con 0x7f1ab41010a0 2024-04-23T18:56:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.248+0000 7f1aab7fe640 1 -- 172.21.15.33:0/2871715025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+10277 (secure 0 0 0) 0x7f1ab00a1330 con 0x7f1ab41010a0 2024-04-23T18:56:28.803 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:56:28.876 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:54:15.143821+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_18390":{"gid":18390,"name":"j","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/2623838743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2623838743},{"type":"v1","addr":"172.21.15.33:6835","nonce":2623838743}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24490,24551,24482,24515,24466]},"id":1}]} 2024-04-23T18:56:28.936 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:27 smithi106 ceph-mon[32716]: pgmap v975: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 374 B/s wr, 14 op/s 2024-04-23T18:56:28.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:27 smithi122 ceph-mon[32546]: pgmap v975: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 374 B/s wr, 14 op/s 2024-04-23T18:56:29.052 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 47 2024-04-23T18:56:29.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 msgr2=0x7f1aa00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:29.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 0x7f1aa00a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4100970 tx=0x7f1aa400b410 comp rx=0 tx=0).stop 2024-04-23T18:56:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 msgr2=0x7f1ab4197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:29.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000e9f0 tx=0x7f1ab000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 shutdown_connections 2024-04-23T18:56:29.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f1aa00a3d20 0x7f1aa00a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1ab419bee0 0x7f1ab419c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ab4106170 0x7f1ab4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 --2- 172.21.15.33:0/2871715025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ab41010a0 0x7f1ab4197620 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 >> 172.21.15.33:0/2871715025 conn(0x7f1ab4077f50 msgr2=0x7f1ab40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 shutdown_connections 2024-04-23T18:56:29.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:55:11.250+0000 7f1abc443640 1 -- 172.21.15.33:0/2871715025 wait complete. 2024-04-23T18:56:29.877 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24551 quiesce db --cancel --all' 2024-04-23T18:56:29.943 DEBUG:teuthology.orchestra.run.smithi033:> timeout 75 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24551 quiesce db --cancel --all 2024-04-23T18:56:29.997 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:11 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:56:30.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:12 smithi033 ceph-mon[29274]: pgmap v930: 97 pgs: 97 active+clean; 15 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:56:30.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:12 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2871715025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T18:56:30.074 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:12 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:56:30.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:12 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:56:30.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:12 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:56:30.198 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:13 smithi033 ceph-mon[29274]: pgmap v931: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 61 KiB/s wr, 35 op/s 2024-04-23T18:56:30.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:13 smithi033 ceph-mon[29274]: pgmap v932: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 79 KiB/s wr, 46 op/s 2024-04-23T18:56:30.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:15 smithi033 ceph-mon[29274]: pgmap v933: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-23T18:56:30.334 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:17 smithi033 ceph-mon[29274]: pgmap v934: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-23T18:56:30.375 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:17 smithi033 ceph-mon[29274]: osdmap e145: 12 total, 12 up, 12 in 2024-04-23T18:56:30.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:19 smithi033 ceph-mon[29274]: pgmap v936: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.2 KiB/s wr, 16 op/s 2024-04-23T18:56:30.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:21 smithi033 ceph-mon[29274]: pgmap v937: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-23T18:56:30.576 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:29 smithi106 ceph-mon[32716]: pgmap v976: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:56:30.635 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:29 smithi122 ceph-mon[32546]: pgmap v976: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:56:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- 172.21.15.33:0/356472428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 msgr2=0x7f702810e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 --2- 172.21.15.33:0/356472428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f702810e2d0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7018009900 tx=0x7f701802a480 comp rx=0 tx=0).stop 2024-04-23T18:56:30.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- 172.21.15.33:0/356472428 shutdown_connections 2024-04-23T18:56:30.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 --2- 172.21.15.33:0/356472428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f702810e810 0x7f7028110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:30.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 --2- 172.21.15.33:0/356472428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f702810e2d0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:30.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 --2- 172.21.15.33:0/356472428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7028074300 0x7f70280746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- 172.21.15.33:0/356472428 >> 172.21.15.33:0/356472428 conn(0x7f70280fda20 msgr2=0x7f70280ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- 172.21.15.33:0/356472428 shutdown_connections 2024-04-23T18:56:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- 172.21.15.33:0/356472428 wait complete. 2024-04-23T18:56:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 Processor -- start 2024-04-23T18:56:30.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.981+0000 7f70305cd640 1 -- start start 2024-04-23T18:56:31.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028074300 0x7f70281a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:31.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f702810e810 0x7f702819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:31.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7028114400 con 0x7f7028074c20 2024-04-23T18:56:31.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7028114280 con 0x7f7028074300 2024-04-23T18:56:31.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f70305cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7028114580 con 0x7f702810e810 2024-04-23T18:56:31.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:31.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51386/0 (socket says 172.21.15.33:51386) 2024-04-23T18:56:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 -- 172.21.15.33:0/502750142 learned_addr learned my addr 172.21.15.33:0/502750142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702e369640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028074300 0x7f70281a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702eb6a640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f702810e810 0x7f702819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 -- 172.21.15.33:0/502750142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f702810e810 msgr2=0x7f702819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f702810e810 0x7f702819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 -- 172.21.15.33:0/502750142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028074300 msgr2=0x7f70281a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028074300 0x7f70281a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 -- 172.21.15.33:0/502750142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702819afe0 con 0x7f7028074c20 2024-04-23T18:56:31.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.982+0000 7f702db68640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:31.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.983+0000 7f701f7fe640 1 -- 172.21.15.33:0/502750142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701803f070 con 0x7f7028074c20 2024-04-23T18:56:31.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.983+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702819b270 con 0x7f7028074c20 2024-04-23T18:56:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.983+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70281a7590 con 0x7f7028074c20 2024-04-23T18:56:31.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.983+0000 7f701f7fe640 1 -- 172.21.15.33:0/502750142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7018031d60 con 0x7f7028074c20 2024-04-23T18:56:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.983+0000 7f701f7fe640 1 -- 172.21.15.33:0/502750142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018049060 con 0x7f7028074c20 2024-04-23T18:56:32.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.985+0000 7f701f7fe640 1 -- 172.21.15.33:0/502750142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7018042070 con 0x7f7028074c20 2024-04-23T18:56:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 Processor -- start 2024-04-23T18:56:32.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 -- start start 2024-04-23T18:56:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80a620 0x555ebd80aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 0x555ebd8152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd817bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ebd76f970 con 0x555ebd815800 2024-04-23T18:56:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555ebd70c4a0 con 0x555ebd80af40 2024-04-23T18:56:32.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.986+0000 7f70333be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555ebd76b930 con 0x555ebd80a620 2024-04-23T18:56:32.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702db68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd817bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:32.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f701f7fe640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 0x7f70140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702db68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd817bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51402/0 (socket says 172.21.15.33:51402) 2024-04-23T18:56:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702db68640 1 -- 172.21.15.33:0/636017581 learned_addr learned my addr 172.21.15.33:0/636017581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702eb6a640 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80a620 0x555ebd80aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 0x555ebd8152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702eb6a640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 0x7f70140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:32.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f701f7fe640 1 -- 172.21.15.33:0/502750142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f70180f0830 con 0x7f7028074c20 2024-04-23T18:56:32.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 -- 172.21.15.33:0/636017581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80a620 msgr2=0x555ebd80aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80a620 0x555ebd80aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 -- 172.21.15.33:0/636017581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 msgr2=0x555ebd817bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd817bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 -- 172.21.15.33:0/636017581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ebd482210 con 0x555ebd80af40 2024-04-23T18:56:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702eb6a640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 0x7f70140a6280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7020005fd0 tx=0x7f7020009500 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.987+0000 7f702e369640 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 0x555ebd8152c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f70240107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec97d45415a90e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f701cbfe640 1 -- 172.21.15.33:0/636017581 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702401c480 con 0x555ebd80af40 2024-04-23T18:56:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f701cbfe640 1 -- 172.21.15.33:0/636017581 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702401cad0 con 0x555ebd80af40 2024-04-23T18:56:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f701cbfe640 1 -- 172.21.15.33:0/636017581 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024011bc0 con 0x555ebd80af40 2024-04-23T18:56:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 -- 172.21.15.33:0/636017581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 msgr2=0x555ebd8152c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 0x555ebd8152c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f70240107a0 comp rx=0 tx=0).stop 2024-04-23T18:56:33.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 -- 172.21.15.33:0/636017581 shutdown_connections 2024-04-23T18:56:33.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd817bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80af40 0x555ebd8152c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:33.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 --2- 172.21.15.33:0/636017581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80a620 0x555ebd80aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:33.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 -- 172.21.15.33:0/636017581 >> 172.21.15.33:0/636017581 conn(0x555ebd806b30 msgr2=0x555ebd806f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 -- 172.21.15.33:0/636017581 shutdown_connections 2024-04-23T18:56:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.988+0000 7f70333be740 1 -- 172.21.15.33:0/636017581 wait complete. 2024-04-23T18:56:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 -- start start 2024-04-23T18:56:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 Processor -- start 2024-04-23T18:56:33.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd815060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd812350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555ebd818430 con 0x555ebd815800 2024-04-23T18:56:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555ebd70a0b0 con 0x555ebd80a620 2024-04-23T18:56:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555ebd8185b0 con 0x555ebd80af40 2024-04-23T18:56:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f70333be740 10 client.0 _ll_register_callbacks cb 0x555ebd6f7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:34.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f702eb6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f702eb6a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51406/0 (socket says 172.21.15.33:51406) 2024-04-23T18:56:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f702eb6a640 1 -- 172.21.15.33:0/4179850755 learned_addr learned my addr 172.21.15.33:0/4179850755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f702e369640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd815060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:34.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.989+0000 7f702db68640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd812350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:34.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 -- 172.21.15.33:0/4179850755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 msgr2=0x555ebd812350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd812350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:34.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 -- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 msgr2=0x555ebd815060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:34.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd815060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 -- 172.21.15.33:0/4179850755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ebd54e620 con 0x555ebd815800 2024-04-23T18:56:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702db68640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd812350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f702eb6a640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f702000ff20 tx=0x7f7020004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a0a093509b46a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f7011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:34.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702002e060 con 0x555ebd815800 2024-04-23T18:56:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f7020021cc0 con 0x555ebd815800 2024-04-23T18:56:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702401cad0 con 0x555ebd815800 2024-04-23T18:56:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7020010430 con 0x555ebd815800 2024-04-23T18:56:34.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f70333be740 10 client.18498 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:35.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555ebd824320 con 0x555ebd815800 2024-04-23T18:56:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.990+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702401cad0 con 0x555ebd815800 2024-04-23T18:56:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f702001d070 con 0x555ebd815800 2024-04-23T18:56:35.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 10 client.18498 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 10 client.18498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 20 client.18498 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:35.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd825990 0x555ebd827e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 4 client.18498 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:56:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ebd77d390 con 0x555ebd825990 2024-04-23T18:56:35.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f702e369640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd825990 0x555ebd827e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:35.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f702e369640 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd825990 0x555ebd827e10 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c0dce958cf8bd server_cookie=3cc1e06cc5e36821 in_seq=0 out_seq=0 2024-04-23T18:56:35.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.991+0000 7f7011ffb640 10 client.18498 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f7011ffb640 1 -- 172.21.15.33:0/4179850755 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ebd77d390 con 0x555ebd825990 2024-04-23T18:56:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f7011ffb640 10 client.18498 handle_command_reply: tid=0 2024-04-23T18:56:35.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 client.18498 shutdown 2024-04-23T18:56:35.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 msgr2=0x555ebd812890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:35.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f702000ff20 tx=0x7f7020004a40 comp rx=0 tx=0).stop 2024-04-23T18:56:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 shutdown_connections 2024-04-23T18:56:35.778 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd825990 0x555ebd827e10 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd815800 0x555ebd812890 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd812350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:35.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 --2- 172.21.15.33:0/4179850755 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd815060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f7011ffb640 0 client.18498 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 >> 172.21.15.33:0/4179850755 conn(0x555ebd806b30 msgr2=0x555ebd80c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 shutdown_connections 2024-04-23T18:56:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.992+0000 7f70333be740 1 -- 172.21.15.33:0/4179850755 wait complete. 2024-04-23T18:56:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:21.993+0000 7f70333be740 20 client.18498 trim_cache size 0 max 16384 2024-04-23T18:56:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 Processor -- start 2024-04-23T18:56:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 -- start start 2024-04-23T18:56:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd805b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd82d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:36.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00d270 con 0x555ebd806b30 2024-04-23T18:56:36.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00fb50 con 0x555ebd80a620 2024-04-23T18:56:36.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f70333be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00d940 con 0x555ebd80af40 2024-04-23T18:56:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51410/0 (socket says 172.21.15.33:51410) 2024-04-23T18:56:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 -- 172.21.15.33:0/2173777374 learned_addr learned my addr 172.21.15.33:0/2173777374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702db68640 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd805b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 -- 172.21.15.33:0/2173777374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 msgr2=0x555ebd82d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd82d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 -- 172.21.15.33:0/2173777374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 msgr2=0x555ebd805b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd805b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:36.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.008+0000 7f702e369640 1 -- 172.21.15.33:0/2173777374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ebd817f10 con 0x555ebd806b30 2024-04-23T18:56:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f702e369640 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f70240116f0 tx=0x7f702401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19cf3f8883a68e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f7013fff640 1 -- 172.21.15.33:0/2173777374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024030070 con 0x555ebd806b30 2024-04-23T18:56:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f7013fff640 1 -- 172.21.15.33:0/2173777374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f702401cdc0 con 0x555ebd806b30 2024-04-23T18:56:37.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f7013fff640 1 -- 172.21.15.33:0/2173777374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702401ca90 con 0x555ebd806b30 2024-04-23T18:56:37.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 -- 172.21.15.33:0/2173777374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 msgr2=0x555ebd821320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f70240116f0 tx=0x7f702401e040 comp rx=0 tx=0).stop 2024-04-23T18:56:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 -- 172.21.15.33:0/2173777374 shutdown_connections 2024-04-23T18:56:37.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd82d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:37.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd805b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 --2- 172.21.15.33:0/2173777374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd821320 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 -- 172.21.15.33:0/2173777374 >> 172.21.15.33:0/2173777374 conn(0x555ebd8129f0 msgr2=0x555ebd805420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 -- 172.21.15.33:0/2173777374 shutdown_connections 2024-04-23T18:56:37.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.009+0000 7f70333be740 1 -- 172.21.15.33:0/2173777374 wait complete. 2024-04-23T18:56:37.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 -- start start 2024-04-23T18:56:37.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 Processor -- start 2024-04-23T18:56:37.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd81b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:37.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd81b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:37.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00e010 con 0x555ebd806b30 2024-04-23T18:56:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00f480 con 0x555ebd80a620 2024-04-23T18:56:37.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f700c00e6e0 con 0x555ebd80af40 2024-04-23T18:56:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f70333be740 10 client.0 _ll_register_callbacks cb 0x7f7020009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:37.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51416/0 (socket says 172.21.15.33:51416) 2024-04-23T18:56:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 -- 172.21.15.33:0/1037382212 learned_addr learned my addr 172.21.15.33:0/1037382212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702db68640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd81b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:37.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702eb6a640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd81b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:38.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 -- 172.21.15.33:0/1037382212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 msgr2=0x555ebd81b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd81b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 -- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 msgr2=0x555ebd81b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:38.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd81b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:38.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702e369640 1 -- 172.21.15.33:0/1037382212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ebd88afa0 con 0x555ebd806b30 2024-04-23T18:56:38.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.010+0000 7f702db68640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd81b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:38.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f702e369640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f702400ba20 tx=0x7f702401c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbf54c46db28032 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702403a060 con 0x555ebd806b30 2024-04-23T18:56:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f702401c910 con 0x555ebd806b30 2024-04-23T18:56:38.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ebd777200 con 0x555ebd806b30 2024-04-23T18:56:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f7024030070 con 0x555ebd806b30 2024-04-23T18:56:38.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 10 client.18504 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:38.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x555ebd88c150 con 0x555ebd806b30 2024-04-23T18:56:38.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555ebd777200 con 0x555ebd806b30 2024-04-23T18:56:38.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f7024011a20 con 0x555ebd806b30 2024-04-23T18:56:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 10 client.18504 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 10 client.18504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 20 client.18504 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd815800 0x555ebd88e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 4 client.18504 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:56:38.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.011+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ebd6f35f0 con 0x555ebd815800 2024-04-23T18:56:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.012+0000 7f702db68640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd815800 0x555ebd88e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:39.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.012+0000 7f702db68640 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd815800 0x555ebd88e740 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfc1d1ab9c8bd5b server_cookie=9fabbcc609409435 in_seq=0 out_seq=0 2024-04-23T18:56:39.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.012+0000 7f7011ffb640 10 client.18504 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.016+0000 7f7011ffb640 1 -- 172.21.15.33:0/1037382212 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555ebd6f35f0 con 0x555ebd815800 2024-04-23T18:56:39.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.016+0000 7f7011ffb640 10 client.18504 handle_command_reply: tid=0 2024-04-23T18:56:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.016+0000 7f70333be740 1 client.18504 shutdown 2024-04-23T18:56:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 msgr2=0x555ebd81dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f702400ba20 tx=0x7f702401c570 comp rx=0 tx=0).stop 2024-04-23T18:56:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 shutdown_connections 2024-04-23T18:56:39.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x555ebd815800 0x555ebd88e740 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ebd80af40 0x555ebd81b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555ebd80a620 0x555ebd81b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 --2- 172.21.15.33:0/1037382212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555ebd806b30 0x555ebd81dd80 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:39.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f7011ffb640 0 client.18504 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 >> 172.21.15.33:0/1037382212 conn(0x555ebd8129f0 msgr2=0x555ebd805cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:39.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 shutdown_connections 2024-04-23T18:56:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 1 -- 172.21.15.33:0/1037382212 wait complete. 2024-04-23T18:56:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70333be740 20 client.18504 trim_cache size 0 max 16384 2024-04-23T18:56:39.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:56:40.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:56:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:56:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:56:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 msgr2=0x7f70140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:40.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 0x7f70140a6280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7020005fd0 tx=0x7f7020009500 comp rx=0 tx=0).stop 2024-04-23T18:56:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 msgr2=0x7f70281a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018003c80 comp rx=0 tx=0).stop 2024-04-23T18:56:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 shutdown_connections 2024-04-23T18:56:40.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f70140a3dc0 0x7f70140a6280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:40.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f702810e810 0x7f702819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7028074c20 0x7f70281a0b50 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 --2- 172.21.15.33:0/502750142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7028074300 0x7f70281a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 >> 172.21.15.33:0/502750142 conn(0x7f70280fda20 msgr2=0x7f70280fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.017+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 shutdown_connections 2024-04-23T18:56:40.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:22.018+0000 7f70305cd640 1 -- 172.21.15.33:0/502750142 wait complete. 2024-04-23T18:56:40.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:23 smithi033 ceph-mon[29274]: pgmap v938: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:56:40.991 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:25 smithi033 ceph-mon[29274]: pgmap v939: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:56:41.049 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:27 smithi033 ceph-mon[29274]: pgmap v940: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:56:41.059 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:29 smithi033 ceph-mon[29274]: pgmap v941: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 268 B/s wr, 9 op/s 2024-04-23T18:56:41.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:31 smithi033 ceph-mon[29274]: pgmap v942: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:41.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:33 smithi033 ceph-mon[29274]: pgmap v943: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:41.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:31 smithi106 ceph-mon[32716]: pgmap v977: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:56:41.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:33 smithi106 ceph-mon[32716]: pgmap v978: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:56:41.254 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:35 smithi106 ceph-mon[32716]: pgmap v979: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:41.263 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:37 smithi106 ceph-mon[32716]: pgmap v980: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:41.271 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:39 smithi106 ceph-mon[32716]: pgmap v981: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:41.286 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:31 smithi122 ceph-mon[32546]: pgmap v977: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:56:41.350 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:33 smithi122 ceph-mon[32546]: pgmap v978: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:56:41.360 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:35 smithi122 ceph-mon[32546]: pgmap v979: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:41.368 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:37 smithi122 ceph-mon[32546]: pgmap v980: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:41.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:39 smithi122 ceph-mon[32546]: pgmap v981: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:41.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 -- 172.21.15.33:0/530521415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff424101a90 msgr2=0x7ff424105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 --2- 172.21.15.33:0/530521415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff424101a90 0x7ff424105ad0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff414009900 tx=0x7ff41402a410 comp rx=0 tx=0).stop 2024-04-23T18:56:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 -- 172.21.15.33:0/530521415 shutdown_connections 2024-04-23T18:56:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 --2- 172.21.15.33:0/530521415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 0x7ff42410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 --2- 172.21.15.33:0/530521415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff424101a90 0x7ff424105ad0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:41.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 --2- 172.21.15.33:0/530521415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4241010e0 0x7ff4241014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:41.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 -- 172.21.15.33:0/530521415 >> 172.21.15.33:0/530521415 conn(0x7ff4240771d0 msgr2=0x7ff4240775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 -- 172.21.15.33:0/530521415 shutdown_connections 2024-04-23T18:56:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.117+0000 7ff42c35d640 1 -- 172.21.15.33:0/530521415 wait complete. 2024-04-23T18:56:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 Processor -- start 2024-04-23T18:56:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 -- start start 2024-04-23T18:56:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 0x7ff4241a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:42.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 0x7ff42419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:42.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241110f0 con 0x7ff4241010e0 2024-04-23T18:56:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff424110f70 con 0x7ff42419a690 2024-04-23T18:56:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42c35d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff424111270 con 0x7ff4241061b0 2024-04-23T18:56:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42a0f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:42.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42a0f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37818/0 (socket says 172.21.15.33:37818) 2024-04-23T18:56:42.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42a0f9640 1 -- 172.21.15.33:0/335837920 learned_addr learned my addr 172.21.15.33:0/335837920 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:42.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff4298f8640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 0x7ff4241a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:42.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.118+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 0x7ff42419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:42.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 -- 172.21.15.33:0/335837920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 msgr2=0x7ff4241a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 0x7ff4241a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:42.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 -- 172.21.15.33:0/335837920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 msgr2=0x7ff42419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:42.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 0x7ff42419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 -- 172.21.15.33:0/335837920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42419b290 con 0x7ff4241010e0 2024-04-23T18:56:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 0x7ff42419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:43.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9f0 tx=0x7ff42000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:43.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff41b7fe640 1 -- 172.21.15.33:0/335837920 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42000cd80 con 0x7ff4241010e0 2024-04-23T18:56:43.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4241a7370 con 0x7ff4241010e0 2024-04-23T18:56:43.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff41b7fe640 1 -- 172.21.15.33:0/335837920 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4200108e0 con 0x7ff4241010e0 2024-04-23T18:56:43.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff41b7fe640 1 -- 172.21.15.33:0/335837920 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420018980 con 0x7ff4241010e0 2024-04-23T18:56:43.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.119+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4241a7770 con 0x7ff4241010e0 2024-04-23T18:56:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.122+0000 7ff41b7fe640 1 -- 172.21.15.33:0/335837920 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff420010430 con 0x7ff4241010e0 2024-04-23T18:56:43.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.122+0000 7ff42f078740 1 Processor -- start 2024-04-23T18:56:43.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 -- start start 2024-04-23T18:56:43.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03506a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:43.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506f40 0x55ab035112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:43.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab0346b970 con 0x55ab03511800 2024-04-23T18:56:43.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ab034084a0 con 0x55ab03506f40 2024-04-23T18:56:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42f078740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ab03467930 con 0x55ab03506620 2024-04-23T18:56:44.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:44.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37826/0 (socket says 172.21.15.33:37826) 2024-04-23T18:56:44.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 -- 172.21.15.33:0/1204223496 learned_addr learned my addr 172.21.15.33:0/1204223496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:44.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506f40 0x55ab035112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:44.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff41b7fe640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:44.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03506a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 -- 172.21.15.33:0/1204223496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 msgr2=0x55ab03506a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:44.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03506a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 -- 172.21.15.33:0/1204223496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506f40 msgr2=0x55ab035112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:44.767 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506f40 0x55ab035112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:44.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.123+0000 7ff4298f8640 1 -- 172.21.15.33:0/1204223496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab0317e210 con 0x55ab03511800 2024-04-23T18:56:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff41b7fe640 1 -- 172.21.15.33:0/335837920 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7ff4200d0cb0 con 0x7ff4241010e0 2024-04-23T18:56:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:45.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff4298f8640 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff414031440 tx=0x7ff414032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0b5d49200604a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:45.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff418bfe640 1 -- 172.21.15.33:0/1204223496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41403f070 con 0x55ab03511800 2024-04-23T18:56:45.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff418bfe640 1 -- 172.21.15.33:0/1204223496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff414004030 con 0x55ab03511800 2024-04-23T18:56:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.124+0000 7ff418bfe640 1 -- 172.21.15.33:0/1204223496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035070 con 0x55ab03511800 2024-04-23T18:56:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- 172.21.15.33:0/1204223496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 msgr2=0x55ab03513bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff414031440 tx=0x7ff414032ba0 comp rx=0 tx=0).stop 2024-04-23T18:56:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 0x7ff4100a6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005fd0 tx=0x7ff41c004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- 172.21.15.33:0/1204223496 shutdown_connections 2024-04-23T18:56:45.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03511800 0x55ab03513bf0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:45.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506f40 0x55ab035112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 --2- 172.21.15.33:0/1204223496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03506a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- 172.21.15.33:0/1204223496 >> 172.21.15.33:0/1204223496 conn(0x55ab03502b30 msgr2=0x55ab03502f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- 172.21.15.33:0/1204223496 shutdown_connections 2024-04-23T18:56:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- 172.21.15.33:0/1204223496 wait complete. 2024-04-23T18:56:45.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 -- start start 2024-04-23T18:56:45.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.125+0000 7ff42f078740 1 Processor -- start 2024-04-23T18:56:45.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 0x55ab03511100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:45.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03511800 0x55ab0350e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab035143d0 con 0x55ab03506f40 2024-04-23T18:56:45.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ab034060b0 con 0x55ab03506620 2024-04-23T18:56:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ab03514550 con 0x55ab03511800 2024-04-23T18:56:46.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 10 client.0 _ll_register_callbacks cb 0x55ab033f3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:46.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37840/0 (socket says 172.21.15.33:37840) 2024-04-23T18:56:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 -- 172.21.15.33:0/335603113 learned_addr learned my addr 172.21.15.33:0/335603113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:46.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 0x55ab03511100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:46.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03511800 0x55ab0350e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:46.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 -- 172.21.15.33:0/335603113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03511800 msgr2=0x55ab0350e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03511800 0x55ab0350e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 -- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 msgr2=0x55ab03511100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 0x55ab03511100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 -- 172.21.15.33:0/335603113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab0324a620 con 0x55ab03506f40 2024-04-23T18:56:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 0x55ab03511100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff4298f8640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff414005c40 tx=0x7ff414005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b822d96e8f592f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:46.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035070 con 0x55ab03506f40 2024-04-23T18:56:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7ff4140316a0 con 0x55ab03506f40 2024-04-23T18:56:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.126+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414004030 con 0x55ab03506f40 2024-04-23T18:56:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff41403f040 con 0x55ab03506f40 2024-04-23T18:56:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 10 client.18516 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ab035202e0 con 0x55ab03506f40 2024-04-23T18:56:46.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff414004030 con 0x55ab03506f40 2024-04-23T18:56:46.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7ff414003660 con 0x55ab03506f40 2024-04-23T18:56:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 10 client.18516 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:46.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 10 client.18516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 20 client.18516 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab03521630 0x55ab03523a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 4 client.18516 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:56:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.127+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab03479390 con 0x55ab03521630 2024-04-23T18:56:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.128+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab03521630 0x55ab03523a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.128+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab03521630 0x55ab03523a20 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9190c41e28366b87 server_cookie=38e5858665ad2a0a in_seq=0 out_seq=0 2024-04-23T18:56:47.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.128+0000 7ff40dffb640 10 client.18516 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:47.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff40dffb640 1 -- 172.21.15.33:0/335603113 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab03479390 con 0x55ab03521630 2024-04-23T18:56:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff40dffb640 10 client.18516 handle_command_reply: tid=0 2024-04-23T18:56:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 client.18516 shutdown 2024-04-23T18:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 msgr2=0x55ab0350e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff414005c40 tx=0x7ff414005280 comp rx=0 tx=0).stop 2024-04-23T18:56:47.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 shutdown_connections 2024-04-23T18:56:47.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab03521630 0x55ab03523a20 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:47.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03511800 0x55ab0350e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff40dffb640 0 client.18516 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:47.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab0350e3f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 --2- 172.21.15.33:0/335603113 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03506620 0x55ab03511100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 >> 172.21.15.33:0/335603113 conn(0x55ab03502b30 msgr2=0x55ab03508860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 shutdown_connections 2024-04-23T18:56:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 1 -- 172.21.15.33:0/335603113 wait complete. 2024-04-23T18:56:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.129+0000 7ff42f078740 20 client.18516 trim_cache size 0 max 16384 2024-04-23T18:56:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 Processor -- start 2024-04-23T18:56:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 -- start start 2024-04-23T18:56:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab0351d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:48.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:48.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab035291b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241ae870 con 0x55ab03506f40 2024-04-23T18:56:48.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b1150 con 0x55ab03511800 2024-04-23T18:56:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff42f078740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241aef40 con 0x55ab03506620 2024-04-23T18:56:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:48.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37846/0 (socket says 172.21.15.33:37846) 2024-04-23T18:56:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.144+0000 7ff4298f8640 1 -- 172.21.15.33:0/3280438577 learned_addr learned my addr 172.21.15.33:0/3280438577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab035291b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab0351d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 -- 172.21.15.33:0/3280438577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 msgr2=0x55ab0351d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab0351d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:48.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 -- 172.21.15.33:0/3280438577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 msgr2=0x55ab035291b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:48.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab035291b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:48.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 -- 172.21.15.33:0/3280438577 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab03506d70 con 0x55ab03506f40 2024-04-23T18:56:49.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab0351d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:49.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff414003c20 tx=0x7ff414009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f65de60143ebfca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff40ffff640 1 -- 172.21.15.33:0/3280438577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414035070 con 0x55ab03506f40 2024-04-23T18:56:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff40ffff640 1 -- 172.21.15.33:0/3280438577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4140037f0 con 0x55ab03506f40 2024-04-23T18:56:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.145+0000 7ff40ffff640 1 -- 172.21.15.33:0/3280438577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414004e50 con 0x55ab03506f40 2024-04-23T18:56:49.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- 172.21.15.33:0/3280438577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 msgr2=0x55ab03501b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff414003c20 tx=0x7ff414009300 comp rx=0 tx=0).stop 2024-04-23T18:56:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- 172.21.15.33:0/3280438577 shutdown_connections 2024-04-23T18:56:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab035291b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03501b30 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:49.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- 172.21.15.33:0/3280438577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab0351d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- 172.21.15.33:0/3280438577 >> 172.21.15.33:0/3280438577 conn(0x55ab03502b30 msgr2=0x55ab03501420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- 172.21.15.33:0/3280438577 shutdown_connections 2024-04-23T18:56:49.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- 172.21.15.33:0/3280438577 wait complete. 2024-04-23T18:56:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- start start 2024-04-23T18:56:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 Processor -- start 2024-04-23T18:56:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03519aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:50.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab03517370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241af610 con 0x55ab03506f40 2024-04-23T18:56:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241b0a80 con 0x55ab03511800 2024-04-23T18:56:50.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.146+0000 7ff42f078740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4241afce0 con 0x55ab03506620 2024-04-23T18:56:50.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42f078740 10 client.0 _ll_register_callbacks cb 0x7ff4140045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:50.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37850/0 (socket says 172.21.15.33:37850) 2024-04-23T18:56:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 -- 172.21.15.33:0/3395217788 learned_addr learned my addr 172.21.15.33:0/3395217788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:50.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03519aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab03517370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 -- 172.21.15.33:0/3395217788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 msgr2=0x55ab03519aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03519aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 -- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 msgr2=0x55ab03517370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab03517370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 -- 172.21.15.33:0/3395217788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab03517b70 con 0x55ab03506f40 2024-04-23T18:56:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42a8fa640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab03517370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:56:51.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff4298f8640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ff414003ab0 tx=0x7ff41402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a3b9ab335988ac server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:51.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff40dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:51.216 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414059070 con 0x55ab03506f40 2024-04-23T18:56:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7ff414004de0 con 0x55ab03506f40 2024-04-23T18:56:51.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4140037f0 con 0x55ab03506f40 2024-04-23T18:56:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff414065050 con 0x55ab03506f40 2024-04-23T18:56:51.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42f078740 10 client.18522 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.147+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ab03587920 con 0x55ab03506f40 2024-04-23T18:56:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4140037f0 con 0x55ab03506f40 2024-04-23T18:56:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7ff414054bf0 con 0x55ab03506f40 2024-04-23T18:56:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 10 client.18522 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:51.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 10 client.18522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:51.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 20 client.18522 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab035880c0 0x55ab0358a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 4 client.18522 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:56:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab0350ecc0 con 0x55ab035880c0 2024-04-23T18:56:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab035880c0 0x55ab0358a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff42a0f9640 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab035880c0 0x55ab0358a4e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a5102df8fe38fc0 server_cookie=df1f82f41f226d89 in_seq=0 out_seq=0 2024-04-23T18:56:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.148+0000 7ff40dffb640 10 client.18522 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:52.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff40dffb640 1 -- 172.21.15.33:0/3395217788 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab0350ecc0 con 0x55ab035880c0 2024-04-23T18:56:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff40dffb640 10 client.18522 handle_command_reply: tid=0 2024-04-23T18:56:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 client.18522 shutdown 2024-04-23T18:56:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 msgr2=0x55ab03516e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ff414003ab0 tx=0x7ff41402a410 comp rx=0 tx=0).stop 2024-04-23T18:56:52.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 shutdown_connections 2024-04-23T18:56:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ab035880c0 0x55ab0358a4e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab03511800 0x55ab03517370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:52.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab03506f40 0x55ab03516e30 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff40dffb640 0 client.18522 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:56:52.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 --2- 172.21.15.33:0/3395217788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ab03506620 0x55ab03519aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 >> 172.21.15.33:0/3395217788 conn(0x55ab03502b30 msgr2=0x55ab03501cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 shutdown_connections 2024-04-23T18:56:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 1 -- 172.21.15.33:0/3395217788 wait complete. 2024-04-23T18:56:52.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42f078740 20 client.18522 trim_cache size 0 max 16384 2024-04-23T18:56:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:56:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:56:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:56:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:56:52.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 msgr2=0x7ff4100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 0x7ff4100a6280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005fd0 tx=0x7ff41c004a40 comp rx=0 tx=0).stop 2024-04-23T18:56:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 msgr2=0x7ff4241a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.153+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7ff42000e9f0 tx=0x7ff42000eeb0 comp rx=0 tx=0).stop 2024-04-23T18:56:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 shutdown_connections 2024-04-23T18:56:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff4100a3dc0 0x7ff4100a6280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff42419a690 0x7ff42419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4241061b0 0x7ff4241a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 --2- 172.21.15.33:0/335837920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4241010e0 0x7ff4241a05d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 >> 172.21.15.33:0/335837920 conn(0x7ff4240771d0 msgr2=0x7ff4240ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 shutdown_connections 2024-04-23T18:56:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:35.154+0000 7ff42c35d640 1 -- 172.21.15.33:0/335837920 wait complete. 2024-04-23T18:56:53.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:35 smithi033 ceph-mon[29274]: pgmap v944: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.601 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:37 smithi033 ceph-mon[29274]: pgmap v945: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:39 smithi033 ceph-mon[29274]: pgmap v946: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:56:53.648 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:41 smithi033 ceph-mon[29274]: pgmap v947: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.737 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:43 smithi033 ceph-mon[29274]: pgmap v948: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:53.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:45 smithi033 ceph-mon[29274]: pgmap v949: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:47 smithi033 ceph-mon[29274]: pgmap v950: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.935 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:41 smithi106 ceph-mon[32716]: pgmap v982: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:53.943 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:43 smithi106 ceph-mon[32716]: pgmap v983: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.050 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:45 smithi106 ceph-mon[32716]: pgmap v984: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:54.064 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:47 smithi106 ceph-mon[32716]: pgmap v985: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.077 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:49 smithi106 ceph-mon[32716]: pgmap v986: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:54.139 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:51 smithi106 ceph-mon[32716]: pgmap v987: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:54.261 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:53 smithi106 ceph-mon[32716]: pgmap v988: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.393 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:41 smithi122 ceph-mon[32546]: pgmap v982: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:54.407 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:43 smithi122 ceph-mon[32546]: pgmap v983: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.420 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:45 smithi122 ceph-mon[32546]: pgmap v984: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:56:54.429 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:47 smithi122 ceph-mon[32546]: pgmap v985: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.437 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:49 smithi122 ceph-mon[32546]: pgmap v986: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:54.505 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:51 smithi122 ceph-mon[32546]: pgmap v987: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:56:54.587 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:53 smithi122 ceph-mon[32546]: pgmap v988: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:56:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 -- 172.21.15.33:0/3053258390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37881021c0 msgr2=0x7f37881025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 --2- 172.21.15.33:0/3053258390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37881021c0 0x7f37881025a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f3780009900 tx=0x7f378002a410 comp rx=0 tx=0).stop 2024-04-23T18:56:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 -- 172.21.15.33:0/3053258390 shutdown_connections 2024-04-23T18:56:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 --2- 172.21.15.33:0/3053258390 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3788107290 0x7f378810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 --2- 172.21.15.33:0/3053258390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3788102b70 0x7f3788106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 --2- 172.21.15.33:0/3053258390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37881021c0 0x7f37881025a0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 -- 172.21.15.33:0/3053258390 >> 172.21.15.33:0/3053258390 conn(0x7f37880fda20 msgr2=0x7f37880ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 -- 172.21.15.33:0/3053258390 shutdown_connections 2024-04-23T18:56:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.253+0000 7f378c86b640 1 -- 172.21.15.33:0/3053258390 wait complete. 2024-04-23T18:56:55.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 Processor -- start 2024-04-23T18:56:55.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 -- start start 2024-04-23T18:56:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37881021c0 0x7f3788198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378819cfa0 0x7f378819d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37881121f0 con 0x7f3788107290 2024-04-23T18:56:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3788112070 con 0x7f37881021c0 2024-04-23T18:56:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378c86b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3788112370 con 0x7f378819cfa0 2024-04-23T18:56:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f3785d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:55.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f3785d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42726/0 (socket says 172.21.15.33:42726) 2024-04-23T18:56:55.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f3785d9b640 1 -- 172.21.15.33:0/400631518 learned_addr learned my addr 172.21.15.33:0/400631518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f3786d9d640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378819cfa0 0x7f378819d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.254+0000 7f378659c640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37881021c0 0x7f3788198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 -- 172.21.15.33:0/400631518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378819cfa0 msgr2=0x7f378819d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:55.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378819cfa0 0x7f378819d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:55.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 -- 172.21.15.33:0/400631518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37881021c0 msgr2=0x7f3788198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37881021c0 0x7f3788198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 -- 172.21.15.33:0/400631518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378819da90 con 0x7f3788107290 2024-04-23T18:56:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f3785d9b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f377000c970 tx=0x7f377000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:56.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f377f7fe640 1 -- 172.21.15.33:0/400631518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770007bb0 con 0x7f3788107290 2024-04-23T18:56:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378819dd80 con 0x7f3788107290 2024-04-23T18:56:56.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f377f7fe640 1 -- 172.21.15.33:0/400631518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3770007d50 con 0x7f3788107290 2024-04-23T18:56:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f377f7fe640 1 -- 172.21.15.33:0/400631518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770018730 con 0x7f3788107290 2024-04-23T18:56:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.255+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881a5520 con 0x7f3788107290 2024-04-23T18:56:56.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 Processor -- start 2024-04-23T18:56:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f377f7fe640 1 -- 172.21.15.33:0/400631518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f377000f990 con 0x7f3788107290 2024-04-23T18:56:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 -- start start 2024-04-23T18:56:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a38eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:56.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38f8800 0x55c6a38fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a3852970 con 0x55c6a38edf40 2024-04-23T18:56:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a37ef4a0 con 0x55c6a38f8800 2024-04-23T18:56:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378f57f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a384e930 con 0x55c6a38ed620 2024-04-23T18:56:56.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:56.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42742/0 (socket says 172.21.15.33:42742) 2024-04-23T18:56:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f378659c640 1 -- 172.21.15.33:0/2599832935 learned_addr learned my addr 172.21.15.33:0/2599832935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:56.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f3786d9d640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a38eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:57.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.258+0000 7f377f7fe640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 0x7f37740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f3785d9b640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38f8800 0x55c6a38fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:57.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 -- 172.21.15.33:0/2599832935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 msgr2=0x55c6a38eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a38eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:57.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 -- 172.21.15.33:0/2599832935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38f8800 msgr2=0x55c6a38fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38f8800 0x55c6a38fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 -- 172.21.15.33:0/2599832935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a3565210 con 0x55c6a38edf40 2024-04-23T18:56:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f3786d9d640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a38eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:56:57.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f3786d9d640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 0x7f37740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:57.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f377f7fe640 1 -- 172.21.15.33:0/400631518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f37700d07c0 con 0x7f3788107290 2024-04-23T18:56:57.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f378659c640 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3780009900 tx=0x7f3780032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf78bf5183096f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:57.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f377cff9640 1 -- 172.21.15.33:0/2599832935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780040070 con 0x55c6a38edf40 2024-04-23T18:56:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f377cff9640 1 -- 172.21.15.33:0/2599832935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37800316a0 con 0x55c6a38edf40 2024-04-23T18:56:57.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.259+0000 7f377cff9640 1 -- 172.21.15.33:0/2599832935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780035040 con 0x55c6a38edf40 2024-04-23T18:56:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- 172.21.15.33:0/2599832935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 msgr2=0x55c6a38f82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:56:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3780009900 tx=0x7f3780032e60 comp rx=0 tx=0).stop 2024-04-23T18:56:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f3786d9d640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 0x7f37740a6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3778005fd0 tx=0x7f3778004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:57.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- 172.21.15.33:0/2599832935 shutdown_connections 2024-04-23T18:56:57.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38f8800 0x55c6a38fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:57.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38edf40 0x55c6a38f82c0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 --2- 172.21.15.33:0/2599832935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a38eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- 172.21.15.33:0/2599832935 >> 172.21.15.33:0/2599832935 conn(0x55c6a38e9b30 msgr2=0x55c6a38e9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:56:58.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- 172.21.15.33:0/2599832935 shutdown_connections 2024-04-23T18:56:58.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- 172.21.15.33:0/2599832935 wait complete. 2024-04-23T18:56:58.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 -- start start 2024-04-23T18:56:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.260+0000 7f378f57f740 1 Processor -- start 2024-04-23T18:56:58.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:58.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38f8800 0x55c6a38f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6a38fb3d0 con 0x55c6a38ed620 2024-04-23T18:56:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c6a37ed0b0 con 0x55c6a38edf40 2024-04-23T18:56:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c6a38fb550 con 0x55c6a38f8800 2024-04-23T18:56:58.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378f57f740 10 client.0 _ll_register_callbacks cb 0x55c6a37dad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:56:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42754/0 (socket says 172.21.15.33:42754) 2024-04-23T18:56:58.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 -- 172.21.15.33:0/3133374718 learned_addr learned my addr 172.21.15.33:0/3133374718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:56:58.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f3785d9b640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f3786d9d640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38f8800 0x55c6a38f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:56:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 -- 172.21.15.33:0/3133374718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38f8800 msgr2=0x55c6a38f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38f8800 0x55c6a38f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:59.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 -- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 msgr2=0x55c6a38f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:56:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:56:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 -- 172.21.15.33:0/3133374718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a3631620 con 0x55c6a38ed620 2024-04-23T18:56:59.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f378659c640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3780031a80 tx=0x7f3780031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fea6886ed41177a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:56:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f376dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:56:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.261+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780035070 con 0x55c6a38ed620 2024-04-23T18:56:59.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f3780004230 con 0x55c6a38ed620 2024-04-23T18:56:59.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37800316a0 con 0x55c6a38ed620 2024-04-23T18:56:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3780055050 con 0x55c6a38ed620 2024-04-23T18:56:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 10 client.18534 fetch_fsmap learned FSMap version 47 2024-04-23T18:56:59.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c6a390b6f0 con 0x55c6a38ed620 2024-04-23T18:56:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37800316a0 con 0x55c6a38ed620 2024-04-23T18:56:59.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f378003c5d0 con 0x55c6a38ed620 2024-04-23T18:56:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 10 client.18534 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:56:59.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 10 client.18534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:56:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 20 client.18534 populate_metadata read hostname 'smithi033' 2024-04-23T18:56:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a390ca40 0x55c6a390ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:56:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 4 client.18534 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:56:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.262+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6a3860390 con 0x55c6a390ca40 2024-04-23T18:56:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.263+0000 7f3785d9b640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a390ca40 0x55c6a390ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.263+0000 7f3785d9b640 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a390ca40 0x55c6a390ee30 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4bbdf55a19ab9ea server_cookie=a6a7f60e9f7943d4 in_seq=0 out_seq=0 2024-04-23T18:57:00.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.263+0000 7f376dffb640 10 client.18534 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:00.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f376dffb640 1 -- 172.21.15.33:0/3133374718 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6a3860390 con 0x55c6a390ca40 2024-04-23T18:57:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f376dffb640 10 client.18534 handle_command_reply: tid=0 2024-04-23T18:57:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 client.18534 shutdown 2024-04-23T18:57:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 msgr2=0x55c6a38f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f3780031a80 tx=0x7f3780031400 comp rx=0 tx=0).stop 2024-04-23T18:57:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 shutdown_connections 2024-04-23T18:57:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a390ca40 0x55c6a390ee30 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38f8800 0x55c6a38f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:00.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f376dffb640 0 client.18534 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 --2- 172.21.15.33:0/3133374718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38ed620 0x55c6a38f8140 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 >> 172.21.15.33:0/3133374718 conn(0x55c6a38e9b30 msgr2=0x55c6a38ef860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 shutdown_connections 2024-04-23T18:57:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 1 -- 172.21.15.33:0/3133374718 wait complete. 2024-04-23T18:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.264+0000 7f378f57f740 20 client.18534 trim_cache size 0 max 16384 2024-04-23T18:57:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 Processor -- start 2024-04-23T18:57:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 -- start start 2024-04-23T18:57:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38ed620 0x55c6a38e8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38edf40 0x55c6a38f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:01.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37881aa390 con 0x55c6a38f8800 2024-04-23T18:57:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37881acc70 con 0x55c6a38ed620 2024-04-23T18:57:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f378f57f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37881aaa60 con 0x55c6a38edf40 2024-04-23T18:57:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.279+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42758/0 (socket says 172.21.15.33:42758) 2024-04-23T18:57:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 -- 172.21.15.33:0/611812756 learned_addr learned my addr 172.21.15.33:0/611812756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:01.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f378659c640 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38ed620 0x55c6a38e8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:01.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3785d9b640 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38edf40 0x55c6a38f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:01.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 -- 172.21.15.33:0/611812756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38edf40 msgr2=0x55c6a38f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:01.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38edf40 0x55c6a38f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 -- 172.21.15.33:0/611812756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38ed620 msgr2=0x55c6a38e8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38ed620 0x55c6a38e8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:01.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 -- 172.21.15.33:0/611812756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6a38fb010 con 0x55c6a38f8800 2024-04-23T18:57:01.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f3786d9d640 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f377800e7c0 tx=0x7f3778015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6618dfb2bdaab211 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f376ffff640 1 -- 172.21.15.33:0/611812756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800ed40 con 0x55c6a38f8800 2024-04-23T18:57:01.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f376ffff640 1 -- 172.21.15.33:0/611812756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377801ee00 con 0x55c6a38f8800 2024-04-23T18:57:01.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.280+0000 7f376ffff640 1 -- 172.21.15.33:0/611812756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377801bdf0 con 0x55c6a38f8800 2024-04-23T18:57:01.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- 172.21.15.33:0/611812756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 msgr2=0x55c6a39144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f377800e7c0 tx=0x7f3778015000 comp rx=0 tx=0).stop 2024-04-23T18:57:02.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- 172.21.15.33:0/611812756 shutdown_connections 2024-04-23T18:57:02.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a39144f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38edf40 0x55c6a38f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- 172.21.15.33:0/611812756 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38ed620 0x55c6a38e8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:02.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- 172.21.15.33:0/611812756 >> 172.21.15.33:0/611812756 conn(0x55c6a38e9b30 msgr2=0x55c6a38e8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- 172.21.15.33:0/611812756 shutdown_connections 2024-04-23T18:57:02.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- 172.21.15.33:0/611812756 wait complete. 2024-04-23T18:57:02.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- start start 2024-04-23T18:57:02.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 Processor -- start 2024-04-23T18:57:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a3904de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:02.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37881ab130 con 0x55c6a38f8800 2024-04-23T18:57:02.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37881ac5a0 con 0x55c6a38edf40 2024-04-23T18:57:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37881ab800 con 0x55c6a38ed620 2024-04-23T18:57:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.281+0000 7f378f57f740 10 client.0 _ll_register_callbacks cb 0x7f378002ab10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:02.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42762/0 (socket says 172.21.15.33:42762) 2024-04-23T18:57:03.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 -- 172.21.15.33:0/3502636322 learned_addr learned my addr 172.21.15.33:0/3502636322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:03.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f378659c640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a3904de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:03.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3785d9b640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 -- 172.21.15.33:0/3502636322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 msgr2=0x55c6a3904de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:03.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a3904de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 -- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 msgr2=0x55c6a38f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:03.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 -- 172.21.15.33:0/3502636322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6a38f2720 con 0x55c6a38f8800 2024-04-23T18:57:03.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3785d9b640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f3786d9d640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f377801b930 tx=0x7f377800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc3b959f86e07c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f376dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778035060 con 0x55c6a38f8800 2024-04-23T18:57:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f377801e950 con 0x55c6a38f8800 2024-04-23T18:57:03.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f377801ee00 con 0x55c6a38f8800 2024-04-23T18:57:03.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f377801eb70 con 0x55c6a38f8800 2024-04-23T18:57:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f378f57f740 10 client.18540 fetch_fsmap learned FSMap version 47 2024-04-23T18:57:03.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.282+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c6a3972ab0 con 0x55c6a38f8800 2024-04-23T18:57:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377801ee00 con 0x55c6a38f8800 2024-04-23T18:57:04.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f37780304d0 con 0x55c6a38f8800 2024-04-23T18:57:04.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 10 client.18540 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:57:04.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 10 client.18540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:04.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 20 client.18540 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:04.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a39731a0 0x55c6a39755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 4 client.18540 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6a385a200 con 0x55c6a39731a0 2024-04-23T18:57:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378659c640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a39731a0 0x55c6a39755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:04.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f378659c640 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a39731a0 0x55c6a39755c0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f39ef0e2c45e8e server_cookie=b2daa16613cc9e22 in_seq=0 out_seq=0 2024-04-23T18:57:04.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.283+0000 7f376dffb640 10 client.18540 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:04.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.287+0000 7f376dffb640 1 -- 172.21.15.33:0/3502636322 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6a385a200 con 0x55c6a39731a0 2024-04-23T18:57:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.287+0000 7f376dffb640 10 client.18540 handle_command_reply: tid=0 2024-04-23T18:57:04.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 client.18540 shutdown 2024-04-23T18:57:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 msgr2=0x55c6a38f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f377801b930 tx=0x7f377800ef70 comp rx=0 tx=0).stop 2024-04-23T18:57:05.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 shutdown_connections 2024-04-23T18:57:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55c6a39731a0 0x55c6a39755c0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6a38f8800 0x55c6a38f21e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c6a38edf40 0x55c6a38f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:05.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f376dffb640 0 client.18540 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:05.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 --2- 172.21.15.33:0/3502636322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c6a38ed620 0x55c6a3904de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:05.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 >> 172.21.15.33:0/3502636322 conn(0x55c6a38e9b30 msgr2=0x55c6a38e8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:05.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 shutdown_connections 2024-04-23T18:57:05.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 1 -- 172.21.15.33:0/3502636322 wait complete. 2024-04-23T18:57:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378f57f740 20 client.18540 trim_cache size 0 max 16384 2024-04-23T18:57:05.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 msgr2=0x7f37740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:06.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 0x7f37740a6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3778005fd0 tx=0x7f3778004a40 comp rx=0 tx=0).stop 2024-04-23T18:57:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 msgr2=0x7f3788198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f377000c970 tx=0x7f377000ce30 comp rx=0 tx=0).stop 2024-04-23T18:57:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 shutdown_connections 2024-04-23T18:57:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f37740a3dc0 0x7f37740a6280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:06.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f378819cfa0 0x7f378819d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3788107290 0x7f3788198c70 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 --2- 172.21.15.33:0/400631518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37881021c0 0x7f3788198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.288+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 >> 172.21.15.33:0/400631518 conn(0x7f37880fda20 msgr2=0x7f37880fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.289+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 shutdown_connections 2024-04-23T18:57:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:55:48.289+0000 7f378c86b640 1 -- 172.21.15.33:0/400631518 wait complete. 2024-04-23T18:57:06.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:49 smithi033 ceph-mon[29274]: pgmap v951: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:06.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:51 smithi033 ceph-mon[29274]: pgmap v952: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:53 smithi033 ceph-mon[29274]: pgmap v953: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:06.574 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:55 smithi033 ceph-mon[29274]: pgmap v954: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:06.667 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:57 smithi033 ceph-mon[29274]: pgmap v955: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:06.727 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:55:59 smithi033 ceph-mon[29274]: pgmap v956: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:06.856 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:55 smithi106 ceph-mon[32716]: pgmap v989: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:06.908 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:57 smithi106 ceph-mon[32716]: pgmap v990: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:06.960 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:56:59 smithi106 ceph-mon[32716]: pgmap v991: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:06.972 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:01 smithi106 ceph-mon[32716]: pgmap v992: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:07.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:03 smithi106 ceph-mon[32716]: pgmap v993: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:07.047 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:05 smithi106 ceph-mon[32716]: pgmap v994: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:07.141 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:55 smithi122 ceph-mon[32546]: pgmap v989: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:07.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:57 smithi122 ceph-mon[32546]: pgmap v990: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:07.220 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:56:59 smithi122 ceph-mon[32546]: pgmap v991: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:07.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:01 smithi122 ceph-mon[32546]: pgmap v992: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:07.279 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:03 smithi122 ceph-mon[32546]: pgmap v993: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:07.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:05 smithi122 ceph-mon[32546]: pgmap v994: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:07.365 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:06 smithi106 ceph-mon[32716]: pgmap v995: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:57:07.428 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:06 smithi122 ceph-mon[32546]: pgmap v995: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:57:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 -- 172.21.15.33:0/1867216660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f04102b70 msgr2=0x7f0f04106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 --2- 172.21.15.33:0/1867216660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f04102b70 0x7f0f04106bb0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009900 tx=0x7f0ef802a410 comp rx=0 tx=0).stop 2024-04-23T18:57:07.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 -- 172.21.15.33:0/1867216660 shutdown_connections 2024-04-23T18:57:07.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 --2- 172.21.15.33:0/1867216660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f0410ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 --2- 172.21.15.33:0/1867216660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f04102b70 0x7f0f04106bb0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 --2- 172.21.15.33:0/1867216660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 0x7f0f041025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 -- 172.21.15.33:0/1867216660 >> 172.21.15.33:0/1867216660 conn(0x7f0f040fda20 msgr2=0x7f0f040ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:07.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 -- 172.21.15.33:0/1867216660 shutdown_connections 2024-04-23T18:57:07.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.388+0000 7f0f09810640 1 -- 172.21.15.33:0/1867216660 wait complete. 2024-04-23T18:57:07.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 Processor -- start 2024-04-23T18:57:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 -- start start 2024-04-23T18:57:08.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 0x7f0f041986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f04198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041121b0 con 0x7f0f0419cf90 2024-04-23T18:57:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04112030 con 0x7f0f041021c0 2024-04-23T18:57:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.389+0000 7f0f09810640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04112330 con 0x7f0f04107290 2024-04-23T18:57:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38206/0 (socket says 172.21.15.33:38206) 2024-04-23T18:57:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 -- 172.21.15.33:0/3550121145 learned_addr learned my addr 172.21.15.33:0/3550121145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 0x7f0f041986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f027fc640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f04198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:08.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 -- 172.21.15.33:0/3550121145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 msgr2=0x7f0f04198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:08.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f04198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 -- 172.21.15.33:0/3550121145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 msgr2=0x7f0f041986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 0x7f0f041986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 -- 172.21.15.33:0/3550121145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f0419da80 con 0x7f0f0419cf90 2024-04-23T18:57:08.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f027fc640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f04198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:08.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f037fe640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009fd0 tx=0x7f0efc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f0880e640 1 -- 172.21.15.33:0/3550121145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc017070 con 0x7f0f0419cf90 2024-04-23T18:57:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f0880e640 1 -- 172.21.15.33:0/3550121145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc003e20 con 0x7f0f0419cf90 2024-04-23T18:57:09.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0419dd70 con 0x7f0f0419cf90 2024-04-23T18:57:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f0880e640 1 -- 172.21.15.33:0/3550121145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc010460 con 0x7f0f0419cf90 2024-04-23T18:57:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.390+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f041a54e0 con 0x7f0f0419cf90 2024-04-23T18:57:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.393+0000 7f0f0880e640 1 -- 172.21.15.33:0/3550121145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0efc003970 con 0x7f0f0419cf90 2024-04-23T18:57:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.393+0000 7f0f0c603740 1 Processor -- start 2024-04-23T18:57:09.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.393+0000 7f0f0c603740 1 -- start start 2024-04-23T18:57:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.393+0000 7f0f0880e640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 0x7f0eec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.393+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fa9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9f40 0x557e94fb42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:09.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0c603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e94f0e970 con 0x557e94fb4800 2024-04-23T18:57:09.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f027fc640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 0x7f0eec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:09.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0c603740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557e94eab4a0 con 0x557e94fa9f40 2024-04-23T18:57:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0c603740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557e94f0a930 con 0x557e94fa9620 2024-04-23T18:57:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f0880e640 1 -- 172.21.15.33:0/3550121145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f0efc0d0550 con 0x7f0f0419cf90 2024-04-23T18:57:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:09.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fa9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:09.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38208/0 (socket says 172.21.15.33:38208) 2024-04-23T18:57:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fa9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:33884/0 (socket says 172.21.15.33:33884) 2024-04-23T18:57:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 -- 172.21.15.33:0/569690863 learned_addr learned my addr 172.21.15.33:0/569690863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f027fc640 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9f40 0x557e94fb42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 -- 172.21.15.33:0/569690863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 msgr2=0x557e94fa9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fa9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:10.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 -- 172.21.15.33:0/569690863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9f40 msgr2=0x557e94fb42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9f40 0x557e94fb42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 -- 172.21.15.33:0/569690863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e94c21210 con 0x557e94fb4800 2024-04-23T18:57:10.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f027fc640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 0x7f0eec0a6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8031580 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:10.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0f037fe640 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0efc098eb0 tx=0x7f0efc09bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b404688e4d156c server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:10.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.394+0000 7f0ee9ffb640 1 -- 172.21.15.33:0/569690863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc020040 con 0x557e94fb4800 2024-04-23T18:57:10.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0ee9ffb640 1 -- 172.21.15.33:0/569690863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc09c4e0 con 0x557e94fb4800 2024-04-23T18:57:10.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0ee9ffb640 1 -- 172.21.15.33:0/569690863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc098b50 con 0x557e94fb4800 2024-04-23T18:57:10.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 -- 172.21.15.33:0/569690863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 msgr2=0x557e94fb6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f0efc098eb0 tx=0x7f0efc09bf00 comp rx=0 tx=0).stop 2024-04-23T18:57:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 -- 172.21.15.33:0/569690863 shutdown_connections 2024-04-23T18:57:10.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fb4800 0x557e94fb6bf0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9f40 0x557e94fb42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 --2- 172.21.15.33:0/569690863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fa9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:10.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 -- 172.21.15.33:0/569690863 >> 172.21.15.33:0/569690863 conn(0x557e94fa5b30 msgr2=0x557e94fa5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 -- 172.21.15.33:0/569690863 shutdown_connections 2024-04-23T18:57:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.395+0000 7f0f0c603740 1 -- 172.21.15.33:0/569690863 wait complete. 2024-04-23T18:57:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 -- start start 2024-04-23T18:57:11.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 Processor -- start 2024-04-23T18:57:11.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 0x557e94fb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:11.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 0x557e94fb1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:11.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557e94fb73d0 con 0x557e94fa9f40 2024-04-23T18:57:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557e94ea90b0 con 0x557e94fa9620 2024-04-23T18:57:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557e94fb7550 con 0x557e94fb4800 2024-04-23T18:57:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f0c603740 10 client.0 _ll_register_callbacks cb 0x557e94e96d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:11.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38222/0 (socket says 172.21.15.33:38222) 2024-04-23T18:57:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f027fc640 1 -- 172.21.15.33:0/570827669 learned_addr learned my addr 172.21.15.33:0/570827669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:11.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f037fe640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 0x557e94fb1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:11.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.396+0000 7f0f027fc640 1 -- 172.21.15.33:0/570827669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 msgr2=0x557e94fb1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:11.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 0x557e94fb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:11.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f027fc640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 0x557e94fb1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:11.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f027fc640 1 -- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 msgr2=0x557e94fb41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:11.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f027fc640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 0x557e94fb41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:12.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f027fc640 1 -- 172.21.15.33:0/570827669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e94ced620 con 0x557e94fa9f40 2024-04-23T18:57:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f037fe640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 0x557e94fb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:12.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 0x557e94fb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f027fc640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0f0406b5f0 tx=0x7f0ef8043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4f45e89fb18394 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:12.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0ee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:12.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8035050 con 0x557e94fa9f40 2024-04-23T18:57:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f0ef8005450 con 0x557e94fa9f40 2024-04-23T18:57:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc09c4e0 con 0x557e94fa9f40 2024-04-23T18:57:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0ef8005650 con 0x557e94fa9f40 2024-04-23T18:57:12.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f0c603740 10 client.18552 fetch_fsmap learned FSMap version 47 2024-04-23T18:57:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557e94fc33b0 con 0x557e94fa9f40 2024-04-23T18:57:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.397+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0efc09c4e0 con 0x557e94fa9f40 2024-04-23T18:57:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f0ef804f020 con 0x557e94fa9f40 2024-04-23T18:57:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 10 client.18552 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:57:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 10 client.18552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 20 client.18552 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:12.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e94fc4820 0x557e94fc6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 4 client.18552 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e94f1c390 con 0x557e94fc4820 2024-04-23T18:57:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e94fc4820 0x557e94fc6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:12.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e94fc4820 0x557e94fc6c30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b1ee5640673669 server_cookie=69b0a9fef410555e in_seq=0 out_seq=0 2024-04-23T18:57:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.398+0000 7f0ee2ffd640 10 client.18552 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/570827669 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e94f1c390 con 0x557e94fc4820 2024-04-23T18:57:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0ee2ffd640 10 client.18552 handle_command_reply: tid=0 2024-04-23T18:57:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 client.18552 shutdown 2024-04-23T18:57:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 msgr2=0x557e94fb14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0f0406b5f0 tx=0x7f0ef8043000 comp rx=0 tx=0).stop 2024-04-23T18:57:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 shutdown_connections 2024-04-23T18:57:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e94fc4820 0x557e94fc6c30 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fb4800 0x557e94fb1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fb14c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:13.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0ee2ffd640 0 client.18552 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:13.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 --2- 172.21.15.33:0/570827669 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fa9620 0x557e94fb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.399+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 >> 172.21.15.33:0/570827669 conn(0x557e94fa5b30 msgr2=0x557e94fa4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:13.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.400+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 shutdown_connections 2024-04-23T18:57:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.400+0000 7f0f0c603740 1 -- 172.21.15.33:0/570827669 wait complete. 2024-04-23T18:57:13.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.400+0000 7f0f0c603740 20 client.18552 trim_cache size 0 max 16384 2024-04-23T18:57:13.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 Processor -- start 2024-04-23T18:57:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 -- start start 2024-04-23T18:57:13.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fc0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:13.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fcc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ac620 con 0x557e94fa9f40 2024-04-23T18:57:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041aef00 con 0x557e94fb4800 2024-04-23T18:57:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f0c603740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041accf0 con 0x557e94fa9620 2024-04-23T18:57:13.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38230/0 (socket says 172.21.15.33:38230) 2024-04-23T18:57:13.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 -- 172.21.15.33:0/4240331383 learned_addr learned my addr 172.21.15.33:0/4240331383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:14.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fc0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:14.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 -- 172.21.15.33:0/4240331383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 msgr2=0x557e94fc0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f037fe640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fcc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fc0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 -- 172.21.15.33:0/4240331383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 msgr2=0x557e94fcc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fcc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 -- 172.21.15.33:0/4240331383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e94fb1e40 con 0x557e94fa9f40 2024-04-23T18:57:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fc0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:14.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.415+0000 7f0f027fc640 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8033000 tx=0x7f0ef8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fd2ca6b50a6e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:14.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0ee97fa640 1 -- 172.21.15.33:0/4240331383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8035070 con 0x557e94fa9f40 2024-04-23T18:57:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0ee97fa640 1 -- 172.21.15.33:0/4240331383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef805f070 con 0x557e94fa9f40 2024-04-23T18:57:14.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0ee97fa640 1 -- 172.21.15.33:0/4240331383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8032d80 con 0x557e94fa9f40 2024-04-23T18:57:14.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 -- 172.21.15.33:0/4240331383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 msgr2=0x557e94fa4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8033000 tx=0x7f0ef8031a50 comp rx=0 tx=0).stop 2024-04-23T18:57:14.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 -- 172.21.15.33:0/4240331383 shutdown_connections 2024-04-23T18:57:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fcc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9f40 0x557e94fa4b30 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 --2- 172.21.15.33:0/4240331383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9620 0x557e94fc0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 -- 172.21.15.33:0/4240331383 >> 172.21.15.33:0/4240331383 conn(0x557e94fa5b30 msgr2=0x557e94fa4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 -- 172.21.15.33:0/4240331383 shutdown_connections 2024-04-23T18:57:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.416+0000 7f0f0c603740 1 -- 172.21.15.33:0/4240331383 wait complete. 2024-04-23T18:57:15.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 -- start start 2024-04-23T18:57:15.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 Processor -- start 2024-04-23T18:57:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 0x557e94fba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:15.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fbabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ad3c0 con 0x557e94fa9620 2024-04-23T18:57:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ae830 con 0x557e94fb4800 2024-04-23T18:57:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ada90 con 0x557e94fa9f40 2024-04-23T18:57:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f0c603740 10 client.0 _ll_register_callbacks cb 0x7f0ef8004440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38242/0 (socket says 172.21.15.33:38242) 2024-04-23T18:57:15.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f02ffd640 1 -- 172.21.15.33:0/1077959287 learned_addr learned my addr 172.21.15.33:0/1077959287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f027fc640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 0x557e94fba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:15.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f037fe640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fbabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.417+0000 7f0f02ffd640 1 -- 172.21.15.33:0/1077959287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 msgr2=0x557e94fba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:15.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 0x557e94fba690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f02ffd640 1 -- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 msgr2=0x557e94fbabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:15.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fbabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f02ffd640 1 -- 172.21.15.33:0/1077959287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e95029dc0 con 0x557e94fa9620 2024-04-23T18:57:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f027fc640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 0x557e94fba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:16.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f037fe640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fbabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:16.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f02ffd640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005d00 tx=0x7f0ef400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5058f6cf4004df7f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:16.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef401d670 con 0x557e94fa9620 2024-04-23T18:57:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 8542+0+0 (secure 0 0 0) 0x7f0ef401d810 con 0x557e94fa9620 2024-04-23T18:57:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef805f070 con 0x557e94fa9620 2024-04-23T18:57:16.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0ef402c050 con 0x557e94fa9620 2024-04-23T18:57:16.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f0c603740 10 client.18558 fetch_fsmap learned FSMap version 47 2024-04-23T18:57:16.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x557e9502b000 con 0x557e94fa9620 2024-04-23T18:57:16.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef805f070 con 0x557e94fa9620 2024-04-23T18:57:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 47) ==== 7144+0+0 (secure 0 0 0) 0x7f0ef4010430 con 0x557e94fa9620 2024-04-23T18:57:16.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.418+0000 7f0f0c603740 10 client.18558 fetch_fsmap finished waiting for FSMap version 47 2024-04-23T18:57:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f0c603740 10 client.18558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:16.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f0c603740 20 client.18558 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:16.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e9502b780 0x557e9502dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f0c603740 4 client.18558 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e94f16200 con 0x557e9502b780 2024-04-23T18:57:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f027fc640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e9502b780 0x557e9502dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0ee2ffd640 10 client.18558 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:01.419+0000 7f0f027fc640 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e9502b780 0x557e9502dba0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cde5bc7813ffa35 server_cookie=2fe7c611735ae9ff in_seq=0 out_seq=0 2024-04-23T18:57:17.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:01 smithi033 ceph-mon[29274]: pgmap v957: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:17.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0ee2ffd640 1 -- 172.21.15.33:0/1077959287 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e94f16200 con 0x557e9502b780 2024-04-23T18:57:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0ee2ffd640 10 client.18558 handle_command_reply: tid=0 2024-04-23T18:57:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 client.18558 shutdown 2024-04-23T18:57:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 msgr2=0x557e94fba150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005d00 tx=0x7f0ef400bf20 comp rx=0 tx=0).stop 2024-04-23T18:57:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 shutdown_connections 2024-04-23T18:57:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x557e9502b780 0x557e9502dba0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557e94fb4800 0x557e94fbabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:17.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e94fa9f40 0x557e94fba690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 --2- 172.21.15.33:0/1077959287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557e94fa9620 0x557e94fba150 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0ee2ffd640 0 client.18558 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:17.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 >> 172.21.15.33:0/1077959287 conn(0x557e94fa5b30 msgr2=0x557e94fa4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:17.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 shutdown_connections 2024-04-23T18:57:17.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.202+0000 7f0f0c603740 1 -- 172.21.15.33:0/1077959287 wait complete. 2024-04-23T18:57:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f0c603740 20 client.18558 trim_cache size 0 max 16384 2024-04-23T18:57:17.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:17.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:17.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:17.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 msgr2=0x7f0eec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 0x7f0eec0a6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8000c00 tx=0x7f0ef8031580 comp rx=0 tx=0).stop 2024-04-23T18:57:17.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 msgr2=0x7f0f0419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:18.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009fd0 tx=0x7f0efc00b260 comp rx=0 tx=0).stop 2024-04-23T18:57:18.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 shutdown_connections 2024-04-23T18:57:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0eec0a3dc0 0x7f0eec0a6280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f0419cf90 0x7f0f0419d440 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:18.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f04107290 0x7f0f04198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:18.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 --2- 172.21.15.33:0/3550121145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f041021c0 0x7f0f041986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.203+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 >> 172.21.15.33:0/3550121145 conn(0x7f0f040fda20 msgr2=0x7f0f040ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.204+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 shutdown_connections 2024-04-23T18:57:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:03.204+0000 7f0f09810640 1 -- 172.21.15.33:0/3550121145 wait complete. 2024-04-23T18:57:18.411 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-mon[29274]: pgmap v958: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:18.421 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:57:18.491 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: in thread 7fdd77a30700 thread_name:md_log_replay 2024-04-23T18:57:18.550 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:57:18.621 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdd88df1d20] 2024-04-23T18:57:18.632 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2: /usr/bin/ceph-mds(+0x549372) [0x556c6a511372] 2024-04-23T18:57:18.641 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c6a514b23] 2024-04-23T18:57:18.650 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 4: (MDLog::_replay_thread()+0x617) [0x556c6a56f647] 2024-04-23T18:57:18.772 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c6a578d31] 2024-04-23T18:57:18.785 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdd88de71ca] 2024-04-23T18:57:18.850 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 7: clone() 2024-04-23T18:57:18.967 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:56:03.365+0000 7fdd77a30700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:57:19.057 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: in thread 7fdd77a30700 thread_name:md_log_replay 2024-04-23T18:57:19.085 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:19.094 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:57:19.168 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdd88df1d20] 2024-04-23T18:57:19.204 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2: /usr/bin/ceph-mds(+0x549372) [0x556c6a511372] 2024-04-23T18:57:19.215 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c6a514b23] 2024-04-23T18:57:19.224 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 4: (MDLog::_replay_thread()+0x617) [0x556c6a56f647] 2024-04-23T18:57:19.290 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c6a578d31] 2024-04-23T18:57:19.304 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdd88de71ca] 2024-04-23T18:57:19.362 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 7: clone() 2024-04-23T18:57:19.369 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:57:19.376 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:19.383 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 0> 2024-04-23T18:56:03.365+0000 7fdd77a30700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:57:19.512 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: in thread 7fdd77a30700 thread_name:md_log_replay 2024-04-23T18:57:19.572 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:19.611 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:57:19.728 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdd88df1d20] 2024-04-23T18:57:19.917 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2: /usr/bin/ceph-mds(+0x549372) [0x556c6a511372] 2024-04-23T18:57:19.940 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c6a514b23] 2024-04-23T18:57:19.989 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 4: (MDLog::_replay_thread()+0x617) [0x556c6a56f647] 2024-04-23T18:57:20.065 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c6a578d31] 2024-04-23T18:57:20.090 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdd88de71ca] 2024-04-23T18:57:20.097 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 7: clone() 2024-04-23T18:57:20.104 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:57:20.199 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:20.221 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 0> 2024-04-23T18:56:03.365+0000 7fdd77a30700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:57:20.229 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: in thread 7fdd77a30700 thread_name:md_log_replay 2024-04-23T18:57:20.236 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:20.312 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:57:20.320 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fdd88df1d20] 2024-04-23T18:57:20.430 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2: /usr/bin/ceph-mds(+0x549372) [0x556c6a511372] 2024-04-23T18:57:20.437 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c6a514b23] 2024-04-23T18:57:20.493 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 4: (MDLog::_replay_thread()+0x617) [0x556c6a56f647] 2024-04-23T18:57:20.589 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c6a578d31] 2024-04-23T18:57:20.694 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fdd88de71ca] 2024-04-23T18:57:20.738 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 7: clone() 2024-04-23T18:57:20.746 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:57:20.755 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[100745]: 2024-04-23T18:57:20.764 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:03 smithi033 podman[101415]: 2024-04-23 18:56:03.421402123 +0000 UTC m=+0.023172065 container died ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T18:57:20.773 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:04 smithi033 podman[101415]: 2024-04-23 18:56:04.225552648 +0000 UTC m=+0.827322578 container remove ee74bae1584e52440301bf3f79bed7bf184359ac3cea9fb7f3783149a8137aba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T18:57:20.840 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:57:20.899 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-23T18:57:20.999 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:09 smithi122 ceph-mon[32546]: osdmap e149: 12 total, 12 up, 12 in 2024-04-23T18:57:21.009 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:09 smithi122 ceph-mon[32546]: pgmap v997: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:21.067 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:11 smithi122 ceph-mon[32546]: pgmap v998: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:21.137 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:13 smithi122 ceph-mon[32546]: pgmap v999: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:21.193 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:13 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:57:21.202 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.283 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.291 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: pgmap v1000: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:21.361 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.426 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.482 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:15 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.499 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.508 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.633 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:21.712 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:21.775 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:21.789 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:21.798 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:57:21.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:57:21.946 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:21.955 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:22.063 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:22.134 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.197 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:22.247 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.319 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.564 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:22.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:57:22.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:57:22.786 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:22.795 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:16 smithi122 ceph-mon[32546]: osdmap e150: 12 total, 12 up, 12 in 2024-04-23T18:57:22.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:57:22.902 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:57:22.910 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:57:22.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:57:22.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: pgmap v1001: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 883 B/s wr, 11 op/s 2024-04-23T18:57:22.959 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: pgmap v1002: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:23.013 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:17 smithi122 ceph-mon[32546]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:57:23.078 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:19 smithi122 ceph-mon[32546]: pgmap v1004: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:23.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:09 smithi106 ceph-mon[32716]: osdmap e149: 12 total, 12 up, 12 in 2024-04-23T18:57:23.159 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:09 smithi106 ceph-mon[32716]: pgmap v997: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:23.167 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:11 smithi106 ceph-mon[32716]: pgmap v998: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:23.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:13 smithi106 ceph-mon[32716]: pgmap v999: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:23.278 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:13 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:57:23.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.343 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: pgmap v1000: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:23.432 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:15 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.625 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.699 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:23.811 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:23.905 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:23.993 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.079 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.117 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:57:24.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:57:24.171 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:24.182 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:24.204 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:24.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.371 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:24.435 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.444 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.460 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.593 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.606 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.669 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:24.739 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:57:24.815 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:57:24.845 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:24.854 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:16 smithi106 ceph-mon[32716]: osdmap e150: 12 total, 12 up, 12 in 2024-04-23T18:57:24.863 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:57:24.872 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:57:24.881 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:57:24.887 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:57:24.938 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: pgmap v1001: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 883 B/s wr, 11 op/s 2024-04-23T18:57:24.970 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: pgmap v1002: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:24.977 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:17 smithi106 ceph-mon[32716]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:57:24.986 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:19 smithi106 ceph-mon[32716]: pgmap v1004: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:25.054 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:05 smithi033 ceph-mon[29274]: pgmap v959: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:25.062 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:05 smithi033 ceph-mon[29274]: osdmap e146: 12 total, 12 up, 12 in 2024-04-23T18:57:25.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:07 smithi033 ceph-mon[29274]: pgmap v961: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T18:57:25.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:09 smithi033 ceph-mon[29274]: pgmap v962: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:25.160 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:11 smithi033 ceph-mon[29274]: pgmap v963: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:25.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:11 smithi033 ceph-mon[29274]: osdmap e147: 12 total, 12 up, 12 in 2024-04-23T18:57:25.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:12 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:57:25.364 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: pgmap v965: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:57:25.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:57:25.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:57:25.489 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: pgmap v966: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 66 KiB/s wr, 37 op/s 2024-04-23T18:57:25.496 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: pgmap v967: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 79 KiB/s wr, 44 op/s 2024-04-23T18:57:25.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:13 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:25.556 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:21 smithi122 ceph-mon[32546]: pgmap v1005: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T18:57:25.563 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:23 smithi122 ceph-mon[32546]: pgmap v1006: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 op/s 2024-04-23T18:57:25.569 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:21 smithi106 ceph-mon[32716]: pgmap v1005: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T18:57:25.578 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:23 smithi106 ceph-mon[32716]: pgmap v1006: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 op/s 2024-04-23T18:57:25.587 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 2. 2024-04-23T18:57:25.595 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:14 smithi033 systemd[1]: Stopped Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:25.706 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:14 smithi033 systemd[1]: Starting Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:57:25.786 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:14 smithi033 podman[101641]: 2024-04-23 18:56:14.635783055 +0000 UTC m=+0.019570452 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:57:25.839 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:14 smithi033 podman[101641]: 2024-04-23 18:56:14.776476071 +0000 UTC m=+0.160263436 container create 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T18:57:25.925 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 podman[101641]: 2024-04-23 18:56:15.037851211 +0000 UTC m=+0.421638575 container init 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-23T18:57:26.048 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 podman[101641]: 2024-04-23 18:56:15.040976023 +0000 UTC m=+0.424763395 container start 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T18:57:26.104 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 ceph-mds[101660]: -- 172.21.15.33:0/711895870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647e78afa40 con 0x5647e7882c00 2024-04-23T18:57:26.120 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: starting mds.j at 2024-04-23T18:57:26.229 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 bash[101641]: 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 2024-04-23T18:57:26.238 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:56:15 smithi033 systemd[1]: Started Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:57:26.254 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:15 smithi033 ceph-mon[29274]: pgmap v968: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 150 B/s wr, 6 op/s 2024-04-23T18:57:26.346 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:15 smithi033 ceph-mon[29274]: daemon mds.j restarted 2024-04-23T18:57:26.430 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:25 smithi122 ceph-mon[32546]: pgmap v1007: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T18:57:26.475 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:25 smithi106 ceph-mon[32716]: pgmap v1007: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T18:57:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 -- 172.21.15.33:0/2368237887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc81010f0 msgr2=0x7f6dc81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:26.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/2368237887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc81010f0 0x7f6dc81014d0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009900 tx=0x7f6dc002a410 comp rx=0 tx=0).stop 2024-04-23T18:57:26.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 -- 172.21.15.33:0/2368237887 shutdown_connections 2024-04-23T18:57:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/2368237887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc8106170 0x7f6dc810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/2368237887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc8101aa0 0x7f6dc8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:26.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/2368237887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc81010f0 0x7f6dc81014d0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:26.797 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 -- 172.21.15.33:0/2368237887 >> 172.21.15.33:0/2368237887 conn(0x7f6dc8077fd0 msgr2=0x7f6dc80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 -- 172.21.15.33:0/2368237887 shutdown_connections 2024-04-23T18:57:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.297+0000 7f6dcf74f640 1 -- 172.21.15.33:0/2368237887 wait complete. 2024-04-23T18:57:27.027 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 Processor -- start 2024-04-23T18:57:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 -- start start 2024-04-23T18:57:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc81010f0 0x7f6dc8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc819bf10 0x7f6dc819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:27.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81110b0 con 0x7f6dc8106170 2024-04-23T18:57:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8110f30 con 0x7f6dc81010f0 2024-04-23T18:57:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcf74f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc8111230 con 0x7f6dc819bf10 2024-04-23T18:57:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:27.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54324/0 (socket says 172.21.15.33:54324) 2024-04-23T18:57:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 -- 172.21.15.33:0/666454591 learned_addr learned my addr 172.21.15.33:0/666454591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:27.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc81010f0 0x7f6dc8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc819bf10 0x7f6dc819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 -- 172.21.15.33:0/666454591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc819bf10 msgr2=0x7f6dc819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:27.961 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc819bf10 0x7f6dc819c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:28.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 -- 172.21.15.33:0/666454591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc81010f0 msgr2=0x7f6dc8197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc81010f0 0x7f6dc8197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:28.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 -- 172.21.15.33:0/666454591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc819ca00 con 0x7f6dc8106170 2024-04-23T18:57:28.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.298+0000 7f6dcccea640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9c0 tx=0x7f6db800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:28.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.299+0000 7f6dbe7fc640 1 -- 172.21.15.33:0/666454591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800cd80 con 0x7f6dc8106170 2024-04-23T18:57:28.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.299+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc819ccf0 con 0x7f6dc8106170 2024-04-23T18:57:28.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.299+0000 7f6dbe7fc640 1 -- 172.21.15.33:0/666454591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db80054d0 con 0x7f6dc8106170 2024-04-23T18:57:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.299+0000 7f6dbe7fc640 1 -- 172.21.15.33:0/666454591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8010430 con 0x7f6dc8106170 2024-04-23T18:57:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.299+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc81a4530 con 0x7f6dc8106170 2024-04-23T18:57:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dbe7fc640 1 -- 172.21.15.33:0/666454591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6db8004fd0 con 0x7f6dc8106170 2024-04-23T18:57:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 Processor -- start 2024-04-23T18:57:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 -- start start 2024-04-23T18:57:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f287b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f28709970 con 0x563f287a6830 2024-04-23T18:57:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563f286a64a0 con 0x563f287a7150 2024-04-23T18:57:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dd246f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f28705930 con 0x563f287b1a10 2024-04-23T18:57:29.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dcdcec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dcdcec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54326/0 (socket says 172.21.15.33:54326) 2024-04-23T18:57:29.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dcdcec640 1 -- 172.21.15.33:0/1120252349 learned_addr learned my addr 172.21.15.33:0/1120252349 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f287b14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:29.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dbe7fc640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 0x7f6db40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:29.116 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.302+0000 7f6dcccea640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1120252349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 msgr2=0x563f287b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:29.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:29.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1120252349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 msgr2=0x563f287a6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1120252349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f2841c210 con 0x563f287a7150 2024-04-23T18:57:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dbe7fc640 1 -- 172.21.15.33:0/666454591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f6db80d0ca0 con 0x7f6dc8106170 2024-04-23T18:57:29.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 0x7f6db40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f287b14d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009230 tx=0x7f6dc0032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ef756d504efc23a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6db37fe640 1 -- 172.21.15.33:0/1120252349 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0040070 con 0x563f287a7150 2024-04-23T18:57:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6db37fe640 1 -- 172.21.15.33:0/1120252349 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc0004030 con 0x563f287a7150 2024-04-23T18:57:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.303+0000 7f6db37fe640 1 -- 172.21.15.33:0/1120252349 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0035040 con 0x563f287a7150 2024-04-23T18:57:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- 172.21.15.33:0/1120252349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 msgr2=0x563f287b14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 0x7f6db40a6280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc4004a40 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f287b14d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009230 tx=0x7f6dc0032b20 comp rx=0 tx=0).stop 2024-04-23T18:57:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- 172.21.15.33:0/1120252349 shutdown_connections 2024-04-23T18:57:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:29.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f287b14d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1120252349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287a6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- 172.21.15.33:0/1120252349 >> 172.21.15.33:0/1120252349 conn(0x563f287a2d40 msgr2=0x563f287a3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:30.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- 172.21.15.33:0/1120252349 shutdown_connections 2024-04-23T18:57:30.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- 172.21.15.33:0/1120252349 wait complete. 2024-04-23T18:57:30.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 -- start start 2024-04-23T18:57:30.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.304+0000 7f6dd246f740 1 Processor -- start 2024-04-23T18:57:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:30.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563f287b45e0 con 0x563f287a6830 2024-04-23T18:57:30.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563f286a40b0 con 0x563f287b1a10 2024-04-23T18:57:30.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563f287b4760 con 0x563f287a7150 2024-04-23T18:57:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 10 client.0 _ll_register_callbacks cb 0x563f28691d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:30.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54338/0 (socket says 172.21.15.33:54338) 2024-04-23T18:57:30.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1277059497 learned_addr learned my addr 172.21.15.33:0/1277059497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:30.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcccea640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1277059497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 msgr2=0x563f287ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 msgr2=0x563f287aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:30.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/1277059497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f284e8620 con 0x563f287a6830 2024-04-23T18:57:31.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005c40 tx=0x7f6dc0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41078c42a8f5dc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6db0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:31.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0040070 con 0x563f287a6830 2024-04-23T18:57:31.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f6dc00316a0 con 0x563f287a6830 2024-04-23T18:57:31.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc0004030 con 0x563f287a6830 2024-04-23T18:57:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6dc0056050 con 0x563f287a6830 2024-04-23T18:57:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.305+0000 7f6dd246f740 10 client.18576 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563f287c0530 con 0x563f287a6830 2024-04-23T18:57:31.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc0004030 con 0x563f287a6830 2024-04-23T18:57:31.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f6dc0004c40 con 0x563f287a6830 2024-04-23T18:57:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 10 client.18576 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 10 client.18576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 20 client.18576 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:31.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f287c1880 0x563f287c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 4 client.18576 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f28717390 con 0x563f287c1880 2024-04-23T18:57:31.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dcccea640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f287c1880 0x563f287c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:31.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6dcccea640 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f287c1880 0x563f287c3c70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f7e533bbf2eddb server_cookie=bff7664496dbb10c in_seq=0 out_seq=0 2024-04-23T18:57:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.306+0000 7f6db0ff9640 10 client.18576 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:31.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6db0ff9640 1 -- 172.21.15.33:0/1277059497 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f28717390 con 0x563f287c1880 2024-04-23T18:57:31.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6db0ff9640 10 client.18576 handle_command_reply: tid=0 2024-04-23T18:57:31.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 client.18576 shutdown 2024-04-23T18:57:31.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 msgr2=0x563f287b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:31.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005c40 tx=0x7f6dc0032f40 comp rx=0 tx=0).stop 2024-04-23T18:57:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 shutdown_connections 2024-04-23T18:57:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f287c1880 0x563f287c3c70 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6db0ff9640 0 client.18576 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 --2- 172.21.15.33:0/1277059497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b1390 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:32.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 >> 172.21.15.33:0/1277059497 conn(0x563f287a2d40 msgr2=0x563f2879fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 shutdown_connections 2024-04-23T18:57:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 1 -- 172.21.15.33:0/1277059497 wait complete. 2024-04-23T18:57:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.307+0000 7f6dd246f740 20 client.18576 trim_cache size 0 max 16384 2024-04-23T18:57:32.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 Processor -- start 2024-04-23T18:57:32.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 -- start start 2024-04-23T18:57:32.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81aa3d0 con 0x563f287a6830 2024-04-23T18:57:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81accb0 con 0x563f287b1a10 2024-04-23T18:57:32.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dd246f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81aaaa0 con 0x563f287a7150 2024-04-23T18:57:32.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.322+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54354/0 (socket says 172.21.15.33:54354) 2024-04-23T18:57:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/788609245 learned_addr learned my addr 172.21.15.33:0/788609245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcccea640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/788609245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 msgr2=0x563f287b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/788609245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 msgr2=0x563f287c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:33.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/788609245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f287acbd0 con 0x563f287a6830 2024-04-23T18:57:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005c10 tx=0x7f6dc0003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d78342b07ac2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6db2ffd640 1 -- 172.21.15.33:0/788609245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0040070 con 0x563f287a6830 2024-04-23T18:57:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6db2ffd640 1 -- 172.21.15.33:0/788609245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc004d070 con 0x563f287a6830 2024-04-23T18:57:33.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.323+0000 7f6db2ffd640 1 -- 172.21.15.33:0/788609245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0035040 con 0x563f287a6830 2024-04-23T18:57:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- 172.21.15.33:0/788609245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 msgr2=0x563f287b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005c10 tx=0x7f6dc0003990 comp rx=0 tx=0).stop 2024-04-23T18:57:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- 172.21.15.33:0/788609245 shutdown_connections 2024-04-23T18:57:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287b1a10 0x563f287c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287a7150 0x563f287b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:33.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 --2- 172.21.15.33:0/788609245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f287b9d90 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- 172.21.15.33:0/788609245 >> 172.21.15.33:0/788609245 conn(0x563f287a2d40 msgr2=0x563f287adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- 172.21.15.33:0/788609245 shutdown_connections 2024-04-23T18:57:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- 172.21.15.33:0/788609245 wait complete. 2024-04-23T18:57:33.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 -- start start 2024-04-23T18:57:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 Processor -- start 2024-04-23T18:57:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.324+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f28822740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81ab170 con 0x563f287a6830 2024-04-23T18:57:34.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81ac5e0 con 0x563f287a7150 2024-04-23T18:57:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc81ab840 con 0x563f287b1a10 2024-04-23T18:57:34.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 10 client.0 _ll_register_callbacks cb 0x7f6dc00095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54370/0 (socket says 172.21.15.33:54370) 2024-04-23T18:57:34.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/831915693 learned_addr learned my addr 172.21.15.33:0/831915693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcccea640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f28822740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:34.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/831915693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 msgr2=0x563f287ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:34.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:34.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 msgr2=0x563f28822740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:34.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f28822740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 -- 172.21.15.33:0/831915693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f287aba70 con 0x563f287a6830 2024-04-23T18:57:34.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcdcec640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:34.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dcd4eb640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005ec0 tx=0x7f6dc0005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecdf86a4daabcb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6db0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:34.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0035070 con 0x563f287a6830 2024-04-23T18:57:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f6dc00314a0 con 0x563f287a6830 2024-04-23T18:57:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc004d070 con 0x563f287a6830 2024-04-23T18:57:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6dc0048070 con 0x563f287a6830 2024-04-23T18:57:34.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 10 client.18582 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.325+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563f28829990 con 0x563f287a6830 2024-04-23T18:57:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc004d070 con 0x563f287a6830 2024-04-23T18:57:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f6dc00046e0 con 0x563f287a6830 2024-04-23T18:57:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 10 client.18582 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:34.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 10 client.18582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 20 client.18582 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:35.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f28829fe0 0x563f2882c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:35.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 4 client.18582 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f28711200 con 0x563f28829fe0 2024-04-23T18:57:35.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dcccea640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f28829fe0 0x563f2882c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:35.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6dcccea640 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f28829fe0 0x563f2882c3d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c4dd0fb206f8a4c server_cookie=c0aac380936a9ac3 in_seq=0 out_seq=0 2024-04-23T18:57:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.326+0000 7f6db0ff9640 10 client.18582 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:35.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6db0ff9640 1 -- 172.21.15.33:0/831915693 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f28711200 con 0x563f28829fe0 2024-04-23T18:57:35.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6db0ff9640 10 client.18582 handle_command_reply: tid=0 2024-04-23T18:57:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 client.18582 shutdown 2024-04-23T18:57:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 msgr2=0x563f28822200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:35.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0005ec0 tx=0x7f6dc0005180 comp rx=0 tx=0).stop 2024-04-23T18:57:35.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 shutdown_connections 2024-04-23T18:57:35.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x563f28829fe0 0x563f2882c3d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563f287b1a10 0x563f287ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:35.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563f287a7150 0x563f28822740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6db0ff9640 0 client.18582 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 --2- 172.21.15.33:0/831915693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563f287a6830 0x563f28822200 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.332+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 >> 172.21.15.33:0/831915693 conn(0x563f287a2d40 msgr2=0x563f287a1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:36.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 shutdown_connections 2024-04-23T18:57:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dd246f740 1 -- 172.21.15.33:0/831915693 wait complete. 2024-04-23T18:57:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dd246f740 20 client.18582 trim_cache size 0 max 16384 2024-04-23T18:57:36.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:36.274 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 msgr2=0x7f6db40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:36.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 0x7f6db40a6280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc4004a40 comp rx=0 tx=0).stop 2024-04-23T18:57:36.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 msgr2=0x7f6dc8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:36.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f6db800e9c0 tx=0x7f6db800ee80 comp rx=0 tx=0).stop 2024-04-23T18:57:36.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 shutdown_connections 2024-04-23T18:57:36.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6db40a3dc0 0x7f6db40a6280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:36.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dc819bf10 0x7f6dc819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dc8106170 0x7f6dc8197b90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:36.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 --2- 172.21.15.33:0/666454591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc81010f0 0x7f6dc8197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:36.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 >> 172.21.15.33:0/666454591 conn(0x7f6dc8077fd0 msgr2=0x7f6dc80ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 shutdown_connections 2024-04-23T18:57:36.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:16.333+0000 7f6dcf74f640 1 -- 172.21.15.33:0/666454591 wait complete. 2024-04-23T18:57:37.109 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:16 smithi033 ceph-mon[29274]: osdmap e148: 12 total, 12 up, 12 in 2024-04-23T18:57:37.203 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6834/395009281,v1:172.21.15.33:6835/395009281] up:boot 2024-04-23T18:57:37.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 4 up:standby-replay 1 up:standby 5 up:active 2024-04-23T18:57:37.302 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:57:37.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:57:37.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:17 smithi033 ceph-mon[29274]: pgmap v970: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 9 op/s 2024-04-23T18:57:37.375 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:19 smithi033 ceph-mon[29274]: pgmap v971: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 7 op/s 2024-04-23T18:57:37.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:21 smithi033 ceph-mon[29274]: pgmap v972: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 6 op/s 2024-04-23T18:57:37.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:23 smithi033 ceph-mon[29274]: pgmap v973: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:57:37.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:25 smithi033 ceph-mon[29274]: pgmap v974: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 409 B/s wr, 10 op/s 2024-04-23T18:57:37.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:27 smithi033 ceph-mon[29274]: pgmap v975: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 374 B/s wr, 14 op/s 2024-04-23T18:57:37.517 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:27 smithi106 ceph-mon[32716]: pgmap v1008: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:57:37.592 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:29 smithi106 ceph-mon[32716]: pgmap v1009: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 267 B/s wr, 8 op/s 2024-04-23T18:57:37.721 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:31 smithi106 ceph-mon[32716]: pgmap v1010: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:37.782 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:33 smithi106 ceph-mon[32716]: pgmap v1011: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:37.790 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:35 smithi106 ceph-mon[32716]: pgmap v1012: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:37.797 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:27 smithi122 ceph-mon[32546]: pgmap v1008: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:57:37.860 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:29 smithi122 ceph-mon[32546]: pgmap v1009: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 267 B/s wr, 8 op/s 2024-04-23T18:57:37.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:31 smithi122 ceph-mon[32546]: pgmap v1010: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:37.939 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:33 smithi122 ceph-mon[32546]: pgmap v1011: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:37.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:35 smithi122 ceph-mon[32546]: pgmap v1012: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:38.037 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:37 smithi122 ceph-mon[32546]: pgmap v1013: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:57:38.047 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:37 smithi106 ceph-mon[32716]: pgmap v1013: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:57:38.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 -- 172.21.15.33:0/953392869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248106190 msgr2=0x7f124810dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 --2- 172.21.15.33:0/953392869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248106190 0x7f124810dd60 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f1240009ca0 tx=0x7f124002a8c0 comp rx=0 tx=0).stop 2024-04-23T18:57:38.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 -- 172.21.15.33:0/953392869 shutdown_connections 2024-04-23T18:57:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 --2- 172.21.15.33:0/953392869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1248106190 0x7f124810dd60 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 --2- 172.21.15.33:0/953392869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248101a70 0x7f1248105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 --2- 172.21.15.33:0/953392869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 0x7f12481014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 -- 172.21.15.33:0/953392869 >> 172.21.15.33:0/953392869 conn(0x7f1248077f60 msgr2=0x7f1248078370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 -- 172.21.15.33:0/953392869 shutdown_connections 2024-04-23T18:57:38.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.434+0000 7f124e54a640 1 -- 172.21.15.33:0/953392869 wait complete. 2024-04-23T18:57:38.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 Processor -- start 2024-04-23T18:57:38.522 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 -- start start 2024-04-23T18:57:38.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 0x7f1248198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248106190 0x7f1248198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12481110e0 con 0x7f124819cef0 2024-04-23T18:57:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1248110f60 con 0x7f12481010c0 2024-04-23T18:57:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124e54a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1248111260 con 0x7f1248106190 2024-04-23T18:57:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124cae7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124cae7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38322/0 (socket says 172.21.15.33:38322) 2024-04-23T18:57:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f124cae7640 1 -- 172.21.15.33:0/817584812 learned_addr learned my addr 172.21.15.33:0/817584812 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f1247fff640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 0x7f1248198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:39.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.435+0000 7f12477fe640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248106190 0x7f1248198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 -- 172.21.15.33:0/817584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248106190 msgr2=0x7f1248198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248106190 0x7f1248198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:39.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 -- 172.21.15.33:0/817584812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 msgr2=0x7f1248198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 0x7f1248198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 -- 172.21.15.33:0/817584812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124819d9e0 con 0x7f124819cef0 2024-04-23T18:57:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124cae7640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f1240005cc0 tx=0x7f124002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:39.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f12457fa640 1 -- 172.21.15.33:0/817584812 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240003bf0 con 0x7f124819cef0 2024-04-23T18:57:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124819dc70 con 0x7f124819cef0 2024-04-23T18:57:39.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f12457fa640 1 -- 172.21.15.33:0/817584812 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1240003d90 con 0x7f124819cef0 2024-04-23T18:57:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f12457fa640 1 -- 172.21.15.33:0/817584812 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124002fb20 con 0x7f124819cef0 2024-04-23T18:57:39.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.436+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12481a5490 con 0x7f124819cef0 2024-04-23T18:57:39.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.439+0000 7f12457fa640 1 -- 172.21.15.33:0/817584812 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1240004580 con 0x7f124819cef0 2024-04-23T18:57:39.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.439+0000 7f125125a740 1 Processor -- start 2024-04-23T18:57:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.439+0000 7f125125a740 1 -- start start 2024-04-23T18:57:40.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a419c72970 con 0x55a419d18800 2024-04-23T18:57:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a419c0f4a0 con 0x55a419d0df40 2024-04-23T18:57:40.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f125125a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a419c6e930 con 0x55a419d0d620 2024-04-23T18:57:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38336/0 (socket says 172.21.15.33:38336) 2024-04-23T18:57:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12477fe640 1 -- 172.21.15.33:0/2812883242 learned_addr learned my addr 172.21.15.33:0/2812883242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:40.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12477fe640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:40.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12457fa640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 0x7f12300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f124cae7640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 0x7f12300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:40.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 -- 172.21.15.33:0/2812883242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 msgr2=0x55a419d0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:40.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:40.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 -- 172.21.15.33:0/2812883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 msgr2=0x55a419d1abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:40.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:40.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f1247fff640 1 -- 172.21.15.33:0/2812883242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a419985210 con 0x55a419d0df40 2024-04-23T18:57:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12477fe640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.440+0000 7f12457fa640 1 -- 172.21.15.33:0/817584812 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f12400f1b60 con 0x7f124819cef0 2024-04-23T18:57:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f1247fff640 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d182c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f123800bbf0 tx=0x7f12380107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d559786a748f64d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:41.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f124cae7640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 0x7f12300a6280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12400ba080 tx=0x7f124009e040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f122e7fc640 1 -- 172.21.15.33:0/2812883242 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238004030 con 0x55a419d0df40 2024-04-23T18:57:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f122e7fc640 1 -- 172.21.15.33:0/2812883242 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12380035f0 con 0x55a419d0df40 2024-04-23T18:57:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f122e7fc640 1 -- 172.21.15.33:0/2812883242 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238011640 con 0x55a419d0df40 2024-04-23T18:57:41.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f125125a740 1 -- 172.21.15.33:0/2812883242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 msgr2=0x55a419d182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:41.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f125125a740 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d182c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f123800bbf0 tx=0x7f12380107a0 comp rx=0 tx=0).stop 2024-04-23T18:57:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.441+0000 7f125125a740 1 -- 172.21.15.33:0/2812883242 shutdown_connections 2024-04-23T18:57:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d182c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:41.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- 172.21.15.33:0/2812883242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 -- 172.21.15.33:0/2812883242 >> 172.21.15.33:0/2812883242 conn(0x55a419d09b30 msgr2=0x55a419d09f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:41.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 -- 172.21.15.33:0/2812883242 shutdown_connections 2024-04-23T18:57:41.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 -- 172.21.15.33:0/2812883242 wait complete. 2024-04-23T18:57:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 -- start start 2024-04-23T18:57:41.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 Processor -- start 2024-04-23T18:57:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d18190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:41.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.442+0000 7f125125a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d15a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f125125a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a419d1b430 con 0x55a419d0df40 2024-04-23T18:57:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f125125a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a419c0d0b0 con 0x55a419d18800 2024-04-23T18:57:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f125125a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a419d1b5b0 con 0x55a419d0d620 2024-04-23T18:57:41.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f125125a740 10 client.0 _ll_register_callbacks cb 0x55a419bfad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38338/0 (socket says 172.21.15.33:38338) 2024-04-23T18:57:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 -- 172.21.15.33:0/2269377133 learned_addr learned my addr 172.21.15.33:0/2269377133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12477fe640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d18190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12477fe640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d15a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 -- 172.21.15.33:0/2269377133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 msgr2=0x55a419d18190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d18190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 -- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 msgr2=0x55a419d15a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:42.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d15a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:42.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 -- 172.21.15.33:0/2269377133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a419a51620 con 0x55a419d0df40 2024-04-23T18:57:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12477fe640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f1247fff640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f123800bfd0 tx=0x7f123800b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0a7cf6e4fcedf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:42.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:42.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123802a060 con 0x55a419d0df40 2024-04-23T18:57:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f1238003a30 con 0x55a419d0df40 2024-04-23T18:57:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.443+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12380035f0 con 0x55a419d0df40 2024-04-23T18:57:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1238003fe0 con 0x55a419d0df40 2024-04-23T18:57:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 10 client.18594 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:42.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a419d273a0 con 0x55a419d0df40 2024-04-23T18:57:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12380035f0 con 0x55a419d0df40 2024-04-23T18:57:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f12380046c0 con 0x55a419d0df40 2024-04-23T18:57:42.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 10 client.18594 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 10 client.18594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:42.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 20 client.18594 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:42.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d28800 0x55a419d2abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 4 client.18594 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a419c80390 con 0x55a419d28800 2024-04-23T18:57:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.444+0000 7f12477fe640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d28800 0x55a419d2abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.445+0000 7f12277fe640 10 client.18594 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.445+0000 7f12477fe640 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d28800 0x55a419d2abf0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4629ba49a264c450 server_cookie=f1aec5b9cbb71fcf in_seq=0 out_seq=0 2024-04-23T18:57:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.445+0000 7f12277fe640 1 -- 172.21.15.33:0/2269377133 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a419c80390 con 0x55a419d28800 2024-04-23T18:57:43.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.445+0000 7f12277fe640 10 client.18594 handle_command_reply: tid=0 2024-04-23T18:57:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.445+0000 7f125125a740 1 client.18594 shutdown 2024-04-23T18:57:43.284 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 msgr2=0x55a419d15480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:43.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f123800bfd0 tx=0x7f123800b820 comp rx=0 tx=0).stop 2024-04-23T18:57:43.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 shutdown_connections 2024-04-23T18:57:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d28800 0x55a419d2abf0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:43.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0df40 0x55a419d15480 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:43.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 --2- 172.21.15.33:0/2269377133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d18190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f12277fe640 0 client.18594 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:43.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 >> 172.21.15.33:0/2269377133 conn(0x55a419d09b30 msgr2=0x55a419d0f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:43.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 shutdown_connections 2024-04-23T18:57:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 1 -- 172.21.15.33:0/2269377133 wait complete. 2024-04-23T18:57:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.446+0000 7f125125a740 20 client.18594 trim_cache size 0 max 16384 2024-04-23T18:57:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 Processor -- start 2024-04-23T18:57:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 -- start start 2024-04-23T18:57:44.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0df40 0x55a419d10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d17cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12481ac420 con 0x55a419d0d620 2024-04-23T18:57:44.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f12481aed00 con 0x55a419d18800 2024-04-23T18:57:44.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f125125a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12481acaf0 con 0x55a419d0df40 2024-04-23T18:57:44.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:44.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38344/0 (socket says 172.21.15.33:38344) 2024-04-23T18:57:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 -- 172.21.15.33:0/3211969413 learned_addr learned my addr 172.21.15.33:0/3211969413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 -- 172.21.15.33:0/3211969413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0df40 msgr2=0x55a419d10970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:44.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0df40 0x55a419d10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 -- 172.21.15.33:0/3211969413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 msgr2=0x55a419d17cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d17cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.461+0000 7f12477fe640 1 -- 172.21.15.33:0/3211969413 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a419d1afb0 con 0x55a419d0d620 2024-04-23T18:57:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f12477fe640 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f123c000c00 tx=0x7f123c00ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e93b5346c9a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f122dffb640 1 -- 172.21.15.33:0/3211969413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c029070 con 0x55a419d0d620 2024-04-23T18:57:44.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f122dffb640 1 -- 172.21.15.33:0/3211969413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c0248c0 con 0x55a419d0d620 2024-04-23T18:57:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f122dffb640 1 -- 172.21.15.33:0/3211969413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c004b50 con 0x55a419d0d620 2024-04-23T18:57:44.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 -- 172.21.15.33:0/3211969413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 msgr2=0x55a419d242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:44.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f123c000c00 tx=0x7f123c00ab20 comp rx=0 tx=0).stop 2024-04-23T18:57:44.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 -- 172.21.15.33:0/3211969413 shutdown_connections 2024-04-23T18:57:44.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d18800 0x55a419d17cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0df40 0x55a419d10970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 --2- 172.21.15.33:0/3211969413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d0d620 0x55a419d242f0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 -- 172.21.15.33:0/3211969413 >> 172.21.15.33:0/3211969413 conn(0x55a419d09b30 msgr2=0x55a419d10260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:44.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 -- 172.21.15.33:0/3211969413 shutdown_connections 2024-04-23T18:57:44.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.462+0000 7f125125a740 1 -- 172.21.15.33:0/3211969413 wait complete. 2024-04-23T18:57:44.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 -- start start 2024-04-23T18:57:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 Processor -- start 2024-04-23T18:57:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d204a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f12481ad1c0 con 0x55a419d18800 2024-04-23T18:57:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f12481ae630 con 0x55a419d0df40 2024-04-23T18:57:44.542 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12481ad890 con 0x55a419d0d620 2024-04-23T18:57:44.550 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f125125a740 10 client.0 _ll_register_callbacks cb 0x7f123800ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:44.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f12477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38352/0 (socket says 172.21.15.33:38352) 2024-04-23T18:57:44.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f12477fe640 1 -- 172.21.15.33:0/3986539072 learned_addr learned my addr 172.21.15.33:0/3986539072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:44.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f1247fff640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.463+0000 7f12477fe640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d204a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:44.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 -- 172.21.15.33:0/3986539072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 msgr2=0x55a419d204a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d204a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 -- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 msgr2=0x55a419d209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 -- 172.21.15.33:0/3986539072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a419d1c0d0 con 0x55a419d18800 2024-04-23T18:57:44.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12477fe640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f123c001eb0 tx=0x7f123c00a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b8b8114f98398b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c029070 con 0x55a419d18800 2024-04-23T18:57:44.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f123c0049d0 con 0x55a419d18800 2024-04-23T18:57:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f123c0248c0 con 0x55a419d18800 2024-04-23T18:57:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f123c004bd0 con 0x55a419d18800 2024-04-23T18:57:44.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f125125a740 10 client.18600 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:44.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a419d8e770 con 0x55a419d18800 2024-04-23T18:57:44.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f123c0248c0 con 0x55a419d18800 2024-04-23T18:57:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.464+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f123c02f070 con 0x55a419d18800 2024-04-23T18:57:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 10 client.18600 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 10 client.18600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 20 client.18600 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d8ecd0 0x55a419d910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:44.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 4 client.18600 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:44.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a419c7a200 con 0x55a419d8ecd0 2024-04-23T18:57:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f1247fff640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d8ecd0 0x55a419d910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:44.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f1247fff640 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d8ecd0 0x55a419d910f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c387fd1d1be29c6 server_cookie=2c84abdec24d3ac7 in_seq=0 out_seq=0 2024-04-23T18:57:44.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.465+0000 7f12277fe640 10 client.18600 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:44.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f12277fe640 1 -- 172.21.15.33:0/3986539072 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a419c7a200 con 0x55a419d8ecd0 2024-04-23T18:57:44.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f12277fe640 10 client.18600 handle_command_reply: tid=0 2024-04-23T18:57:44.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 client.18600 shutdown 2024-04-23T18:57:44.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 msgr2=0x55a419d1b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:44.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f123c001eb0 tx=0x7f123c00a400 comp rx=0 tx=0).stop 2024-04-23T18:57:44.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 shutdown_connections 2024-04-23T18:57:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55a419d8ecd0 0x55a419d910f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a419d18800 0x55a419d1b960 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a419d0df40 0x55a419d209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 --2- 172.21.15.33:0/3986539072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a419d0d620 0x55a419d204a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f12277fe640 0 client.18600 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:44.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.469+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 >> 172.21.15.33:0/3986539072 conn(0x55a419d09b30 msgr2=0x55a419d08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:44.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 shutdown_connections 2024-04-23T18:57:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f125125a740 1 -- 172.21.15.33:0/3986539072 wait complete. 2024-04-23T18:57:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f125125a740 20 client.18600 trim_cache size 0 max 16384 2024-04-23T18:57:45.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:45.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:45.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 msgr2=0x7f12300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:45.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 0x7f12300a6280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12400ba080 tx=0x7f124009e040 comp rx=0 tx=0).stop 2024-04-23T18:57:45.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 msgr2=0x7f124819d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:45.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f1240005cc0 tx=0x7f124002f380 comp rx=0 tx=0).stop 2024-04-23T18:57:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 shutdown_connections 2024-04-23T18:57:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f12300a3dc0 0x7f12300a6280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f124819cef0 0x7f124819d3a0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1248106190 0x7f1248198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 --2- 172.21.15.33:0/817584812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f12481010c0 0x7f1248198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 >> 172.21.15.33:0/817584812 conn(0x7f1248077f60 msgr2=0x7f124806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 shutdown_connections 2024-04-23T18:57:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:29.470+0000 7f124e54a640 1 -- 172.21.15.33:0/817584812 wait complete. 2024-04-23T18:57:45.220 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:29 smithi033 ceph-mon[29274]: pgmap v976: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:57:45.228 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:31 smithi033 ceph-mon[29274]: pgmap v977: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-23T18:57:45.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:33 smithi033 ceph-mon[29274]: pgmap v978: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T18:57:45.241 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:35 smithi033 ceph-mon[29274]: pgmap v979: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.248 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:37 smithi033 ceph-mon[29274]: pgmap v980: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:45.255 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:39 smithi033 ceph-mon[29274]: pgmap v981: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:41 smithi033 ceph-mon[29274]: pgmap v982: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.275 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:39 smithi106 ceph-mon[32716]: pgmap v1014: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:41 smithi106 ceph-mon[32716]: pgmap v1015: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.348 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:43 smithi106 ceph-mon[32716]: pgmap v1016: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:45.358 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:39 smithi122 ceph-mon[32546]: pgmap v1014: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.367 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:41 smithi122 ceph-mon[32546]: pgmap v1015: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:45.381 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:43 smithi122 ceph-mon[32546]: pgmap v1016: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:45.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.569+0000 7f558a0d0640 1 -- 172.21.15.33:0/515661530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5584107280 msgr2=0x7f558410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.569+0000 7f558a0d0640 1 --2- 172.21.15.33:0/515661530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5584107280 0x7f558410ee50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f557c009ca0 tx=0x7f557c02a8c0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 -- 172.21.15.33:0/515661530 shutdown_connections 2024-04-23T18:57:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 --2- 172.21.15.33:0/515661530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5584107280 0x7f558410ee50 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 --2- 172.21.15.33:0/515661530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584101070 0x7f5584106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 --2- 172.21.15.33:0/515661530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55841006c0 0x7f5584100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 -- 172.21.15.33:0/515661530 >> 172.21.15.33:0/515661530 conn(0x7f5584077390 msgr2=0x7f55840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 -- 172.21.15.33:0/515661530 shutdown_connections 2024-04-23T18:57:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 -- 172.21.15.33:0/515661530 wait complete. 2024-04-23T18:57:45.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 Processor -- start 2024-04-23T18:57:45.511 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.570+0000 7f558a0d0640 1 -- start start 2024-04-23T18:57:45.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584107280 0x7f5584198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558419cf60 0x7f558419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55841121f0 con 0x7f55841006c0 2024-04-23T18:57:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5584112070 con 0x7f558419cf60 2024-04-23T18:57:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f558a0d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5584112370 con 0x7f5584107280 2024-04-23T18:57:45.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:45.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52920/0 (socket says 172.21.15.33:52920) 2024-04-23T18:57:45.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 -- 172.21.15.33:0/3787363310 learned_addr learned my addr 172.21.15.33:0/3787363310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:45.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f5583fff640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558419cf60 0x7f558419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:45.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f5582ffd640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584107280 0x7f5584198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:45.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 -- 172.21.15.33:0/3787363310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584107280 msgr2=0x7f5584198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:45.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584107280 0x7f5584198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 -- 172.21.15.33:0/3787363310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558419cf60 msgr2=0x7f558419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:45.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558419cf60 0x7f558419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:45.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 -- 172.21.15.33:0/3787363310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558419da50 con 0x7f55841006c0 2024-04-23T18:57:45.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.571+0000 7f55837fe640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f557400e9f0 tx=0x7f557400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:45.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.572+0000 7f5580ff9640 1 -- 172.21.15.33:0/3787363310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557400cd80 con 0x7f55841006c0 2024-04-23T18:57:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.572+0000 7f5580ff9640 1 -- 172.21.15.33:0/3787363310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55740108e0 con 0x7f55841006c0 2024-04-23T18:57:45.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.572+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558419dd40 con 0x7f55841006c0 2024-04-23T18:57:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.572+0000 7f5580ff9640 1 -- 172.21.15.33:0/3787363310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574018980 con 0x7f55841006c0 2024-04-23T18:57:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.572+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55841a5580 con 0x7f55841006c0 2024-04-23T18:57:45.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f5580ff9640 1 -- 172.21.15.33:0/3787363310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5574010430 con 0x7f55841006c0 2024-04-23T18:57:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 Processor -- start 2024-04-23T18:57:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 -- start start 2024-04-23T18:57:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c96318a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318f40 0x556c963232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c9627d970 con 0x556c96323800 2024-04-23T18:57:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c9621a4a0 con 0x556c96318620 2024-04-23T18:57:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f558cdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c96279930 con 0x556c96318f40 2024-04-23T18:57:45.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52930/0 (socket says 172.21.15.33:52930) 2024-04-23T18:57:45.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.575+0000 7f5582ffd640 1 -- 172.21.15.33:0/1956068560 learned_addr learned my addr 172.21.15.33:0/1956068560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5580ff9640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 0x7f556c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:45.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f55837fe640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318f40 0x556c963232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5583fff640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c96318a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 -- 172.21.15.33:0/1956068560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318f40 msgr2=0x556c963232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318f40 0x556c963232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 -- 172.21.15.33:0/1956068560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 msgr2=0x556c96318a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c96318a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 -- 172.21.15.33:0/1956068560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c95f90210 con 0x556c96323800 2024-04-23T18:57:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5583fff640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c96318a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5583fff640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 0x7f556c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5580ff9640 1 -- 172.21.15.33:0/3787363310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f55740d0b70 con 0x7f55841006c0 2024-04-23T18:57:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f557800bbc0 tx=0x7f557800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c2784d4bc6da5d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:46.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5569ffb640 1 -- 172.21.15.33:0/1956068560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578015070 con 0x556c96323800 2024-04-23T18:57:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5569ffb640 1 -- 172.21.15.33:0/1956068560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55780054d0 con 0x556c96323800 2024-04-23T18:57:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.576+0000 7f5569ffb640 1 -- 172.21.15.33:0/1956068560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578011440 con 0x556c96323800 2024-04-23T18:57:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- 172.21.15.33:0/1956068560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 msgr2=0x556c96325bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f5583fff640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 0x7f556c0a6300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f557c006010 tx=0x7f557c02f3a0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f557800bbc0 tx=0x7f557800f730 comp rx=0 tx=0).stop 2024-04-23T18:57:46.149 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- 172.21.15.33:0/1956068560 shutdown_connections 2024-04-23T18:57:46.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96323800 0x556c96325bf0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318f40 0x556c963232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 --2- 172.21.15.33:0/1956068560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c96318a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- 172.21.15.33:0/1956068560 >> 172.21.15.33:0/1956068560 conn(0x556c96314b30 msgr2=0x556c96314f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- 172.21.15.33:0/1956068560 shutdown_connections 2024-04-23T18:57:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- 172.21.15.33:0/1956068560 wait complete. 2024-04-23T18:57:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.577+0000 7f558cdde740 1 -- start start 2024-04-23T18:57:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 Processor -- start 2024-04-23T18:57:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c963231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.278 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 0x556c96320a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556c96326430 con 0x556c96318f40 2024-04-23T18:57:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556c962180b0 con 0x556c96318620 2024-04-23T18:57:46.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556c963265b0 con 0x556c96323800 2024-04-23T18:57:46.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f558cdde740 10 client.0 _ll_register_callbacks cb 0x556c96205d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52934/0 (socket says 172.21.15.33:52934) 2024-04-23T18:57:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 -- 172.21.15.33:0/531628306 learned_addr learned my addr 172.21.15.33:0/531628306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f55837fe640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c963231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5583fff640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 0x556c96320a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 -- 172.21.15.33:0/531628306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 msgr2=0x556c96320a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 0x556c96320a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 -- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 msgr2=0x556c963231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c963231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5582ffd640 1 -- 172.21.15.33:0/531628306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c9605c620 con 0x556c96318f40 2024-04-23T18:57:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.578+0000 7f5583fff640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 0x556c96320a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5582ffd640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f55780057c0 tx=0x7f5578004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152122090ffec8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578028060 con 0x556c96318f40 2024-04-23T18:57:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f5578004c90 con 0x556c96318f40 2024-04-23T18:57:46.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55780054d0 con 0x556c96318f40 2024-04-23T18:57:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5578015070 con 0x556c96318f40 2024-04-23T18:57:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 10 client.18612 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:46.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c96332310 con 0x556c96318f40 2024-04-23T18:57:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55780054d0 con 0x556c96318f40 2024-04-23T18:57:46.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f5578011960 con 0x556c96318f40 2024-04-23T18:57:46.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 10 client.18612 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:46.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 20 client.18612 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:46.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96333770 0x556c96335b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 4 client.18612 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.579+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c9628b390 con 0x556c96333770 2024-04-23T18:57:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.580+0000 7f55837fe640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96333770 0x556c96335b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:46.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.580+0000 7f55837fe640 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96333770 0x556c96335b60 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a9b60b77369a18d server_cookie=ae57bd214b279035 in_seq=0 out_seq=0 2024-04-23T18:57:46.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.580+0000 7f5562ffd640 10 client.18612 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f5562ffd640 1 -- 172.21.15.33:0/531628306 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c9628b390 con 0x556c96333770 2024-04-23T18:57:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f5562ffd640 10 client.18612 handle_command_reply: tid=0 2024-04-23T18:57:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 client.18612 shutdown 2024-04-23T18:57:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 msgr2=0x556c96320490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f55780057c0 tx=0x7f5578004ba0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 shutdown_connections 2024-04-23T18:57:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96333770 0x556c96335b60 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96323800 0x556c96320a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320490 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f5562ffd640 0 client.18612 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:46.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 --2- 172.21.15.33:0/531628306 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96318620 0x556c963231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:46.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 >> 172.21.15.33:0/531628306 conn(0x556c96314b30 msgr2=0x556c9631a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:46.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 shutdown_connections 2024-04-23T18:57:46.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 1 -- 172.21.15.33:0/531628306 wait complete. 2024-04-23T18:57:46.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.581+0000 7f558cdde740 20 client.18612 trim_cache size 0 max 16384 2024-04-23T18:57:46.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 Processor -- start 2024-04-23T18:57:46.945 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 -- start start 2024-04-23T18:57:46.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c96322cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ac580 con 0x556c96318f40 2024-04-23T18:57:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55841aee60 con 0x556c96323800 2024-04-23T18:57:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.596+0000 7f558cdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55841acc50 con 0x556c96318620 2024-04-23T18:57:47.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52940/0 (socket says 172.21.15.33:52940) 2024-04-23T18:57:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 -- 172.21.15.33:0/1380148705 learned_addr learned my addr 172.21.15.33:0/1380148705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:47.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f55837fe640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5583fff640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c96322cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 -- 172.21.15.33:0/1380148705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 msgr2=0x556c9632f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:47.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 -- 172.21.15.33:0/1380148705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 msgr2=0x556c96322cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:47.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c96322cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 -- 172.21.15.33:0/1380148705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c96335ed0 con 0x556c96318f40 2024-04-23T18:57:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5583fff640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c96322cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f5582ffd640 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5578011f20 tx=0x7f557800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e9acb60aee6a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:47.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f55697fa640 1 -- 172.21.15.33:0/1380148705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578028040 con 0x556c96318f40 2024-04-23T18:57:47.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f55697fa640 1 -- 172.21.15.33:0/1380148705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5578004640 con 0x556c96318f40 2024-04-23T18:57:47.201 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.597+0000 7f55697fa640 1 -- 172.21.15.33:0/1380148705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55780116a0 con 0x556c96318f40 2024-04-23T18:57:47.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- 172.21.15.33:0/1380148705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 msgr2=0x556c96320430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5578011f20 tx=0x7f557800fc30 comp rx=0 tx=0).stop 2024-04-23T18:57:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- 172.21.15.33:0/1380148705 shutdown_connections 2024-04-23T18:57:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c96322cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96320430 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 --2- 172.21.15.33:0/1380148705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- 172.21.15.33:0/1380148705 >> 172.21.15.33:0/1380148705 conn(0x556c96314b30 msgr2=0x556c9631fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- 172.21.15.33:0/1380148705 shutdown_connections 2024-04-23T18:57:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- 172.21.15.33:0/1380148705 wait complete. 2024-04-23T18:57:47.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 -- start start 2024-04-23T18:57:47.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.598+0000 7f558cdde740 1 Processor -- start 2024-04-23T18:57:47.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:47.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:47.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c963273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:47.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ad320 con 0x556c96318f40 2024-04-23T18:57:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ae790 con 0x556c96323800 2024-04-23T18:57:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55841ad9f0 con 0x556c96318620 2024-04-23T18:57:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f558cdde740 10 client.0 _ll_register_callbacks cb 0x7f55780048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52946/0 (socket says 172.21.15.33:52946) 2024-04-23T18:57:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 -- 172.21.15.33:0/2877860788 learned_addr learned my addr 172.21.15.33:0/2877860788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:47.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f55837fe640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5583fff640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c963273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 -- 172.21.15.33:0/2877860788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 msgr2=0x556c9632bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:47.526 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 -- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 msgr2=0x556c963273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:47.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c963273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 -- 172.21.15.33:0/2877860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c96320da0 con 0x556c96318f40 2024-04-23T18:57:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f55837fe640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5582ffd640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5578000c00 tx=0x7f55780049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b2e33847ff933 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:47.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.599+0000 7f5562ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5578028040 con 0x556c96318f40 2024-04-23T18:57:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7f557800ddc0 con 0x556c96318f40 2024-04-23T18:57:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5578004640 con 0x556c96318f40 2024-04-23T18:57:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5578011440 con 0x556c96318f40 2024-04-23T18:57:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 10 client.18618 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c96399970 con 0x556c96318f40 2024-04-23T18:57:47.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5578004640 con 0x556c96318f40 2024-04-23T18:57:47.738 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f5578048a10 con 0x556c96318f40 2024-04-23T18:57:47.747 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 10 client.18618 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 10 client.18618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 20 client.18618 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:47.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96399f80 0x556c9639c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 4 client.18618 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:47.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c96285200 con 0x556c96399f80 2024-04-23T18:57:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.600+0000 7f55837fe640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96399f80 0x556c9639c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:47.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.601+0000 7f55837fe640 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96399f80 0x556c9639c3a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44c6de47a571bce server_cookie=44ff0181ef7d42f3 in_seq=0 out_seq=0 2024-04-23T18:57:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.601+0000 7f5562ffd640 10 client.18618 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f5562ffd640 1 -- 172.21.15.33:0/2877860788 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c96285200 con 0x556c96399f80 2024-04-23T18:57:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f5562ffd640 10 client.18618 handle_command_reply: tid=0 2024-04-23T18:57:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 client.18618 shutdown 2024-04-23T18:57:47.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 msgr2=0x556c96326e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5578000c00 tx=0x7f55780049e0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 shutdown_connections 2024-04-23T18:57:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x556c96399f80 0x556c9639c3a0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f5562ffd640 0 client.18618 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556c96323800 0x556c963273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556c96318f40 0x556c96326e80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 --2- 172.21.15.33:0/2877860788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556c96318620 0x556c9632bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:47.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 >> 172.21.15.33:0/2877860788 conn(0x556c96314b30 msgr2=0x556c96313cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:47.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 shutdown_connections 2024-04-23T18:57:47.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 1 -- 172.21.15.33:0/2877860788 wait complete. 2024-04-23T18:57:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.605+0000 7f558cdde740 20 client.18618 trim_cache size 0 max 16384 2024-04-23T18:57:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 msgr2=0x7f556c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 0x7f556c0a6300 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f557c006010 tx=0x7f557c02f3a0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 msgr2=0x7f55841986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f557400e9f0 tx=0x7f557400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 shutdown_connections 2024-04-23T18:57:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f556c0a3e40 0x7f556c0a6300 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f558419cf60 0x7f558419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5584107280 0x7f5584198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 --2- 172.21.15.33:0/3787363310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55841006c0 0x7f55841986f0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 >> 172.21.15.33:0/3787363310 conn(0x7f5584077390 msgr2=0x7f55840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 shutdown_connections 2024-04-23T18:57:48.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:42.606+0000 7f558a0d0640 1 -- 172.21.15.33:0/3787363310 wait complete. 2024-04-23T18:57:48.235 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:43 smithi033 ceph-mon[29274]: pgmap v983: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:48.244 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:45 smithi033 ceph-mon[29274]: pgmap v984: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:48.252 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:47 smithi033 ceph-mon[29274]: pgmap v985: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:48.261 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:49 smithi033 ceph-mon[29274]: pgmap v986: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:48.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:51 smithi033 ceph-mon[29274]: pgmap v987: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:48.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:53 smithi033 ceph-mon[29274]: pgmap v988: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:48.288 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:45 smithi106 ceph-mon[32716]: pgmap v1017: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:48.296 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:47 smithi106 ceph-mon[32716]: pgmap v1018: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:48.370 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:45 smithi122 ceph-mon[32546]: pgmap v1017: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:48.379 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:47 smithi122 ceph-mon[32546]: pgmap v1018: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 -- 172.21.15.33:0/1990634737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50112f60 msgr2=0x7fed50115350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 --2- 172.21.15.33:0/1990634737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50112f60 0x7fed50115350 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fed40009ca0 tx=0x7fed4002a950 comp rx=0 tx=0).stop 2024-04-23T18:57:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 -- 172.21.15.33:0/1990634737 shutdown_connections 2024-04-23T18:57:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 --2- 172.21.15.33:0/1990634737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50112f60 0x7fed50115350 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 --2- 172.21.15.33:0/1990634737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed50074c10 0x7fed50112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.441 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 --2- 172.21.15.33:0/1990634737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed500742f0 0x7fed500746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 -- 172.21.15.33:0/1990634737 >> 172.21.15.33:0/1990634737 conn(0x7fed500fda20 msgr2=0x7fed500ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 -- 172.21.15.33:0/1990634737 shutdown_connections 2024-04-23T18:57:48.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 -- 172.21.15.33:0/1990634737 wait complete. 2024-04-23T18:57:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.707+0000 7fed57584640 1 Processor -- start 2024-04-23T18:57:48.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 -- start start 2024-04-23T18:57:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed500742f0 0x7fed501a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed50112f60 0x7fed5019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed501187d0 con 0x7fed50074c10 2024-04-23T18:57:48.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed50118650 con 0x7fed50112f60 2024-04-23T18:57:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed57584640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed50118950 con 0x7fed500742f0 2024-04-23T18:57:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56602/0 (socket says 172.21.15.33:56602) 2024-04-23T18:57:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 -- 172.21.15.33:0/149139985 learned_addr learned my addr 172.21.15.33:0/149139985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:48.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed55320640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed500742f0 0x7fed501a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed55b21640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed50112f60 0x7fed5019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 -- 172.21.15.33:0/149139985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed500742f0 msgr2=0x7fed501a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed500742f0 0x7fed501a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 -- 172.21.15.33:0/149139985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed50112f60 msgr2=0x7fed5019eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:48.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed50112f60 0x7fed5019eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.708+0000 7fed54b1f640 1 -- 172.21.15.33:0/149139985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5019f270 con 0x7fed50074c10 2024-04-23T18:57:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed54b1f640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e9f0 tx=0x7fed4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:48.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed4e7fc640 1 -- 172.21.15.33:0/149139985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4400cd80 con 0x7fed50074c10 2024-04-23T18:57:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed4e7fc640 1 -- 172.21.15.33:0/149139985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed440054d0 con 0x7fed50074c10 2024-04-23T18:57:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5019f560 con 0x7fed50074c10 2024-04-23T18:57:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed4e7fc640 1 -- 172.21.15.33:0/149139985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44010690 con 0x7fed50074c10 2024-04-23T18:57:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.709+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed501ab7b0 con 0x7fed50074c10 2024-04-23T18:57:48.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.711+0000 7fed4e7fc640 1 -- 172.21.15.33:0/149139985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed44004e90 con 0x7fed50074c10 2024-04-23T18:57:48.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 Processor -- start 2024-04-23T18:57:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 -- start start 2024-04-23T18:57:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e6aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5634d2dcf970 con 0x5634d2e75800 2024-04-23T18:57:48.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634d2d6c4a0 con 0x5634d2e6a620 2024-04-23T18:57:48.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed5a378740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634d2dcb930 con 0x5634d2e6af40 2024-04-23T18:57:48.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed4e7fc640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 0x7fed380a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:48.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed54b1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed54b1f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56608/0 (socket says 172.21.15.33:56608) 2024-04-23T18:57:48.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed54b1f640 1 -- 172.21.15.33:0/257257974 learned_addr learned my addr 172.21.15.33:0/257257974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:48.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed55b21640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e6aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed55b21640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 0x7fed380a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed55320640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:48.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.712+0000 7fed4e7fc640 1 -- 172.21.15.33:0/149139985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7fed440d0c30 con 0x7fed50074c10 2024-04-23T18:57:48.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 -- 172.21.15.33:0/257257974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 msgr2=0x5634d2e752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:48.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 -- 172.21.15.33:0/257257974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 msgr2=0x5634d2e6aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e6aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:48.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 -- 172.21.15.33:0/257257974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d2ae2210 con 0x5634d2e75800 2024-04-23T18:57:48.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed55b21640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed54b1f640 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fed440ce3f0 tx=0x7fed44004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dd2d6c7e3897 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:49.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed55b21640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 0x7fed380a6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed40004af0 tx=0x7fed400095c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:49.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed377fe640 1 -- 172.21.15.33:0/257257974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44005180 con 0x5634d2e75800 2024-04-23T18:57:49.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed377fe640 1 -- 172.21.15.33:0/257257974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed440a5850 con 0x5634d2e75800 2024-04-23T18:57:49.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.713+0000 7fed377fe640 1 -- 172.21.15.33:0/257257974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440046c0 con 0x5634d2e75800 2024-04-23T18:57:49.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- 172.21.15.33:0/257257974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 msgr2=0x5634d2e77bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:49.057 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fed440ce3f0 tx=0x7fed44004a90 comp rx=0 tx=0).stop 2024-04-23T18:57:49.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- 172.21.15.33:0/257257974 shutdown_connections 2024-04-23T18:57:49.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e77bf0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 --2- 172.21.15.33:0/257257974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e6aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- 172.21.15.33:0/257257974 >> 172.21.15.33:0/257257974 conn(0x5634d2e66b30 msgr2=0x5634d2e66f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- 172.21.15.33:0/257257974 shutdown_connections 2024-04-23T18:57:49.200 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- 172.21.15.33:0/257257974 wait complete. 2024-04-23T18:57:49.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 -- start start 2024-04-23T18:57:49.219 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.714+0000 7fed5a378740 1 Processor -- start 2024-04-23T18:57:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.237 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e72410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e75800 0x5634d2e72950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5634d2e783d0 con 0x5634d2e6a620 2024-04-23T18:57:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5634d2d6a0b0 con 0x5634d2e75800 2024-04-23T18:57:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634d2e78550 con 0x5634d2e6af40 2024-04-23T18:57:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed5a378740 10 client.0 _ll_register_callbacks cb 0x5634d2d57d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56616/0 (socket says 172.21.15.33:56616) 2024-04-23T18:57:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 -- 172.21.15.33:0/3909066290 learned_addr learned my addr 172.21.15.33:0/3909066290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed54b1f640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e72410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55b21640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e75800 0x5634d2e72950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 -- 172.21.15.33:0/3909066290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 msgr2=0x5634d2e72410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:49.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e72410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 -- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e75800 msgr2=0x5634d2e72950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:49.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e75800 0x5634d2e72950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 -- 172.21.15.33:0/3909066290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d2bae620 con 0x5634d2e6a620 2024-04-23T18:57:49.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed55320640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fed48012700 tx=0x7fed48012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136ee7dadeb46be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:49.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.715+0000 7fed34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:49.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48004260 con 0x5634d2e6a620 2024-04-23T18:57:49.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7fed48004400 con 0x5634d2e6a620 2024-04-23T18:57:49.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5634d2ddd390 con 0x5634d2e6a620 2024-04-23T18:57:49.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fed480119a0 con 0x5634d2e6a620 2024-04-23T18:57:49.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 10 client.18630 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5634d2e843a0 con 0x5634d2e6a620 2024-04-23T18:57:49.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5634d2ddd390 con 0x5634d2e6a620 2024-04-23T18:57:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fed48025470 con 0x5634d2e6a620 2024-04-23T18:57:49.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 10 client.18630 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:49.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 10 client.18630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:49.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 20 client.18630 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:49.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2e85810 0x5634d2e87c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 4 client.18630 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:49.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.716+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634d2dd7200 con 0x5634d2e85810 2024-04-23T18:57:49.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.717+0000 7fed54b1f640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2e85810 0x5634d2e87c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:49.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.717+0000 7fed54b1f640 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2e85810 0x5634d2e87c20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b898762f60a822b server_cookie=1aab3bc6551637e3 in_seq=0 out_seq=0 2024-04-23T18:57:49.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.717+0000 7fed34ff9640 10 client.18630 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:49.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed34ff9640 1 -- 172.21.15.33:0/3909066290 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5634d2dd7200 con 0x5634d2e85810 2024-04-23T18:57:49.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed34ff9640 10 client.18630 handle_command_reply: tid=0 2024-04-23T18:57:49.672 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 client.18630 shutdown 2024-04-23T18:57:49.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 msgr2=0x5634d2e75120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:49.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fed48012700 tx=0x7fed48012bc0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 shutdown_connections 2024-04-23T18:57:49.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2e85810 0x5634d2e87c20 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e75800 0x5634d2e72950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e72410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed34ff9640 0 client.18630 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 --2- 172.21.15.33:0/3909066290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e75120 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 >> 172.21.15.33:0/3909066290 conn(0x5634d2e66b30 msgr2=0x5634d2e6c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 shutdown_connections 2024-04-23T18:57:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 1 -- 172.21.15.33:0/3909066290 wait complete. 2024-04-23T18:57:49.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.718+0000 7fed5a378740 20 client.18630 trim_cache size 0 max 16384 2024-04-23T18:57:49.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 Processor -- start 2024-04-23T18:57:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 -- start start 2024-04-23T18:57:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e74c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e8d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:49.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b2bd0 con 0x5634d2e75800 2024-04-23T18:57:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b54b0 con 0x5634d2e6a620 2024-04-23T18:57:49.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed5a378740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b32a0 con 0x5634d2e6af40 2024-04-23T18:57:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.733+0000 7fed55b21640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:49.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56620/0 (socket says 172.21.15.33:56620) 2024-04-23T18:57:49.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 -- 172.21.15.33:0/2631815626 learned_addr learned my addr 172.21.15.33:0/2631815626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:50.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed54b1f640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e8d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55320640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e74c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 -- 172.21.15.33:0/2631815626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 msgr2=0x5634d2e8d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:50.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e8d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 -- 172.21.15.33:0/2631815626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 msgr2=0x5634d2e74c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e74c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 -- 172.21.15.33:0/2631815626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634d2e72cb0 con 0x5634d2e75800 2024-04-23T18:57:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55320640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e74c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:50.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed54b1f640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e8d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:50.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed55b21640 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fed40009ca0 tx=0x7fed40042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210d429c56ebba4b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed36ffd640 1 -- 172.21.15.33:0/2631815626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40034040 con 0x5634d2e75800 2024-04-23T18:57:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed36ffd640 1 -- 172.21.15.33:0/2631815626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40003b70 con 0x5634d2e75800 2024-04-23T18:57:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.734+0000 7fed36ffd640 1 -- 172.21.15.33:0/2631815626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4004c6a0 con 0x5634d2e75800 2024-04-23T18:57:50.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- 172.21.15.33:0/2631815626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 msgr2=0x5634d2e8da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fed40009ca0 tx=0x7fed40042000 comp rx=0 tx=0).stop 2024-04-23T18:57:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- 172.21.15.33:0/2631815626 shutdown_connections 2024-04-23T18:57:50.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e75800 0x5634d2e8da70 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.279 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e6af40 0x5634d2e8d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 --2- 172.21.15.33:0/2631815626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6a620 0x5634d2e74c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- 172.21.15.33:0/2631815626 >> 172.21.15.33:0/2631815626 conn(0x5634d2e66b30 msgr2=0x5634d2e65cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:50.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- 172.21.15.33:0/2631815626 shutdown_connections 2024-04-23T18:57:50.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- 172.21.15.33:0/2631815626 wait complete. 2024-04-23T18:57:50.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 -- start start 2024-04-23T18:57:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.735+0000 7fed5a378740 1 Processor -- start 2024-04-23T18:57:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:50.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 0x5634d2e7adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e75800 0x5634d2e7b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b3970 con 0x5634d2e6a620 2024-04-23T18:57:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b4de0 con 0x5634d2e6af40 2024-04-23T18:57:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed501b4040 con 0x5634d2e75800 2024-04-23T18:57:50.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed5a378740 10 client.0 _ll_register_callbacks cb 0x7fed48009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:50.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:50.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56630/0 (socket says 172.21.15.33:56630) 2024-04-23T18:57:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 -- 172.21.15.33:0/1615800653 learned_addr learned my addr 172.21.15.33:0/1615800653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed54b1f640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 0x5634d2e7adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 -- 172.21.15.33:0/1615800653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e75800 msgr2=0x5634d2e7b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e75800 0x5634d2e7b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 -- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 msgr2=0x5634d2e7adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 0x5634d2e7adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 -- 172.21.15.33:0/1615800653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634d2e7b930 con 0x5634d2e6a620 2024-04-23T18:57:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed54b1f640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 0x5634d2e7adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed55320640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fed4800a010 tx=0x7fed48025d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbb8cfca55da0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:50.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.736+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48026e60 con 0x5634d2e6a620 2024-04-23T18:57:50.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(148..148 src has 1..148) ==== 8498+0+0 (secure 0 0 0) 0x7fed48018050 con 0x5634d2e6a620 2024-04-23T18:57:50.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed40003b70 con 0x5634d2e6a620 2024-04-23T18:57:50.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fed480054e0 con 0x5634d2e6a620 2024-04-23T18:57:50.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 10 client.18636 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:50.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5634d2eebc30 con 0x5634d2e6a620 2024-04-23T18:57:50.626 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40003b70 con 0x5634d2e6a620 2024-04-23T18:57:50.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:55 smithi033 ceph-mon[29274]: pgmap v989: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:50.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fed48023510 con 0x5634d2e6a620 2024-04-23T18:57:50.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 10 client.18636 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:50.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:50.777 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 20 client.18636 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:50.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2eec300 0x5634d2eee740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 4 client.18636 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634d2d535f0 con 0x5634d2eec300 2024-04-23T18:57:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.737+0000 7fed54b1f640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2eec300 0x5634d2eee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.738+0000 7fed54b1f640 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2eec300 0x5634d2eee740 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=558a4cae7f3892b server_cookie=56b711543eca9f79 in_seq=0 out_seq=0 2024-04-23T18:57:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.738+0000 7fed34ff9640 10 client.18636 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed34ff9640 1 -- 172.21.15.33:0/1615800653 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634d2d535f0 con 0x5634d2eec300 2024-04-23T18:57:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed34ff9640 10 client.18636 handle_command_reply: tid=0 2024-04-23T18:57:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 client.18636 shutdown 2024-04-23T18:57:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 msgr2=0x5634d2e7d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:50.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fed4800a010 tx=0x7fed48025d30 comp rx=0 tx=0).stop 2024-04-23T18:57:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 shutdown_connections 2024-04-23T18:57:50.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5634d2eec300 0x5634d2eee740 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634d2e75800 0x5634d2e7b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5634d2e6af40 0x5634d2e7adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed34ff9640 0 client.18636 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:51.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 --2- 172.21.15.33:0/1615800653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5634d2e6a620 0x5634d2e7d9d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 >> 172.21.15.33:0/1615800653 conn(0x5634d2e66b30 msgr2=0x5634d2e65420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 shutdown_connections 2024-04-23T18:57:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.742+0000 7fed5a378740 1 -- 172.21.15.33:0/1615800653 wait complete. 2024-04-23T18:57:51.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed5a378740 20 client.18636 trim_cache size 0 max 16384 2024-04-23T18:57:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:51.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 msgr2=0x7fed380a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 0x7fed380a6300 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed40004af0 tx=0x7fed400095c0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 msgr2=0x7fed501a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:51.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fed4400e9f0 tx=0x7fed4400edf0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 shutdown_connections 2024-04-23T18:57:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fed380a3e40 0x7fed380a6300 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed50112f60 0x7fed5019eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed50074c10 0x7fed501a4e30 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 --2- 172.21.15.33:0/149139985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed500742f0 0x7fed501a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 >> 172.21.15.33:0/149139985 conn(0x7fed500fda20 msgr2=0x7fed500ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:51.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 shutdown_connections 2024-04-23T18:57:51.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:56:55.743+0000 7fed57584640 1 -- 172.21.15.33:0/149139985 wait complete. 2024-04-23T18:57:51.248 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:49 smithi106 ceph-mon[32716]: pgmap v1019: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:51.256 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:49 smithi122 ceph-mon[32546]: pgmap v1019: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:51.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:57 smithi033 ceph-mon[29274]: pgmap v990: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:51.273 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:56:59 smithi033 ceph-mon[29274]: pgmap v991: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:51.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:01 smithi033 ceph-mon[29274]: pgmap v992: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:51.290 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:03 smithi033 ceph-mon[29274]: pgmap v993: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:51.301 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:05 smithi033 ceph-mon[29274]: pgmap v994: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:51.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:06 smithi033 ceph-mon[29274]: pgmap v995: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 30 op/s 2024-04-23T18:57:51.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 -- 172.21.15.33:0/3148141663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c101aa0 msgr2=0x7f939c105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:51.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 --2- 172.21.15.33:0/3148141663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c101aa0 0x7f939c105a90 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9398009900 tx=0x7f939802a410 comp rx=0 tx=0).stop 2024-04-23T18:57:51.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 -- 172.21.15.33:0/3148141663 shutdown_connections 2024-04-23T18:57:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 --2- 172.21.15.33:0/3148141663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c106170 0x7f939c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 --2- 172.21.15.33:0/3148141663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c101aa0 0x7f939c105a90 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 --2- 172.21.15.33:0/3148141663 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c1010f0 0x7f939c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.843+0000 7f93a33ba640 1 -- 172.21.15.33:0/3148141663 >> 172.21.15.33:0/3148141663 conn(0x7f939c077fd0 msgr2=0x7f939c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:51.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.844+0000 7f93a33ba640 1 -- 172.21.15.33:0/3148141663 shutdown_connections 2024-04-23T18:57:51.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.844+0000 7f93a33ba640 1 -- 172.21.15.33:0/3148141663 wait complete. 2024-04-23T18:57:51.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.844+0000 7f93a33ba640 1 Processor -- start 2024-04-23T18:57:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.844+0000 7f93a33ba640 1 -- start start 2024-04-23T18:57:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.844+0000 7f93a33ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c1010f0 0x7f939c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 0x7f939c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111090 con 0x7f939c106170 2024-04-23T18:57:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f939c110f10 con 0x7f939c1010f0 2024-04-23T18:57:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f939c111210 con 0x7f939c19cf90 2024-04-23T18:57:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56356/0 (socket says 172.21.15.33:56356) 2024-04-23T18:57:51.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 -- 172.21.15.33:0/17712891 learned_addr learned my addr 172.21.15.33:0/17712891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a1156640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c1010f0 0x7f939c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a1957640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 0x7f939c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 -- 172.21.15.33:0/17712891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 msgr2=0x7f939c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 0x7f939c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 -- 172.21.15.33:0/17712891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c1010f0 msgr2=0x7f939c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:51.736 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c1010f0 0x7f939c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:51.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 -- 172.21.15.33:0/17712891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c19da80 con 0x7f939c106170 2024-04-23T18:57:51.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a1957640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 0x7f939c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:51.759 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a0955640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9398032fd0 tx=0x7f9398004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:51.766 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93927fc640 1 -- 172.21.15.33:0/17712891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939803f030 con 0x7f939c106170 2024-04-23T18:57:51.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93927fc640 1 -- 172.21.15.33:0/17712891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9398004490 con 0x7f939c106170 2024-04-23T18:57:51.780 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c19dd10 con 0x7f939c106170 2024-04-23T18:57:51.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.845+0000 7f93927fc640 1 -- 172.21.15.33:0/17712891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398035070 con 0x7f939c106170 2024-04-23T18:57:51.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.846+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c1a5530 con 0x7f939c106170 2024-04-23T18:57:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93927fc640 1 -- 172.21.15.33:0/17712891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9398042030 con 0x7f939c106170 2024-04-23T18:57:51.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 Processor -- start 2024-04-23T18:57:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 -- start start 2024-04-23T18:57:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 0x55ae4569e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 0x55ae456b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae45603970 con 0x55ae4569ee40 2024-04-23T18:57:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ae455a04a0 con 0x55ae4569e520 2024-04-23T18:57:51.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.848+0000 7f93a61a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae455ff930 con 0x55ae456adc10 2024-04-23T18:57:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93927fc640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 0x7f93880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56372/0 (socket says 172.21.15.33:56372) 2024-04-23T18:57:51.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 -- 172.21.15.33:0/3910918223 learned_addr learned my addr 172.21.15.33:0/3910918223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1957640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 0x7f93880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a0955640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 0x55ae456b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1957640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 0x55ae4569e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93927fc640 1 -- 172.21.15.33:0/17712891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 8586+0+0 (secure 0 0 0) 0x7f93980f0c90 con 0x7f939c106170 2024-04-23T18:57:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 -- 172.21.15.33:0/3910918223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 msgr2=0x55ae456b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 0x55ae456b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 -- 172.21.15.33:0/3910918223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 msgr2=0x55ae4569e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:52.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 0x55ae4569e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 -- 172.21.15.33:0/3910918223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae45316210 con 0x55ae4569ee40 2024-04-23T18:57:52.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1957640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 0x55ae4569e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:52.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a0955640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 0x55ae456b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:52.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1957640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 0x7f93880a6300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9394005c40 tx=0x7f93940094a0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:52.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93a1156640 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f938c00ac20 tx=0x7f938c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a993b3c319b17a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:52.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.849+0000 7f93877fe640 1 -- 172.21.15.33:0/3910918223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c01d070 con 0x55ae4569ee40 2024-04-23T18:57:52.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93877fe640 1 -- 172.21.15.33:0/3910918223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c018af0 con 0x55ae4569ee40 2024-04-23T18:57:52.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93877fe640 1 -- 172.21.15.33:0/3910918223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c010a40 con 0x55ae4569ee40 2024-04-23T18:57:52.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 -- 172.21.15.33:0/3910918223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 msgr2=0x55ae456ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:52.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f938c00ac20 tx=0x7f938c00f790 comp rx=0 tx=0).stop 2024-04-23T18:57:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 -- 172.21.15.33:0/3910918223 shutdown_connections 2024-04-23T18:57:52.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae456adc10 0x55ae456b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456ad6d0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3910918223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569e520 0x55ae4569e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 -- 172.21.15.33:0/3910918223 >> 172.21.15.33:0/3910918223 conn(0x55ae4569ab30 msgr2=0x55ae4569af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 -- 172.21.15.33:0/3910918223 shutdown_connections 2024-04-23T18:57:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.850+0000 7f93a61a8740 1 -- 172.21.15.33:0/3910918223 wait complete. 2024-04-23T18:57:52.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 -- start start 2024-04-23T18:57:52.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 Processor -- start 2024-04-23T18:57:52.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 0x55ae456aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae456b0840 con 0x55ae456adc10 2024-04-23T18:57:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ae4559e0b0 con 0x55ae4569ee40 2024-04-23T18:57:52.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ae456b09c0 con 0x55ae4569e520 2024-04-23T18:57:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a61a8740 10 client.0 _ll_register_callbacks cb 0x55ae4558bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a1957640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a1957640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56380/0 (socket says 172.21.15.33:56380) 2024-04-23T18:57:52.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a1957640 1 -- 172.21.15.33:0/3081015999 learned_addr learned my addr 172.21.15.33:0/3081015999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a0955640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 0x55ae456aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:52.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.851+0000 7f93a1156640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:52.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 -- 172.21.15.33:0/3081015999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 msgr2=0x55ae456ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:52.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 -- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 msgr2=0x55ae456aa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 0x55ae456aa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 -- 172.21.15.33:0/3081015999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae453e2620 con 0x55ae456adc10 2024-04-23T18:57:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a0955640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 0x55ae456aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:52.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a1957640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f9394014fc0 tx=0x7f9394004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10912c9813c8fe95 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:52.606 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f9384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394024030 con 0x55ae456adc10 2024-04-23T18:57:52.623 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 8586+0+0 (secure 0 0 0) 0x7f9394012970 con 0x55ae456adc10 2024-04-23T18:57:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c018af0 con 0x55ae456adc10 2024-04-23T18:57:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9394012b70 con 0x55ae456adc10 2024-04-23T18:57:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a61a8740 10 client.18648 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ae456bc730 con 0x55ae456adc10 2024-04-23T18:57:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.852+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c018af0 con 0x55ae456adc10 2024-04-23T18:57:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f939401d050 con 0x55ae456adc10 2024-04-23T18:57:52.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 10 client.18648 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 10 client.18648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:52.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 20 client.18648 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:52.765 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae456bdc40 0x55ae456c0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:52.774 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 4 client.18648 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae45611390 con 0x55ae456bdc40 2024-04-23T18:57:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a1156640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae456bdc40 0x55ae456c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f93a1156640 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae456bdc40 0x55ae456c0030 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62474d8de54d8074 server_cookie=572f0c24c6d1d1b9 in_seq=0 out_seq=0 2024-04-23T18:57:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.853+0000 7f9384ff9640 10 client.18648 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f9384ff9640 1 -- 172.21.15.33:0/3081015999 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae45611390 con 0x55ae456bdc40 2024-04-23T18:57:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f9384ff9640 10 client.18648 handle_command_reply: tid=0 2024-04-23T18:57:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 client.18648 shutdown 2024-04-23T18:57:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 msgr2=0x55ae456aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f9394014fc0 tx=0x7f9394004ab0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 shutdown_connections 2024-04-23T18:57:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae456bdc40 0x55ae456c0030 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae456adc10 0x55ae456aad10 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae4569ee40 0x55ae456aa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f9384ff9640 0 client.18648 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 --2- 172.21.15.33:0/3081015999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:52.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.854+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 >> 172.21.15.33:0/3081015999 conn(0x55ae4569ab30 msgr2=0x55ae456a0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.855+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 shutdown_connections 2024-04-23T18:57:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.855+0000 7f93a61a8740 1 -- 172.21.15.33:0/3081015999 wait complete. 2024-04-23T18:57:53.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.855+0000 7f93a61a8740 20 client.18648 trim_cache size 0 max 16384 2024-04-23T18:57:53.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 Processor -- start 2024-04-23T18:57:53.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 -- start start 2024-04-23T18:57:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569ee40 0x55ae456a5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae456ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1aa4e0 con 0x55ae4569e520 2024-04-23T18:57:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1acdc0 con 0x55ae456adc10 2024-04-23T18:57:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a61a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1aabb0 con 0x55ae4569ee40 2024-04-23T18:57:53.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56394/0 (socket says 172.21.15.33:56394) 2024-04-23T18:57:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 -- 172.21.15.33:0/169341499 learned_addr learned my addr 172.21.15.33:0/169341499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 -- 172.21.15.33:0/169341499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569ee40 msgr2=0x55ae456a5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1957640 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae456ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569ee40 0x55ae456a5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 -- 172.21.15.33:0/169341499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 msgr2=0x55ae456ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae456ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1156640 1 -- 172.21.15.33:0/169341499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4569ec70 con 0x55ae4569e520 2024-04-23T18:57:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.870+0000 7f93a1957640 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae456ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:53.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a1156640 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f938c0105b0 tx=0x7f938c0105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a614c328a58817 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:53.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f9386ffd640 1 -- 172.21.15.33:0/169341499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c030070 con 0x55ae4569e520 2024-04-23T18:57:53.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f9386ffd640 1 -- 172.21.15.33:0/169341499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c018440 con 0x55ae4569e520 2024-04-23T18:57:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f9386ffd640 1 -- 172.21.15.33:0/169341499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c01d040 con 0x55ae4569e520 2024-04-23T18:57:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 -- 172.21.15.33:0/169341499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 msgr2=0x55ae456b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f938c0105b0 tx=0x7f938c0105e0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 -- 172.21.15.33:0/169341499 shutdown_connections 2024-04-23T18:57:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae456ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569ee40 0x55ae456a5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 --2- 172.21.15.33:0/169341499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569e520 0x55ae456b9700 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 -- 172.21.15.33:0/169341499 >> 172.21.15.33:0/169341499 conn(0x55ae4569ab30 msgr2=0x55ae456a5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 -- 172.21.15.33:0/169341499 shutdown_connections 2024-04-23T18:57:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.871+0000 7f93a61a8740 1 -- 172.21.15.33:0/169341499 wait complete. 2024-04-23T18:57:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 -- start start 2024-04-23T18:57:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 Processor -- start 2024-04-23T18:57:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae457221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1ab280 con 0x55ae4569ee40 2024-04-23T18:57:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1ac6f0 con 0x55ae456adc10 2024-04-23T18:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f939c1ab950 con 0x55ae4569e520 2024-04-23T18:57:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a61a8740 10 client.0 _ll_register_callbacks cb 0x7f9394009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a0955640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a0955640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56406/0 (socket says 172.21.15.33:56406) 2024-04-23T18:57:53.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.872+0000 7f93a0955640 1 -- 172.21.15.33:0/241456614 learned_addr learned my addr 172.21.15.33:0/241456614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a1156640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a1957640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae457221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 -- 172.21.15.33:0/241456614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 msgr2=0x55ae456b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:53.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 -- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 msgr2=0x55ae457221a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae457221a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.590 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 -- 172.21.15.33:0/241456614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae45722760 con 0x55ae4569ee40 2024-04-23T18:57:53.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a1156640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a0955640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f93980ee9a0 tx=0x7f939802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c283070246796496 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398049070 con 0x55ae4569ee40 2024-04-23T18:57:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(149..149 src has 1..149) ==== 8586+0+0 (secure 0 0 0) 0x7f93980dd0b0 con 0x55ae4569ee40 2024-04-23T18:57:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938c018440 con 0x55ae4569ee40 2024-04-23T18:57:53.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f93980c5860 con 0x55ae4569ee40 2024-04-23T18:57:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a61a8740 10 client.18654 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ae45723a20 con 0x55ae4569ee40 2024-04-23T18:57:53.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938c018440 con 0x55ae4569ee40 2024-04-23T18:57:53.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f93980bce50 con 0x55ae4569ee40 2024-04-23T18:57:53.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a61a8740 10 client.18654 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:53.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.873+0000 7f93a61a8740 10 client.18654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a61a8740 20 client.18654 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:53.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae457241a0 0x55ae457265c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:53.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a61a8740 4 client.18654 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae4560b200 con 0x55ae457241a0 2024-04-23T18:57:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a1156640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae457241a0 0x55ae457265c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f9384ff9640 10 client.18654 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.874+0000 7f93a1156640 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae457241a0 0x55ae457265c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12595c738824285c server_cookie=94cce5265c7f54ce in_seq=0 out_seq=0 2024-04-23T18:57:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f9384ff9640 1 -- 172.21.15.33:0/241456614 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae4560b200 con 0x55ae457241a0 2024-04-23T18:57:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f9384ff9640 10 client.18654 handle_command_reply: tid=0 2024-04-23T18:57:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 client.18654 shutdown 2024-04-23T18:57:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 msgr2=0x55ae456b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f93980ee9a0 tx=0x7f939802a410 comp rx=0 tx=0).stop 2024-04-23T18:57:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 shutdown_connections 2024-04-23T18:57:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55ae457241a0 0x55ae457265c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ae456adc10 0x55ae457221a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4569ee40 0x55ae456b1300 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f9384ff9640 0 client.18654 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:53.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 --2- 172.21.15.33:0/241456614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ae4569e520 0x55ae456b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.881+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 >> 172.21.15.33:0/241456614 conn(0x55ae4569ab30 msgr2=0x55ae45699420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 shutdown_connections 2024-04-23T18:57:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a61a8740 1 -- 172.21.15.33:0/241456614 wait complete. 2024-04-23T18:57:54.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a61a8740 20 client.18654 trim_cache size 0 max 16384 2024-04-23T18:57:54.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:54.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:54.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 msgr2=0x7f93880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:54.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 0x7f93880a6300 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9394005c40 tx=0x7f93940094a0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.095 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 msgr2=0x7f939c198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9398032fd0 tx=0x7f9398004270 comp rx=0 tx=0).stop 2024-04-23T18:57:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 shutdown_connections 2024-04-23T18:57:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f93880a3e40 0x7f93880a6300 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f939c19cf90 0x7f939c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f939c106170 0x7f939c198c10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.226 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 --2- 172.21.15.33:0/17712891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f939c1010f0 0x7f939c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 >> 172.21.15.33:0/17712891 conn(0x7f939c077fd0 msgr2=0x7f939c0699b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 shutdown_connections 2024-04-23T18:57:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:08.882+0000 7f93a33ba640 1 -- 172.21.15.33:0/17712891 wait complete. 2024-04-23T18:57:54.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:09 smithi033 ceph-mon[29274]: osdmap e149: 12 total, 12 up, 12 in 2024-04-23T18:57:54.271 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:09 smithi033 ceph-mon[29274]: pgmap v997: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:54.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:11 smithi033 ceph-mon[29274]: pgmap v998: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 31 op/s 2024-04-23T18:57:54.289 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:13 smithi033 ceph-mon[29274]: pgmap v999: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:54.298 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:13 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:57:54.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: pgmap v1000: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:57:54.330 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.347 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.354 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:15 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.361 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.471 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T18:57:54.478 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T18:57:54.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.498 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.512 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.536 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.609 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.617 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T18:57:54.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:57:54.632 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:57:54.639 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:57:54.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:16 smithi033 ceph-mon[29274]: osdmap e150: 12 total, 12 up, 12 in 2024-04-23T18:57:54.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T18:57:54.661 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T18:57:54.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T18:57:54.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T18:57:54.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: pgmap v1001: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 883 B/s wr, 11 op/s 2024-04-23T18:57:54.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: pgmap v1002: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:54.699 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:17 smithi033 ceph-mon[29274]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T18:57:54.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:19 smithi033 ceph-mon[29274]: pgmap v1004: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 15 op/s 2024-04-23T18:57:54.714 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:21 smithi033 ceph-mon[29274]: pgmap v1005: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T18:57:54.723 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:51 smithi106 ceph-mon[32716]: pgmap v1020: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:54.729 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:53 smithi106 ceph-mon[32716]: pgmap v1021: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:54.788 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:51 smithi122 ceph-mon[32546]: pgmap v1020: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:57:54.796 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:53 smithi122 ceph-mon[32546]: pgmap v1021: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 -- 172.21.15.33:0/1347546334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb92410c520 msgr2=0x7fb92410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:54.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/1347546334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb92410c520 0x7fb92410e9c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009900 tx=0x7fb91c02a510 comp rx=0 tx=0).stop 2024-04-23T18:57:54.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 -- 172.21.15.33:0/1347546334 shutdown_connections 2024-04-23T18:57:54.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/1347546334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb92410c520 0x7fb92410e9c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/1347546334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 0x7fb92410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/1347546334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9241031e0 0x7fb9241035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:54.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 -- 172.21.15.33:0/1347546334 >> 172.21.15.33:0/1347546334 conn(0x7fb924077fd0 msgr2=0x7fb9240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:54.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 -- 172.21.15.33:0/1347546334 shutdown_connections 2024-04-23T18:57:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.982+0000 7fb92a1e8640 1 -- 172.21.15.33:0/1347546334 wait complete. 2024-04-23T18:57:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 Processor -- start 2024-04-23T18:57:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 -- start start 2024-04-23T18:57:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:54.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 0x7fb924198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 0x7fb92419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241121b0 con 0x7fb9241031e0 2024-04-23T18:57:55.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb924112030 con 0x7fb92410c520 2024-04-23T18:57:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb92a1e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb924112330 con 0x7fb924103b00 2024-04-23T18:57:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.983+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51136/0 (socket says 172.21.15.33:51136) 2024-04-23T18:57:55.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 -- 172.21.15.33:0/2831999735 learned_addr learned my addr 172.21.15.33:0/2831999735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb922ffd640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 0x7fb924198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb923fff640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 0x7fb92419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 -- 172.21.15.33:0/2831999735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 msgr2=0x7fb924198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 0x7fb924198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 -- 172.21.15.33:0/2831999735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 msgr2=0x7fb92419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:55.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 0x7fb92419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 -- 172.21.15.33:0/2831999735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92419d640 con 0x7fb9241031e0 2024-04-23T18:57:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb923fff640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 0x7fb92419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ebc0 tx=0x7fb91400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb920ff9640 1 -- 172.21.15.33:0/2831999735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914010630 con 0x7fb9241031e0 2024-04-23T18:57:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92419d930 con 0x7fb9241031e0 2024-04-23T18:57:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb920ff9640 1 -- 172.21.15.33:0/2831999735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb914010c80 con 0x7fb9241031e0 2024-04-23T18:57:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb920ff9640 1 -- 172.21.15.33:0/2831999735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914018b60 con 0x7fb9241031e0 2024-04-23T18:57:55.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.984+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9241a51c0 con 0x7fb9241031e0 2024-04-23T18:57:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb920ff9640 1 -- 172.21.15.33:0/2831999735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb914018d00 con 0x7fb9241031e0 2024-04-23T18:57:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 Processor -- start 2024-04-23T18:57:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 -- start start 2024-04-23T18:57:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b50d970 con 0x55eb8b5b7c10 2024-04-23T18:57:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b4aa4a0 con 0x55eb8b5ad350 2024-04-23T18:57:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b509930 con 0x55eb8b5aca30 2024-04-23T18:57:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb920ff9640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 0x7fb90c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.987+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51152/0 (socket says 172.21.15.33:51152) 2024-04-23T18:57:55.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 -- 172.21.15.33:0/1810045265 learned_addr learned my addr 172.21.15.33:0/1810045265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb923fff640 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb920ff9640 1 -- 172.21.15.33:0/2831999735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fb9140d0ea0 con 0x7fb9241031e0 2024-04-23T18:57:55.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb923fff640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 0x7fb90c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb9237fe640 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 -- 172.21.15.33:0/1810045265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 msgr2=0x55eb8b5ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 -- 172.21.15.33:0/1810045265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 msgr2=0x55eb8b5b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 -- 172.21.15.33:0/1810045265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8b220210 con 0x55eb8b5b7c10 2024-04-23T18:57:55.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb922ffd640 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b690 tx=0x7fb91800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a6245866a2408a server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:55.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb923fff640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 0x7fb90c0a6280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0057c0 tx=0x7fb91c02a510 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb909ffb640 1 -- 172.21.15.33:0/1810045265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918003af0 con 0x55eb8b5b7c10 2024-04-23T18:57:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb909ffb640 1 -- 172.21.15.33:0/1810045265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918015070 con 0x55eb8b5b7c10 2024-04-23T18:57:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.988+0000 7fb909ffb640 1 -- 172.21.15.33:0/1810045265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918004c20 con 0x55eb8b5b7c10 2024-04-23T18:57:55.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- 172.21.15.33:0/1810045265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 msgr2=0x55eb8b5ba000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:55.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b690 tx=0x7fb91800ba90 comp rx=0 tx=0).stop 2024-04-23T18:57:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- 172.21.15.33:0/1810045265 shutdown_connections 2024-04-23T18:57:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5ba000 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/1810045265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- 172.21.15.33:0/1810045265 >> 172.21.15.33:0/1810045265 conn(0x55eb8b5a8f40 msgr2=0x55eb8b5a9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- 172.21.15.33:0/1810045265 shutdown_connections 2024-04-23T18:57:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- 172.21.15.33:0/1810045265 wait complete. 2024-04-23T18:57:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.989+0000 7fb92cfcf740 1 -- start start 2024-04-23T18:57:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 Processor -- start 2024-04-23T18:57:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 0x55eb8b5b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b5ba840 con 0x55eb8b5b7c10 2024-04-23T18:57:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b4a80b0 con 0x55eb8b5aca30 2024-04-23T18:57:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb8b5ba9c0 con 0x55eb8b5ad350 2024-04-23T18:57:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb92cfcf740 10 client.0 _ll_register_callbacks cb 0x55eb8b495d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51158/0 (socket says 172.21.15.33:51158) 2024-04-23T18:57:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb923fff640 1 -- 172.21.15.33:0/2638021157 learned_addr learned my addr 172.21.15.33:0/2638021157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 0x55eb8b5b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.990+0000 7fb922ffd640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:55.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 -- 172.21.15.33:0/2638021157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5ad350 msgr2=0x55eb8b5b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 -- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 msgr2=0x55eb8b5b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:55.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 0x55eb8b5b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:55.876 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 -- 172.21.15.33:0/2638021157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8b2ec620 con 0x55eb8b5b7c10 2024-04-23T18:57:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 0x55eb8b5b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:57:55.891 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb923fff640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009900 tx=0x7fb91c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a258a390495de6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:55.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:55.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c04f020 con 0x55eb8b5b7c10 2024-04-23T18:57:55.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fb91c003d40 con 0x55eb8b5b7c10 2024-04-23T18:57:55.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb918015070 con 0x55eb8b5b7c10 2024-04-23T18:57:55.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb91c05a050 con 0x55eb8b5b7c10 2024-04-23T18:57:55.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb92cfcf740 10 client.18666 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:55.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55eb8b5c67c0 con 0x55eb8b5b7c10 2024-04-23T18:57:55.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.991+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb918015070 con 0x55eb8b5b7c10 2024-04-23T18:57:55.960 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fb91c03a040 con 0x55eb8b5b7c10 2024-04-23T18:57:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 10 client.18666 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:56.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 10 client.18666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 20 client.18666 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:56.055 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b5c7c80 0x55eb8b5ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 4 client.18666 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb8b51b390 con 0x55eb8b5c7c80 2024-04-23T18:57:56.082 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b5c7c80 0x55eb8b5ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b5c7c80 0x55eb8b5ca090 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f124bf9d85f962 server_cookie=d1baa58f22b17d3 in_seq=0 out_seq=0 2024-04-23T18:57:56.101 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.992+0000 7fb902ffd640 10 client.18666 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb902ffd640 1 -- 172.21.15.33:0/2638021157 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb8b51b390 con 0x55eb8b5c7c80 2024-04-23T18:57:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb902ffd640 10 client.18666 handle_command_reply: tid=0 2024-04-23T18:57:56.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 client.18666 shutdown 2024-04-23T18:57:56.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 msgr2=0x55eb8b5b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:56.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fb91c009900 tx=0x7fb91c009260 comp rx=0 tx=0).stop 2024-04-23T18:57:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 shutdown_connections 2024-04-23T18:57:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b5c7c80 0x55eb8b5ca090 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5b4dd0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.239 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5ad350 0x55eb8b5b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb902ffd640 0 client.18666 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2638021157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5aca30 0x55eb8b5b7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 >> 172.21.15.33:0/2638021157 conn(0x55eb8b5a8f40 msgr2=0x55eb8b5a3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:56.273 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 shutdown_connections 2024-04-23T18:57:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2638021157 wait complete. 2024-04-23T18:57:56.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:21.993+0000 7fb92cfcf740 20 client.18666 trim_cache size 0 max 16384 2024-04-23T18:57:56.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.008+0000 7fb92cfcf740 1 Processor -- start 2024-04-23T18:57:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 -- start start 2024-04-23T18:57:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241aa330 con 0x55eb8b5aca30 2024-04-23T18:57:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241acc10 con 0x55eb8b5ad350 2024-04-23T18:57:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241aaa00 con 0x55eb8b5b7c10 2024-04-23T18:57:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51166/0 (socket says 172.21.15.33:51166) 2024-04-23T18:57:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 -- 172.21.15.33:0/579558693 learned_addr learned my addr 172.21.15.33:0/579558693 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb922ffd640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb923fff640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 -- 172.21.15.33:0/579558693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5b7c10 msgr2=0x55eb8b5cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5cfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 -- 172.21.15.33:0/579558693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 msgr2=0x55eb8b5cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 -- 172.21.15.33:0/579558693 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb8b5a02e0 con 0x55eb8b5aca30 2024-04-23T18:57:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb922ffd640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:57:56.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9237fe640 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb91409cb30 tx=0x7fb914023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a66830b56b34ea server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.009+0000 7fb9097fa640 1 -- 172.21.15.33:0/579558693 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914018420 con 0x55eb8b5aca30 2024-04-23T18:57:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb9097fa640 1 -- 172.21.15.33:0/579558693 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91409d530 con 0x55eb8b5aca30 2024-04-23T18:57:56.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb9097fa640 1 -- 172.21.15.33:0/579558693 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb914025170 con 0x55eb8b5aca30 2024-04-23T18:57:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 -- 172.21.15.33:0/579558693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 msgr2=0x55eb8b5c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fb91409cb30 tx=0x7fb914023040 comp rx=0 tx=0).stop 2024-04-23T18:57:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 -- 172.21.15.33:0/579558693 shutdown_connections 2024-04-23T18:57:56.605 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/579558693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5aca30 0x55eb8b5c3700 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 -- 172.21.15.33:0/579558693 >> 172.21.15.33:0/579558693 conn(0x55eb8b5a8f40 msgr2=0x55eb8b5b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.010+0000 7fb92cfcf740 1 -- 172.21.15.33:0/579558693 shutdown_connections 2024-04-23T18:57:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 -- 172.21.15.33:0/579558693 wait complete. 2024-04-23T18:57:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 -- start start 2024-04-23T18:57:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 Processor -- start 2024-04-23T18:57:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5bff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:56.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ab0d0 con 0x55eb8b5b7c10 2024-04-23T18:57:56.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ac540 con 0x55eb8b5ad350 2024-04-23T18:57:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9241ab7a0 con 0x55eb8b5aca30 2024-04-23T18:57:56.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.011+0000 7fb92cfcf740 10 client.0 _ll_register_callbacks cb 0x7fb91c005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51174/0 (socket says 172.21.15.33:51174) 2024-04-23T18:57:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 -- 172.21.15.33:0/2527954910 learned_addr learned my addr 172.21.15.33:0/2527954910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:56.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5bff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb922ffd640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:56.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 -- 172.21.15.33:0/2527954910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 msgr2=0x55eb8b5bff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:56.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5bff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 -- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 msgr2=0x55eb8b5bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:56.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 -- 172.21.15.33:0/2527954910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb8b5bbf90 con 0x55eb8b5b7c10 2024-04-23T18:57:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb923fff640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb91c04ae90 tx=0x7fb91c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4720b4afcbeb58c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb902ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c04f070 con 0x55eb8b5b7c10 2024-04-23T18:57:57.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fb91c031d10 con 0x55eb8b5b7c10 2024-04-23T18:57:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91409d530 con 0x55eb8b5b7c10 2024-04-23T18:57:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb91c035040 con 0x55eb8b5b7c10 2024-04-23T18:57:57.026 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb92cfcf740 10 client.18672 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:57.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.012+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55eb8b62e0c0 con 0x55eb8b5b7c10 2024-04-23T18:57:57.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb91409d530 con 0x55eb8b5b7c10 2024-04-23T18:57:57.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fb91c04b940 con 0x55eb8b5b7c10 2024-04-23T18:57:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 10 client.18672 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 20 client.18672 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:57.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b62e770 0x55eb8b630b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:57.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 4 client.18672 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:57:57.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb8b515200 con 0x55eb8b62e770 2024-04-23T18:57:57.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b62e770 0x55eb8b630b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:57.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb9237fe640 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b62e770 0x55eb8b630b60 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cadb91baff1352a server_cookie=92c996dd167b14fe in_seq=0 out_seq=0 2024-04-23T18:57:57.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.013+0000 7fb902ffd640 10 client.18672 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:57.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb902ffd640 1 -- 172.21.15.33:0/2527954910 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb8b515200 con 0x55eb8b62e770 2024-04-23T18:57:57.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb902ffd640 10 client.18672 handle_command_reply: tid=0 2024-04-23T18:57:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 client.18672 shutdown 2024-04-23T18:57:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 msgr2=0x55eb8b5bb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fb91c04ae90 tx=0x7fb91c009590 comp rx=0 tx=0).stop 2024-04-23T18:57:57.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 shutdown_connections 2024-04-23T18:57:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x55eb8b62e770 0x55eb8b630b60 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eb8b5b7c10 0x55eb8b5bb850 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55eb8b5ad350 0x55eb8b5bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb902ffd640 0 client.18672 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:57.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 --2- 172.21.15.33:0/2527954910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb8b5aca30 0x55eb8b5bff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 >> 172.21.15.33:0/2527954910 conn(0x55eb8b5a8f40 msgr2=0x55eb8b5ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:57.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 shutdown_connections 2024-04-23T18:57:57.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 1 -- 172.21.15.33:0/2527954910 wait complete. 2024-04-23T18:57:57.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92cfcf740 20 client.18672 trim_cache size 0 max 16384 2024-04-23T18:57:57.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:57:57.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:57:57.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:57:57.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:57:57.442 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:55 smithi106 ceph-mon[32716]: pgmap v1022: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:57.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:55 smithi122 ceph-mon[32546]: pgmap v1022: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.018+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 msgr2=0x7fb90c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 0x7fb90c0a6280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0057c0 tx=0x7fb91c02a510 comp rx=0 tx=0).stop 2024-04-23T18:57:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 msgr2=0x7fb924198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fb91400ebc0 tx=0x7fb91400c6a0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 shutdown_connections 2024-04-23T18:57:57.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fb90c0a3dc0 0x7fb90c0a6280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.563 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92410c520 0x7fb92419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb924103b00 0x7fb924198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 --2- 172.21.15.33:0/2831999735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9241031e0 0x7fb924198650 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 >> 172.21.15.33:0/2831999735 conn(0x7fb924077fd0 msgr2=0x7fb9240fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:57.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 shutdown_connections 2024-04-23T18:57:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:22.019+0000 7fb92a1e8640 1 -- 172.21.15.33:0/2831999735 wait complete. 2024-04-23T18:57:57.618 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:23 smithi033 ceph-mon[29274]: pgmap v1006: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 13 op/s 2024-04-23T18:57:57.628 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:25 smithi033 ceph-mon[29274]: pgmap v1007: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-23T18:57:57.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:27 smithi033 ceph-mon[29274]: pgmap v1008: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-04-23T18:57:57.646 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:29 smithi033 ceph-mon[29274]: pgmap v1009: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 267 B/s wr, 8 op/s 2024-04-23T18:57:57.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:31 smithi033 ceph-mon[29274]: pgmap v1010: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:57:57.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:33 smithi033 ceph-mon[29274]: pgmap v1011: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:57.675 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:57 smithi106 ceph-mon[32716]: pgmap v1023: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:57.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:57 smithi122 ceph-mon[32546]: pgmap v1023: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:57:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 -- 172.21.15.33:0/689378154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103200 msgr2=0x7f7e281035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 --2- 172.21.15.33:0/689378154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103200 0x7f7e281035e0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009900 tx=0x7f7e2402a480 comp rx=0 tx=0).stop 2024-04-23T18:57:57.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 -- 172.21.15.33:0/689378154 shutdown_connections 2024-04-23T18:57:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 --2- 172.21.15.33:0/689378154 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e2810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 --2- 172.21.15.33:0/689378154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103b20 0x7f7e2810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 --2- 172.21.15.33:0/689378154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103200 0x7f7e281035e0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 -- 172.21.15.33:0/689378154 >> 172.21.15.33:0/689378154 conn(0x7f7e280771d0 msgr2=0x7f7e280775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:57.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 -- 172.21.15.33:0/689378154 shutdown_connections 2024-04-23T18:57:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.119+0000 7f7e301ad640 1 -- 172.21.15.33:0/689378154 wait complete. 2024-04-23T18:57:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 Processor -- start 2024-04-23T18:57:57.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 -- start start 2024-04-23T18:57:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103200 0x7f7e28100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:57.905 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:57.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e281018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:57.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e28112270 con 0x7f7e28103b20 2024-04-23T18:57:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281120f0 con 0x7f7e2810c5d0 2024-04-23T18:57:57.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e301ad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281123f0 con 0x7f7e28103200 2024-04-23T18:57:57.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43126/0 (socket says 172.21.15.33:43126) 2024-04-23T18:57:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.120+0000 7f7e2d748640 1 -- 172.21.15.33:0/836145580 learned_addr learned my addr 172.21.15.33:0/836145580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2df49640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103200 0x7f7e28100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e281018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 -- 172.21.15.33:0/836145580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103200 msgr2=0x7f7e28100e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:58.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103200 0x7f7e28100e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.079 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 -- 172.21.15.33:0/836145580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 msgr2=0x7f7e281018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:58.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e281018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 -- 172.21.15.33:0/836145580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e281a6b90 con 0x7f7e28103b20 2024-04-23T18:57:58.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e281018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e2d748640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d770 tx=0x7f7e1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:58.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e1effd640 1 -- 172.21.15.33:0/836145580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18017070 con 0x7f7e28103b20 2024-04-23T18:57:58.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e1effd640 1 -- 172.21.15.33:0/836145580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e180054d0 con 0x7f7e28103b20 2024-04-23T18:57:58.150 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e281a6e80 con 0x7f7e28103b20 2024-04-23T18:57:58.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e1effd640 1 -- 172.21.15.33:0/836145580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18004c30 con 0x7f7e28103b20 2024-04-23T18:57:58.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.121+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e281a72f0 con 0x7f7e28103b20 2024-04-23T18:57:58.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e1effd640 1 -- 172.21.15.33:0/836145580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7e18004dd0 con 0x7f7e28103b20 2024-04-23T18:57:58.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e32fa3740 1 Processor -- start 2024-04-23T18:57:58.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e32fa3740 1 -- start start 2024-04-23T18:57:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e1effd640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 0x7f7e140a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.124+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x5565325342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532536bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55653248e970 con 0x556532534800 2024-04-23T18:57:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55653242b4a0 con 0x556532529f40 2024-04-23T18:57:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55653248a930 con 0x556532529620 2024-04-23T18:57:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 0x7f7e140a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e1effd640 1 -- 172.21.15.33:0/836145580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f7e180d02a0 con 0x7f7e28103b20 2024-04-23T18:57:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2e74a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532536bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.33:44820/0 (socket says 172.21.15.33:44820) 2024-04-23T18:57:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2e74a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532536bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43132/0 (socket says 172.21.15.33:43132) 2024-04-23T18:57:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 -- 172.21.15.33:0/1114102586 learned_addr learned my addr 172.21.15.33:0/1114102586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x5565325342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.578 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 -- 172.21.15.33:0/1114102586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 msgr2=0x5565325342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x5565325342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 -- 172.21.15.33:0/1114102586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 msgr2=0x556532536bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532536bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 -- 172.21.15.33:0/1114102586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565321a1210 con 0x556532529620 2024-04-23T18:57:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 0x7f7e140a62b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005c40 tx=0x7f7e20009940 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:58.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.125+0000 7f7e2df49640 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e24004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b49ebdff7128edd server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:58.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e13fff640 1 -- 172.21.15.33:0/1114102586 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24035040 con 0x556532529620 2024-04-23T18:57:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e13fff640 1 -- 172.21.15.33:0/1114102586 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240054f0 con 0x556532529620 2024-04-23T18:57:58.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e13fff640 1 -- 172.21.15.33:0/1114102586 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240035f0 con 0x556532529620 2024-04-23T18:57:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1114102586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 msgr2=0x556532529a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e24004170 comp rx=0 tx=0).stop 2024-04-23T18:57:58.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1114102586 shutdown_connections 2024-04-23T18:57:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532536bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x5565325342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1114102586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532529a00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1114102586 >> 172.21.15.33:0/1114102586 conn(0x556532525b30 msgr2=0x556532525f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:58.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1114102586 shutdown_connections 2024-04-23T18:57:58.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.126+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1114102586 wait complete. 2024-04-23T18:57:58.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 -- start start 2024-04-23T18:57:58.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 Processor -- start 2024-04-23T18:57:58.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532534060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x556532531350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565325373d0 con 0x556532534800 2024-04-23T18:57:58.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5565324290b0 con 0x556532529f40 2024-04-23T18:57:58.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556532537550 con 0x556532529620 2024-04-23T18:57:58.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e32fa3740 10 client.0 _ll_register_callbacks cb 0x556532416d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:57:58.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.127+0000 7f7e2e74a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43142/0 (socket says 172.21.15.33:43142) 2024-04-23T18:57:58.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 -- 172.21.15.33:0/3644804582 learned_addr learned my addr 172.21.15.33:0/3644804582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:58.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2df49640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532534060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2d748640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x556532531350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:58.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 -- 172.21.15.33:0/3644804582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 msgr2=0x556532534060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532534060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 -- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 msgr2=0x556532531350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x556532531350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 -- 172.21.15.33:0/3644804582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653226d620 con 0x556532534800 2024-04-23T18:57:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2d748640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x556532531350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:59.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7e20012ea0 tx=0x7f7e2001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00e4e7a364ee5b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:57:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20012030 con 0x556532534800 2024-04-23T18:57:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f7e200121d0 con 0x556532534800 2024-04-23T18:57:59.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e240054f0 con 0x556532534800 2024-04-23T18:57:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7e20011440 con 0x556532534800 2024-04-23T18:57:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e32fa3740 10 client.18684 fetch_fsmap learned FSMap version 49 2024-04-23T18:57:59.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.128+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556532543230 con 0x556532534800 2024-04-23T18:57:59.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e240054f0 con 0x556532534800 2024-04-23T18:57:59.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f7e2001f050 con 0x556532534800 2024-04-23T18:57:59.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 10 client.18684 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:57:59.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 10 client.18684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:57:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 20 client.18684 populate_metadata read hostname 'smithi033' 2024-04-23T18:57:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325449e0 0x556532546e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 4 client.18684 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:57:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653249c390 con 0x5565325449e0 2024-04-23T18:57:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.129+0000 7f7e2df49640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325449e0 0x556532546e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.130+0000 7f7e2df49640 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325449e0 0x556532546e60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb27b3934a61e4d6 server_cookie=57a50de04bd144e5 in_seq=0 out_seq=0 2024-04-23T18:57:59.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.130+0000 7f7e117fa640 10 client.18684 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.130+0000 7f7e117fa640 1 -- 172.21.15.33:0/3644804582 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55653249c390 con 0x5565325449e0 2024-04-23T18:57:59.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.130+0000 7f7e117fa640 10 client.18684 handle_command_reply: tid=0 2024-04-23T18:57:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 client.18684 shutdown 2024-04-23T18:57:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 msgr2=0x556532531890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f7e20012ea0 tx=0x7f7e2001c000 comp rx=0 tx=0).stop 2024-04-23T18:57:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 shutdown_connections 2024-04-23T18:57:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325449e0 0x556532546e60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532534800 0x556532531890 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529f40 0x556532531350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e117fa640 0 client.18684 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:57:59.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/3644804582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x556532534060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 >> 172.21.15.33:0/3644804582 conn(0x556532525b30 msgr2=0x55653252b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:57:59.521 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 shutdown_connections 2024-04-23T18:57:59.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 1 -- 172.21.15.33:0/3644804582 wait complete. 2024-04-23T18:57:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.131+0000 7f7e32fa3740 20 client.18684 trim_cache size 0 max 16384 2024-04-23T18:57:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 Processor -- start 2024-04-23T18:57:59.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 -- start start 2024-04-23T18:57:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 0x556532540290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532534800 0x55653254c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:57:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281ae800 con 0x556532529f40 2024-04-23T18:57:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281b10e0 con 0x556532529620 2024-04-23T18:57:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281aeed0 con 0x556532534800 2024-04-23T18:57:59.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:59.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43154/0 (socket says 172.21.15.33:43154) 2024-04-23T18:57:59.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 -- 172.21.15.33:0/1993481696 learned_addr learned my addr 172.21.15.33:0/1993481696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:57:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2df49640 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 0x556532540290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:57:59.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 -- 172.21.15.33:0/1993481696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532534800 msgr2=0x55653254c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532534800 0x55653254c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 -- 172.21.15.33:0/1993481696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 msgr2=0x556532540290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:57:59.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 0x556532540290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2d748640 1 -- 172.21.15.33:0/1993481696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565325255e0 con 0x556532529f40 2024-04-23T18:57:59.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.147+0000 7f7e2df49640 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 0x556532540290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:57:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e2d748640 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800df30 tx=0x7f7e180104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8ad33fc856b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:57:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e137fe640 1 -- 172.21.15.33:0/1993481696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18020040 con 0x556532529f40 2024-04-23T18:57:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e137fe640 1 -- 172.21.15.33:0/1993481696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1809c230 con 0x556532529f40 2024-04-23T18:57:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e137fe640 1 -- 172.21.15.33:0/1993481696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e180cdc60 con 0x556532529f40 2024-04-23T18:57:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1993481696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 msgr2=0x556532524b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:57:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800df30 tx=0x7f7e180104e0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1993481696 shutdown_connections 2024-04-23T18:57:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532534800 0x55653254c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532524b30 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:57:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/1993481696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532529620 0x556532540290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.148+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1993481696 >> 172.21.15.33:0/1993481696 conn(0x556532525b30 msgr2=0x556532524420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1993481696 shutdown_connections 2024-04-23T18:58:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- 172.21.15.33:0/1993481696 wait complete. 2024-04-23T18:58:00.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- start start 2024-04-23T18:58:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 Processor -- start 2024-04-23T18:58:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x55653253cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532534800 0x55653253a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:00.172 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281af5a0 con 0x556532529f40 2024-04-23T18:58:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281b0a10 con 0x556532534800 2024-04-23T18:58:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e281afc70 con 0x556532529620 2024-04-23T18:58:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e32fa3740 10 client.0 _ll_register_callbacks cb 0x7f7e20012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:00.206 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.149+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43160/0 (socket says 172.21.15.33:43160) 2024-04-23T18:58:00.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 -- 172.21.15.33:0/320417750 learned_addr learned my addr 172.21.15.33:0/320417750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2e74a640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532534800 0x55653253a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2df49640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x55653253cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 -- 172.21.15.33:0/320417750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 msgr2=0x55653253cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x55653253cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 -- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532534800 msgr2=0x55653253a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532534800 0x55653253a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 -- 172.21.15.33:0/320417750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653253ad30 con 0x556532529f40 2024-04-23T18:58:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e2d748640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009610 tx=0x7f7e180cde90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343b49a1efd1e7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e117fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:00.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18020030 con 0x556532529f40 2024-04-23T18:58:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f7e180a4a10 con 0x556532529f40 2024-04-23T18:58:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e1809c230 con 0x556532529f40 2024-04-23T18:58:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7e180a4ea0 con 0x556532529f40 2024-04-23T18:58:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e32fa3740 10 client.18690 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5565325ab070 con 0x556532529f40 2024-04-23T18:58:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.150+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1809c230 con 0x556532529f40 2024-04-23T18:58:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f7e180a0190 con 0x556532529f40 2024-04-23T18:58:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 10 client.18690 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:58:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 20 client.18690 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:00.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325ab680 0x5565325adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 4 client.18690 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:58:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556532496200 con 0x5565325ab680 2024-04-23T18:58:00.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e2df49640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325ab680 0x5565325adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e117fa640 10 client.18690 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.151+0000 7f7e2df49640 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325ab680 0x5565325adaa0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5046940b525c8b9 server_cookie=ffca30b467624a30 in_seq=0 out_seq=0 2024-04-23T18:58:00.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.155+0000 7f7e117fa640 1 -- 172.21.15.33:0/320417750 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556532496200 con 0x5565325ab680 2024-04-23T18:58:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.155+0000 7f7e117fa640 10 client.18690 handle_command_reply: tid=0 2024-04-23T18:58:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.155+0000 7f7e32fa3740 1 client.18690 shutdown 2024-04-23T18:58:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 msgr2=0x556532539f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009610 tx=0x7f7e180cde90 comp rx=0 tx=0).stop 2024-04-23T18:58:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 shutdown_connections 2024-04-23T18:58:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5565325ab680 0x5565325adaa0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556532534800 0x55653253a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556532529f40 0x556532539f50 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e117fa640 0 client.18690 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 --2- 172.21.15.33:0/320417750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556532529620 0x55653253cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 >> 172.21.15.33:0/320417750 conn(0x556532525b30 msgr2=0x556532524cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 shutdown_connections 2024-04-23T18:58:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 1 -- 172.21.15.33:0/320417750 wait complete. 2024-04-23T18:58:00.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e32fa3740 20 client.18690 trim_cache size 0 max 16384 2024-04-23T18:58:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:58:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:58:00.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:58:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:58:01.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 msgr2=0x7f7e140a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:01.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 0x7f7e140a62b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e20005c40 tx=0x7f7e20009940 comp rx=0 tx=0).stop 2024-04-23T18:58:01.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 msgr2=0x7f7e28101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:01.056 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7e1800d770 tx=0x7f7e1800dc30 comp rx=0 tx=0).stop 2024-04-23T18:58:01.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 shutdown_connections 2024-04-23T18:58:01.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f7e140a3df0 0x7f7e140a62b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e2810c5d0 0x7f7e281018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e28103b20 0x7f7e28101390 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 --2- 172.21.15.33:0/836145580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e28103200 0x7f7e28100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.156+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 >> 172.21.15.33:0/836145580 conn(0x7f7e280771d0 msgr2=0x7f7e280fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.157+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 shutdown_connections 2024-04-23T18:58:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:35.157+0000 7f7e301ad640 1 -- 172.21.15.33:0/836145580 wait complete. 2024-04-23T18:58:01.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:35 smithi033 ceph-mon[29274]: pgmap v1012: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:01.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:37 smithi033 ceph-mon[29274]: pgmap v1013: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-23T18:58:01.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:39 smithi033 ceph-mon[29274]: pgmap v1014: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:01.357 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:41 smithi033 ceph-mon[29274]: pgmap v1015: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:01.365 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:43 smithi033 ceph-mon[29274]: pgmap v1016: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:01.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:57:59 smithi106 ceph-mon[32716]: pgmap v1024: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:01.387 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:57:59 smithi122 ceph-mon[32546]: pgmap v1024: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:01.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- 172.21.15.33:0/716852043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564101a50 msgr2=0x7ff564105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:01.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- 172.21.15.33:0/716852043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564101a50 0x7ff564105a90 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7ff554009900 tx=0x7ff55402a410 comp rx=0 tx=0).stop 2024-04-23T18:58:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- 172.21.15.33:0/716852043 shutdown_connections 2024-04-23T18:58:01.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- 172.21.15.33:0/716852043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff564106170 0x7ff56410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- 172.21.15.33:0/716852043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff564101a50 0x7ff564105a90 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- 172.21.15.33:0/716852043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5641010a0 0x7ff564101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- 172.21.15.33:0/716852043 >> 172.21.15.33:0/716852043 conn(0x7ff564077eb0 msgr2=0x7ff5640782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:01.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- 172.21.15.33:0/716852043 shutdown_connections 2024-04-23T18:58:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- 172.21.15.33:0/716852043 wait complete. 2024-04-23T18:58:01.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 Processor -- start 2024-04-23T18:58:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- start start 2024-04-23T18:58:01.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:01.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff564106170 0x7ff564198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:01.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff56419d010 0x7ff56419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641110b0 con 0x7ff5641010a0 2024-04-23T18:58:01.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff564110f30 con 0x7ff56419d010 2024-04-23T18:58:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56bacf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff564111230 con 0x7ff564106170 2024-04-23T18:58:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:01.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39660/0 (socket says 172.21.15.33:39660) 2024-04-23T18:58:01.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 -- 172.21.15.33:0/3407800310 learned_addr learned my addr 172.21.15.33:0/3407800310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:01.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56a06c640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff56419d010 0x7ff56419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56906a640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff564106170 0x7ff564198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:01.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 -- 172.21.15.33:0/3407800310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff564106170 msgr2=0x7ff564198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:01.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff564106170 0x7ff564198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 -- 172.21.15.33:0/3407800310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff56419d010 msgr2=0x7ff56419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:01.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff56419d010 0x7ff56419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:01.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 -- 172.21.15.33:0/3407800310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56419db00 con 0x7ff5641010a0 2024-04-23T18:58:01.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.345+0000 7ff56986b640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.347+0000 7ff55affd640 1 -- 172.21.15.33:0/3407800310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56000cd80 con 0x7ff5641010a0 2024-04-23T18:58:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.347+0000 7ff55affd640 1 -- 172.21.15.33:0/3407800310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5600054d0 con 0x7ff5641010a0 2024-04-23T18:58:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.347+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56419ddf0 con 0x7ff5641010a0 2024-04-23T18:58:01.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.347+0000 7ff55affd640 1 -- 172.21.15.33:0/3407800310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560010690 con 0x7ff5641010a0 2024-04-23T18:58:01.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.347+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5641a5560 con 0x7ff5641010a0 2024-04-23T18:58:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff55affd640 1 -- 172.21.15.33:0/3407800310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff560004e90 con 0x7ff5641010a0 2024-04-23T18:58:01.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 Processor -- start 2024-04-23T18:58:01.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 -- start start 2024-04-23T18:58:01.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1799db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f179c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f16f5130 con 0x5613f178f360 2024-04-23T18:58:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5613f1691c50 con 0x5613f179a380 2024-04-23T18:58:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613f16f10f0 con 0x5613f178fd10 2024-04-23T18:58:02.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.349+0000 7ff56a06c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39670/0 (socket says 172.21.15.33:39670) 2024-04-23T18:58:02.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 -- 172.21.15.33:0/1525959423 learned_addr learned my addr 172.21.15.33:0/1525959423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff55affd640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 0x7ff5500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56986b640 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1799db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56906a640 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f179c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 0x7ff5500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 -- 172.21.15.33:0/1525959423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 msgr2=0x5613f1799db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1799db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff55affd640 1 -- 172.21.15.33:0/3407800310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7ff5600d0cc0 con 0x7ff5641010a0 2024-04-23T18:58:02.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 -- 172.21.15.33:0/1525959423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 msgr2=0x5613f179c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f179c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 -- 172.21.15.33:0/1525959423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f16abfd0 con 0x5613f178f360 2024-04-23T18:58:02.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff56a06c640 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff55c014a60 tx=0x7ff55c014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eba937d9b69e6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff54ffff640 1 -- 172.21.15.33:0/1525959423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c01ec50 con 0x5613f178f360 2024-04-23T18:58:02.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.351+0000 7ff54ffff640 1 -- 172.21.15.33:0/1525959423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff55c023070 con 0x5613f178f360 2024-04-23T18:58:02.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff54ffff640 1 -- 172.21.15.33:0/1525959423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c005520 con 0x5613f178f360 2024-04-23T18:58:02.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56a06c640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 0x7ff5500a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff55c022000 tx=0x7ff55c01e000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:02.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 -- 172.21.15.33:0/1525959423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 msgr2=0x5613f178f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:02.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7ff55c014a60 tx=0x7ff55c014f20 comp rx=0 tx=0).stop 2024-04-23T18:58:02.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 -- 172.21.15.33:0/1525959423 shutdown_connections 2024-04-23T18:58:02.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f179c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1799db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/1525959423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f178f740 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 -- 172.21.15.33:0/1525959423 >> 172.21.15.33:0/1525959423 conn(0x5613f178b620 msgr2=0x5613f178ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:02.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 -- 172.21.15.33:0/1525959423 shutdown_connections 2024-04-23T18:58:02.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.352+0000 7ff56e7d8740 1 -- 172.21.15.33:0/1525959423 wait complete. 2024-04-23T18:58:02.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- start start 2024-04-23T18:58:02.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 Processor -- start 2024-04-23T18:58:02.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 0x5613f1797270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f1797cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:02.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613f179d2f0 con 0x5613f178fd10 2024-04-23T18:58:02.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5613f168f640 con 0x5613f179a380 2024-04-23T18:58:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613f179d470 con 0x5613f178f360 2024-04-23T18:58:02.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 10 client.0 _ll_register_callbacks cb 0x5613f168f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:02.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39676/0 (socket says 172.21.15.33:39676) 2024-04-23T18:58:02.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 -- 172.21.15.33:0/2016122219 learned_addr learned my addr 172.21.15.33:0/2016122219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:02.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56a06c640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f1797cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56986b640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 0x5613f1797270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:02.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 -- 172.21.15.33:0/2016122219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 msgr2=0x5613f1797270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:02.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 0x5613f1797270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 -- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 msgr2=0x5613f1797cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:02.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f1797cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:02.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 -- 172.21.15.33:0/2016122219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f14d3a20 con 0x5613f178fd10 2024-04-23T18:58:02.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56986b640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 0x5613f1797270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:58:02.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56906a640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff554005ab0 tx=0x7ff5540316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad84ab6d0c24f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff54d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:02.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554035040 con 0x5613f178fd10 2024-04-23T18:58:02.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7ff554004240 con 0x5613f178fd10 2024-04-23T18:58:02.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55c023070 con 0x5613f178fd10 2024-04-23T18:58:02.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff554004440 con 0x5613f178fd10 2024-04-23T18:58:02.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 10 client.18702 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:02.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613f15341b0 con 0x5613f178fd10 2024-04-23T18:58:02.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.353+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff55c023070 con 0x5613f178fd10 2024-04-23T18:58:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7ff5540049f0 con 0x5613f178fd10 2024-04-23T18:58:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 10 client.18702 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:03.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 10 client.18702 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:58:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 20 client.18702 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:03.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f17aa860 0x5613f17acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:03.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 4 client.18702 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:58:03.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613f1702b50 con 0x5613f17aa860 2024-04-23T18:58:03.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56986b640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f17aa860 0x5613f17acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff54d7fa640 10 client.18702 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:03.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56986b640 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f17aa860 0x5613f17acc70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f939f220b1bbe0d server_cookie=415cf6aff71c66df in_seq=0 out_seq=0 2024-04-23T18:58:03.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff54d7fa640 1 -- 172.21.15.33:0/2016122219 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613f1702b50 con 0x5613f17aa860 2024-04-23T18:58:03.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff54d7fa640 10 client.18702 handle_command_reply: tid=0 2024-04-23T18:58:03.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 client.18702 shutdown 2024-04-23T18:58:03.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 msgr2=0x5613f17977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:03.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7ff554005ab0 tx=0x7ff5540316d0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 shutdown_connections 2024-04-23T18:58:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f17aa860 0x5613f17acc70 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f1797cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178fd10 0x5613f17977b0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2016122219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178f360 0x5613f1797270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff54d7fa640 0 client.18702 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:03.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.355+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 >> 172.21.15.33:0/2016122219 conn(0x5613f178b620 msgr2=0x5613f1791490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:03.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.356+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 shutdown_connections 2024-04-23T18:58:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.356+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2016122219 wait complete. 2024-04-23T18:58:03.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.356+0000 7ff56e7d8740 20 client.18702 trim_cache size 0 max 16384 2024-04-23T18:58:03.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 Processor -- start 2024-04-23T18:58:03.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 -- start start 2024-04-23T18:58:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:03.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1791260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:03.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f17b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:03.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641ac670 con 0x5613f178f360 2024-04-23T18:58:03.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641aef50 con 0x5613f179a380 2024-04-23T18:58:03.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641acd40 con 0x5613f178fd10 2024-04-23T18:58:03.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:03.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39678/0 (socket says 172.21.15.33:39678) 2024-04-23T18:58:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 -- 172.21.15.33:0/2826103777 learned_addr learned my addr 172.21.15.33:0/2826103777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:03.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56906a640 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1791260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:03.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 -- 172.21.15.33:0/2826103777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 msgr2=0x5613f1791260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:03.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1791260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 -- 172.21.15.33:0/2826103777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 msgr2=0x5613f17b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:03.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f17b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 -- 172.21.15.33:0/2826103777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613f13c7310 con 0x5613f178f360 2024-04-23T18:58:03.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff56986b640 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9c0 tx=0x7ff5600993f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1d89da8817155d server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:03.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff54f7fe640 1 -- 172.21.15.33:0/2826103777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5600d2040 con 0x5613f178f360 2024-04-23T18:58:03.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff54f7fe640 1 -- 172.21.15.33:0/2826103777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff560005680 con 0x5613f178f360 2024-04-23T18:58:03.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.374+0000 7ff54f7fe640 1 -- 172.21.15.33:0/2826103777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5600046c0 con 0x5613f178f360 2024-04-23T18:58:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2826103777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 msgr2=0x5613f17a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:03.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9c0 tx=0x7ff5600993f0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2826103777 shutdown_connections 2024-04-23T18:58:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f179a380 0x5613f17b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f1791260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/2826103777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f178f360 0x5613f17a61b0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2826103777 >> 172.21.15.33:0/2826103777 conn(0x5613f178b620 msgr2=0x5613f1790b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2826103777 shutdown_connections 2024-04-23T18:58:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- 172.21.15.33:0/2826103777 wait complete. 2024-04-23T18:58:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 -- start start 2024-04-23T18:58:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.376+0000 7ff56e7d8740 1 Processor -- start 2024-04-23T18:58:03.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 0x5613f17a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f179f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:04.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641ad410 con 0x5613f179a380 2024-04-23T18:58:04.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641ae880 con 0x5613f178f360 2024-04-23T18:58:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641adae0 con 0x5613f178fd10 2024-04-23T18:58:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 10 client.0 _ll_register_callbacks cb 0x7ff554005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:04.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:04.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39694/0 (socket says 172.21.15.33:39694) 2024-04-23T18:58:04.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 -- 172.21.15.33:0/903300592 learned_addr learned my addr 172.21.15.33:0/903300592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56906a640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f179f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:04.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56986b640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 0x5613f17a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 -- 172.21.15.33:0/903300592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 msgr2=0x5613f179f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:04.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f179f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 -- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 msgr2=0x5613f17a2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:04.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 0x5613f17a2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 -- 172.21.15.33:0/903300592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613f1810dc0 con 0x5613f179a380 2024-04-23T18:58:04.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56986b640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 0x5613f17a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:58:04.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56a06c640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7ff55c014930 tx=0x7ff55c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f487cc6e2cfb154 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:04.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff54d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c012c70 con 0x5613f179a380 2024-04-23T18:58:04.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7ff55c012e10 con 0x5613f179a380 2024-04-23T18:58:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.377+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff560005680 con 0x5613f179a380 2024-04-23T18:58:04.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff55c03f050 con 0x5613f179a380 2024-04-23T18:58:04.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 10 client.18708 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:04.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613f1811f80 con 0x5613f179a380 2024-04-23T18:58:04.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff560005680 con 0x5613f179a380 2024-04-23T18:58:04.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7ff55c03a030 con 0x5613f179a380 2024-04-23T18:58:04.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 10 client.18708 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:04.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 10 client.18708 resolve_mds: validated gid 24551 aka daemon mds.b 2024-04-23T18:58:04.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 20 client.18708 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:04.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f1812750 0x5613f1814b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 4 client.18708 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-23T18:58:04.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5613f16fc9c0 con 0x5613f1812750 2024-04-23T18:58:04.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.378+0000 7ff56986b640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f1812750 0x5613f1814b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:04.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56986b640 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f1812750 0x5613f1814b70 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b14a0f84256c34c server_cookie=f7d2629fcb4c3297 in_seq=0 out_seq=0 2024-04-23T18:58:04.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff54d7fa640 10 client.18708 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:04.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff54d7fa640 1 -- 172.21.15.33:0/903300592 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5613f16fc9c0 con 0x5613f1812750 2024-04-23T18:58:04.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff54d7fa640 10 client.18708 handle_command_reply: tid=0 2024-04-23T18:58:04.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 client.18708 shutdown 2024-04-23T18:58:04.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 msgr2=0x5613f179fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:04.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7ff55c014930 tx=0x7ff55c005e80 comp rx=0 tx=0).stop 2024-04-23T18:58:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 shutdown_connections 2024-04-23T18:58:04.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5613f1812750 0x5613f1814b70 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613f179a380 0x5613f179fdd0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613f178fd10 0x5613f179f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff54d7fa640 0 client.18708 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:04.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 --2- 172.21.15.33:0/903300592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5613f178f360 0x5613f17a2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 >> 172.21.15.33:0/903300592 conn(0x5613f178b620 msgr2=0x5613f17913c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:04.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 shutdown_connections 2024-04-23T18:58:04.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 1 -- 172.21.15.33:0/903300592 wait complete. 2024-04-23T18:58:04.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.379+0000 7ff56e7d8740 20 client.18708 trim_cache size 0 max 16384 2024-04-23T18:58:04.866 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:58:04.875 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 49, 2024-04-23T18:58:04.884 INFO:teuthology.orchestra.run.smithi033.stdout: "leader": 24551, 2024-04-23T18:58:04.894 INFO:teuthology.orchestra.run.smithi033.stdout: "set_version": 35, 2024-04-23T18:58:04.965 INFO:teuthology.orchestra.run.smithi033.stdout: "sets": {} 2024-04-23T18:58:04.974 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T18:58:05.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:01 smithi106 ceph-mon[32716]: pgmap v1025: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:05.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:03 smithi106 ceph-mon[32716]: pgmap v1026: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:05.033 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:01 smithi122 ceph-mon[32546]: pgmap v1025: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:05.042 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:03 smithi122 ceph-mon[32546]: pgmap v1026: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 msgr2=0x7ff5500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:05.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 0x7ff5500a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff55c022000 tx=0x7ff55c01e000 comp rx=0 tx=0).stop 2024-04-23T18:58:05.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 msgr2=0x7ff5641987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:05.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7ff56000e9f0 tx=0x7ff56000edf0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 shutdown_connections 2024-04-23T18:58:05.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7ff5500a3d20 0x7ff5500a61e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff56419d010 0x7ff56419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff564106170 0x7ff564198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 --2- 172.21.15.33:0/3407800310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5641010a0 0x7ff5641987a0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 >> 172.21.15.33:0/3407800310 conn(0x7ff564077eb0 msgr2=0x7ff56406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:05.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 shutdown_connections 2024-04-23T18:58:05.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:57:45.380+0000 7ff56bacf640 1 -- 172.21.15.33:0/3407800310 wait complete. 2024-04-23T18:58:05.254 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 408, in task stop_all_quiescers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/quiescer.py", line 361, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error releasing set '6bf9e7c1': 1 (EPERM) 2024-04-23T18:58:05.264 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-23T18:58:05.287 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:58:05.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:45 smithi033 ceph-mon[29274]: pgmap v1017: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:05.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:47 smithi033 ceph-mon[29274]: pgmap v1018: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:05.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 -- 172.21.15.33:0/1393585752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc101aa0 msgr2=0x7fc1bc105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 --2- 172.21.15.33:0/1393585752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc101aa0 0x7fc1bc105a90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009ca0 tx=0x7fc1b802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 -- 172.21.15.33:0/1393585752 shutdown_connections 2024-04-23T18:58:05.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 --2- 172.21.15.33:0/1393585752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc106170 0x7fc1bc10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 --2- 172.21.15.33:0/1393585752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc101aa0 0x7fc1bc105a90 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 --2- 172.21.15.33:0/1393585752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1010f0 0x7fc1bc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 -- 172.21.15.33:0/1393585752 >> 172.21.15.33:0/1393585752 conn(0x7fc1bc077fd0 msgr2=0x7fc1bc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:05.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 -- 172.21.15.33:0/1393585752 shutdown_connections 2024-04-23T18:58:05.424 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.246+0000 7fc1c3631640 1 -- 172.21.15.33:0/1393585752 wait complete. 2024-04-23T18:58:05.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 Processor -- start 2024-04-23T18:58:05.470 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 -- start start 2024-04-23T18:58:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc1010f0 0x7fc1bc102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1035b0 0x7fc1bc1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1110b0 con 0x7fc1bc106170 2024-04-23T18:58:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc110f30 con 0x7fc1bc1035b0 2024-04-23T18:58:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c3631640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc111230 con 0x7fc1bc1010f0 2024-04-23T18:58:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51834/0 (socket says 172.21.15.33:51834) 2024-04-23T18:58:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2573741578 learned_addr learned my addr 172.21.15.33:0/2573741578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.247+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc1010f0 0x7fc1bc102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1035b0 0x7fc1bc1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2573741578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc1010f0 msgr2=0x7fc1bc102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc1010f0 0x7fc1bc102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2573741578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1035b0 msgr2=0x7fc1bc1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1035b0 0x7fc1bc1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2573741578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc1a6f00 con 0x7fc1bc106170 2024-04-23T18:58:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc1b802ade0 tx=0x7fc1b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1b27fc640 1 -- 172.21.15.33:0/2573741578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8003d60 con 0x7fc1bc106170 2024-04-23T18:58:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1b27fc640 1 -- 172.21.15.33:0/2573741578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b80044f0 con 0x7fc1bc106170 2024-04-23T18:58:05.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1b27fc640 1 -- 172.21.15.33:0/2573741578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b802fce0 con 0x7fc1bc106170 2024-04-23T18:58:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc1a7190 con 0x7fc1bc106170 2024-04-23T18:58:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.248+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc1a7730 con 0x7fc1bc106170 2024-04-23T18:58:05.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.250+0000 7fc1b27fc640 1 -- 172.21.15.33:0/2573741578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc1b80046a0 con 0x7fc1bc106170 2024-04-23T18:58:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 Processor -- start 2024-04-23T18:58:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 -- start start 2024-04-23T18:58:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a05a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559c6796a970 con 0x559c67a10800 2024-04-23T18:58:05.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559c679074a0 con 0x559c67a05f40 2024-04-23T18:58:05.959 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c6425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c67966930 con 0x559c67a05620 2024-04-23T18:58:05.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1b27fc640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 0x7fc1a80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51850/0 (socket says 172.21.15.33:51850) 2024-04-23T18:58:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/1859477631 learned_addr learned my addr 172.21.15.33:0/1859477631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.251+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a05a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 0x7fc1a80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1b27fc640 1 -- 172.21.15.33:0/2573741578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fc1b80f1ef0 con 0x7fc1bc106170 2024-04-23T18:58:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/1859477631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 msgr2=0x559c67a05a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a05a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/1859477631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 msgr2=0x559c67a102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/1859477631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6767d210 con 0x559c67a10800 2024-04-23T18:58:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005cc0 tx=0x7fc1b802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb2d3048db063ca server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:06.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 0x7fc1a80a6300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4005fd0 tx=0x7fc1b40058a0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1a77fe640 1 -- 172.21.15.33:0/1859477631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b802f660 con 0x559c67a10800 2024-04-23T18:58:06.247 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1a77fe640 1 -- 172.21.15.33:0/1859477631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b8036020 con 0x559c67a10800 2024-04-23T18:58:06.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.252+0000 7fc1a77fe640 1 -- 172.21.15.33:0/1859477631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80c2340 con 0x559c67a10800 2024-04-23T18:58:06.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- 172.21.15.33:0/1859477631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 msgr2=0x559c67a12bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:06.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8005cc0 tx=0x7fc1b802a8c0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- 172.21.15.33:0/1859477631 shutdown_connections 2024-04-23T18:58:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a10800 0x559c67a12bf0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 --2- 172.21.15.33:0/1859477631 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a05a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- 172.21.15.33:0/1859477631 >> 172.21.15.33:0/1859477631 conn(0x559c67a01b30 msgr2=0x559c67a01f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- 172.21.15.33:0/1859477631 shutdown_connections 2024-04-23T18:58:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- 172.21.15.33:0/1859477631 wait complete. 2024-04-23T18:58:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 -- start start 2024-04-23T18:58:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.253+0000 7fc1c6425740 1 Processor -- start 2024-04-23T18:58:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a10800 0x559c67a0da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559c6796a970 con 0x559c67a05f40 2024-04-23T18:58:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559c679050b0 con 0x559c67a10800 2024-04-23T18:58:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c67a131f0 con 0x559c67a05620 2024-04-23T18:58:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c6425740 10 client.0 _ll_register_callbacks cb 0x559c678f2d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:06.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51860/0 (socket says 172.21.15.33:51860) 2024-04-23T18:58:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2093799703 learned_addr learned my addr 172.21.15.33:0/2093799703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a10800 0x559c67a0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2093799703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 msgr2=0x559c67a10190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a10190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a10800 msgr2=0x559c67a0da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a10800 0x559c67a0da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/2093799703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c67749620 con 0x559c67a05f40 2024-04-23T18:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ef7f0 tx=0x7fc1b80efb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff77a798159ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.254+0000 7fc1a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:06.748 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80d6020 con 0x559c67a05f40 2024-04-23T18:58:06.756 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fc1b80be1f0 con 0x559c67a05f40 2024-04-23T18:58:06.788 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1b8036020 con 0x559c67a05f40 2024-04-23T18:58:06.794 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc1b80be470 con 0x559c67a05f40 2024-04-23T18:58:06.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 10 client.18720 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:06.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559c67a1ef30 con 0x559c67a05f40 2024-04-23T18:58:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1b8036020 con 0x559c67a05f40 2024-04-23T18:58:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fc1b80c6770 con 0x559c67a05f40 2024-04-23T18:58:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 10 client.18720 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 10 client.18720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:58:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 20 client.18720 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a20390 0x559c67a22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 4 client.18720 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:58:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.255+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c67978390 con 0x559c67a20390 2024-04-23T18:58:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.256+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a20390 0x559c67a22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.256+0000 7fc1a4ff9640 10 client.18720 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.256+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a20390 0x559c67a22780 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1713bf4bc6c7706 server_cookie=bb2cb7b17cefbafe in_seq=0 out_seq=0 2024-04-23T18:58:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/2093799703 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c67978390 con 0x559c67a20390 2024-04-23T18:58:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1a4ff9640 10 client.18720 handle_command_reply: tid=0 2024-04-23T18:58:07.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 client.18720 shutdown 2024-04-23T18:58:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 msgr2=0x559c67a0d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80ef7f0 tx=0x7fc1b80efb60 comp rx=0 tx=0).stop 2024-04-23T18:58:07.114 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 shutdown_connections 2024-04-23T18:58:07.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a20390 0x559c67a22780 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a10800 0x559c67a0da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.249 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a0d480 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1a4ff9640 0 client.18720 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 --2- 172.21.15.33:0/2093799703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a05620 0x559c67a10190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 >> 172.21.15.33:0/2093799703 conn(0x559c67a01b30 msgr2=0x559c67a07860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 shutdown_connections 2024-04-23T18:58:07.297 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 1 -- 172.21.15.33:0/2093799703 wait complete. 2024-04-23T18:58:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.257+0000 7fc1c6425740 20 client.18720 trim_cache size 0 max 16384 2024-04-23T18:58:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 Processor -- start 2024-04-23T18:58:07.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 -- start start 2024-04-23T18:58:07.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:07.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a08970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:07.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a0fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1ae7e0 con 0x559c67a05620 2024-04-23T18:58:07.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1b10c0 con 0x559c67a05f40 2024-04-23T18:58:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c6425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1aeeb0 con 0x559c67a10800 2024-04-23T18:58:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c13cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c13cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51872/0 (socket says 172.21.15.33:51872) 2024-04-23T18:58:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c13cd640 1 -- 172.21.15.33:0/3524292675 learned_addr learned my addr 172.21.15.33:0/3524292675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a0fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c13cd640 1 -- 172.21.15.33:0/3524292675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 msgr2=0x559c67a0fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:07.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a08970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.272+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a0fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c13cd640 1 -- 172.21.15.33:0/3524292675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 msgr2=0x559c67a08970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:07.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a08970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c13cd640 1 -- 172.21.15.33:0/3524292675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c67a0dd70 con 0x559c67a05620 2024-04-23T18:58:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:58:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:58:07.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00ad30 tx=0x7fc1ac010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da7d8c6d673d4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:07.739 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1a6ffd640 1 -- 172.21.15.33:0/3524292675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac005220 con 0x559c67a05620 2024-04-23T18:58:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1a6ffd640 1 -- 172.21.15.33:0/3524292675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ac023070 con 0x559c67a05620 2024-04-23T18:58:07.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1a6ffd640 1 -- 172.21.15.33:0/3524292675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac01f3d0 con 0x559c67a05620 2024-04-23T18:58:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c6425740 1 -- 172.21.15.33:0/3524292675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 msgr2=0x559c67a1bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.273+0000 7fc1c6425740 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00ad30 tx=0x7fc1ac010800 comp rx=0 tx=0).stop 2024-04-23T18:58:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- 172.21.15.33:0/3524292675 shutdown_connections 2024-04-23T18:58:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05f40 0x559c67a08970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- 172.21.15.33:0/3524292675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05620 0x559c67a1bea0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- 172.21.15.33:0/3524292675 >> 172.21.15.33:0/3524292675 conn(0x559c67a01b30 msgr2=0x559c67a08260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- 172.21.15.33:0/3524292675 shutdown_connections 2024-04-23T18:58:07.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- 172.21.15.33:0/3524292675 wait complete. 2024-04-23T18:58:07.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- start start 2024-04-23T18:58:07.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 Processor -- start 2024-04-23T18:58:07.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 0x559c67a183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1af580 con 0x559c67a05f40 2024-04-23T18:58:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1b09f0 con 0x559c67a05620 2024-04-23T18:58:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1afc50 con 0x559c67a10800 2024-04-23T18:58:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c6425740 10 client.0 _ll_register_callbacks cb 0x7fc1b8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.274+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51876/0 (socket says 172.21.15.33:51876) 2024-04-23T18:58:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/243278486 learned_addr learned my addr 172.21.15.33:0/243278486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c1bce640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 0x559c67a183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:08.218 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/243278486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 msgr2=0x559c67a139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 msgr2=0x559c67a183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 0x559c67a183c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 -- 172.21.15.33:0/243278486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c67a14210 con 0x559c67a05f40 2024-04-23T18:58:08.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 0x559c67a183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c0bcc640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009430 tx=0x7fc1b8003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b92ca52f5f94fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80be3f0 con 0x559c67a05f40 2024-04-23T18:58:08.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7fc1b8004040 con 0x559c67a05f40 2024-04-23T18:58:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1ac023070 con 0x559c67a05f40 2024-04-23T18:58:08.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fc1b80be690 con 0x559c67a05f40 2024-04-23T18:58:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c6425740 10 client.18726 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.275+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559c67a14210 con 0x559c67a05f40 2024-04-23T18:58:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1ac023070 con 0x559c67a05f40 2024-04-23T18:58:08.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7fc1b80bd3e0 con 0x559c67a05f40 2024-04-23T18:58:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 10 client.18726 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 10 client.18726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:58:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 20 client.18726 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a867f0 0x559c67a88c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 4 client.18726 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:58:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c67972200 con 0x559c67a867f0 2024-04-23T18:58:08.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.276+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a867f0 0x559c67a88c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.277+0000 7fc1a4ff9640 10 client.18726 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.277+0000 7fc1c13cd640 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a867f0 0x559c67a88c10 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=770c19cf9ab666d6 server_cookie=64824b4fba3522d2 in_seq=0 out_seq=0 2024-04-23T18:58:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1a4ff9640 1 -- 172.21.15.33:0/243278486 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c67972200 con 0x559c67a867f0 2024-04-23T18:58:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1a4ff9640 10 client.18726 handle_command_reply: tid=0 2024-04-23T18:58:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 client.18726 shutdown 2024-04-23T18:58:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 msgr2=0x559c67a18900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:08.771 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009430 tx=0x7fc1b8003f00 comp rx=0 tx=0).stop 2024-04-23T18:58:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 shutdown_connections 2024-04-23T18:58:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x559c67a867f0 0x559c67a88c10 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c67a10800 0x559c67a139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559c67a05f40 0x559c67a18900 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1a4ff9640 0 client.18726 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:08.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 --2- 172.21.15.33:0/243278486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c67a05620 0x559c67a183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:08.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 >> 172.21.15.33:0/243278486 conn(0x559c67a01b30 msgr2=0x559c67a1b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:08.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 shutdown_connections 2024-04-23T18:58:08.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 1 -- 172.21.15.33:0/243278486 wait complete. 2024-04-23T18:58:08.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.281+0000 7fc1c6425740 20 client.18726 trim_cache size 0 max 16384 2024-04-23T18:58:08.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:58:08.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:58:08.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:58:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:58:08.947 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 msgr2=0x7fc1a80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 0x7fc1a80a6300 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4005fd0 tx=0x7fc1b40058a0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 msgr2=0x7fc1bc103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc1b802ade0 tx=0x7fc1b8003970 comp rx=0 tx=0).stop 2024-04-23T18:58:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 shutdown_connections 2024-04-23T18:58:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fc1a80a3e40 0x7fc1a80a6300 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1bc1035b0 0x7fc1bc1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1bc106170 0x7fc1bc103070 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 --2- 172.21.15.33:0/2573741578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc1010f0 0x7fc1bc102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.094 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 >> 172.21.15.33:0/2573741578 conn(0x7fc1bc077fd0 msgr2=0x7fc1bc0698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:09.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 shutdown_connections 2024-04-23T18:58:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:57:48.282+0000 7fc1c3631640 1 -- 172.21.15.33:0/2573741578 wait complete. 2024-04-23T18:58:09.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:05 smithi106 ceph-mon[32716]: pgmap v1027: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:58:09.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:05 smithi106 ceph-mon[32716]: osdmap e151: 12 total, 12 up, 12 in 2024-04-23T18:58:09.146 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:07 smithi106 ceph-mon[32716]: pgmap v1029: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:09.205 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:05 smithi122 ceph-mon[32546]: pgmap v1027: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:58:09.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:05 smithi122 ceph-mon[32546]: osdmap e151: 12 total, 12 up, 12 in 2024-04-23T18:58:09.224 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:07 smithi122 ceph-mon[32546]: pgmap v1029: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:09.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:49 smithi033 ceph-mon[29274]: pgmap v1019: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:58:09.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:51 smithi033 ceph-mon[29274]: pgmap v1020: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:58:09.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:53 smithi033 ceph-mon[29274]: pgmap v1021: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:55 smithi033 ceph-mon[29274]: pgmap v1022: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:09.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:57 smithi033 ceph-mon[29274]: pgmap v1023: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:09.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:57:59 smithi033 ceph-mon[29274]: pgmap v1024: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 -- 172.21.15.33:0/402809169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 msgr2=0x7f761c103560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 --2- 172.21.15.33:0/402809169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c103560 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f760c009900 tx=0x7f760c02a480 comp rx=0 tx=0).stop 2024-04-23T18:58:09.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 -- 172.21.15.33:0/402809169 shutdown_connections 2024-04-23T18:58:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 --2- 172.21.15.33:0/402809169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c10c550 0x7f761c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 --2- 172.21.15.33:0/402809169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c103aa0 0x7f761c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 --2- 172.21.15.33:0/402809169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c103560 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 -- 172.21.15.33:0/402809169 >> 172.21.15.33:0/402809169 conn(0x7f761c077fd0 msgr2=0x7f761c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 -- 172.21.15.33:0/402809169 shutdown_connections 2024-04-23T18:58:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.378+0000 7f7621336640 1 -- 172.21.15.33:0/402809169 wait complete. 2024-04-23T18:58:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 Processor -- start 2024-04-23T18:58:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 -- start start 2024-04-23T18:58:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 0x7f761c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c10c550 0x7f761c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f761c112200 con 0x7f761c103180 2024-04-23T18:58:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f761c112080 con 0x7f761c10c550 2024-04-23T18:58:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f7621336640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f761c112380 con 0x7f761c103aa0 2024-04-23T18:58:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f761affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f761affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39168/0 (socket says 172.21.15.33:39168) 2024-04-23T18:58:09.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.379+0000 7f761affd640 1 -- 172.21.15.33:0/2415727324 learned_addr learned my addr 172.21.15.33:0/2415727324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:09.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761a7fc640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 0x7f761c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761b7fe640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c10c550 0x7f761c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:09.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 -- 172.21.15.33:0/2415727324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 msgr2=0x7f761c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:09.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 0x7f761c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 -- 172.21.15.33:0/2415727324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c10c550 msgr2=0x7f761c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c10c550 0x7f761c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 -- 172.21.15.33:0/2415727324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761c19d690 con 0x7f761c103180 2024-04-23T18:58:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761a7fc640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 0x7f761c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f761affd640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f760c005ab0 tx=0x7f760c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f7603fff640 1 -- 172.21.15.33:0/2415727324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c03f070 con 0x7f761c103180 2024-04-23T18:58:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f761c19d920 con 0x7f761c103180 2024-04-23T18:58:09.949 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f7603fff640 1 -- 172.21.15.33:0/2415727324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c031d60 con 0x7f761c103180 2024-04-23T18:58:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f7603fff640 1 -- 172.21.15.33:0/2415727324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c035030 con 0x7f761c103180 2024-04-23T18:58:09.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.380+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f761c1a5210 con 0x7f761c103180 2024-04-23T18:58:09.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.382+0000 7f7603fff640 1 -- 172.21.15.33:0/2415727324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f760c042070 con 0x7f761c103180 2024-04-23T18:58:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 Processor -- start 2024-04-23T18:58:09.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 -- start start 2024-04-23T18:58:10.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.020 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d39800 0x5611d2d3bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5611d2c93970 con 0x5611d2d2e620 2024-04-23T18:58:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5611d2c304a0 con 0x5611d2d39800 2024-04-23T18:58:10.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7624053740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5611d2c8f930 con 0x5611d2d2ef40 2024-04-23T18:58:10.113 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39180/0 (socket says 172.21.15.33:39180) 2024-04-23T18:58:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 -- 172.21.15.33:0/755958912 learned_addr learned my addr 172.21.15.33:0/755958912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7603fff640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 0x7f76040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761affd640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.272 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761a7fc640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d39800 0x5611d2d3bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.282 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 -- 172.21.15.33:0/755958912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 msgr2=0x5611d2d392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:10.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 -- 172.21.15.33:0/755958912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d39800 msgr2=0x5611d2d3bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d39800 0x5611d2d3bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761b7fe640 1 -- 172.21.15.33:0/755958912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d29a6210 con 0x5611d2d2e620 2024-04-23T18:58:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f761affd640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:10.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.383+0000 7f7603fff640 1 -- 172.21.15.33:0/2415727324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f760c0f1120 con 0x7f761c103180 2024-04-23T18:58:10.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.384+0000 7f761b7fe640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 0x7f76040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.384+0000 7f761b7fe640 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7614010520 tx=0x7f76140109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babe08b5396631b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:10.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.384+0000 7f76017fa640 1 -- 172.21.15.33:0/755958912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614022070 con 0x5611d2d2e620 2024-04-23T18:58:10.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.384+0000 7f76017fa640 1 -- 172.21.15.33:0/755958912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76140044e0 con 0x5611d2d2e620 2024-04-23T18:58:10.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.384+0000 7f76017fa640 1 -- 172.21.15.33:0/755958912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614019070 con 0x5611d2d2e620 2024-04-23T18:58:10.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- 172.21.15.33:0/755958912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 msgr2=0x5611d2d2ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:10.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f761b7fe640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 0x7f76040a6280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f761401f000 tx=0x7f761401d000 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f7614010520 tx=0x7f76140109e0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- 172.21.15.33:0/755958912 shutdown_connections 2024-04-23T18:58:10.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d39800 0x5611d2d3bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.588 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- 172.21.15.33:0/755958912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d2ea00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- 172.21.15.33:0/755958912 >> 172.21.15.33:0/755958912 conn(0x5611d2d2ab30 msgr2=0x5611d2d2af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- 172.21.15.33:0/755958912 shutdown_connections 2024-04-23T18:58:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- 172.21.15.33:0/755958912 wait complete. 2024-04-23T18:58:10.624 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- start start 2024-04-23T18:58:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 Processor -- start 2024-04-23T18:58:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:10.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5611d2d3c430 con 0x5611d2d39800 2024-04-23T18:58:10.741 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5611d2c2e0b0 con 0x5611d2d2e620 2024-04-23T18:58:10.750 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5611d2d3c5b0 con 0x5611d2d2ef40 2024-04-23T18:58:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.385+0000 7f7624053740 10 client.0 _ll_register_callbacks cb 0x5611d2c1bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39188/0 (socket says 172.21.15.33:39188) 2024-04-23T18:58:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 -- 172.21.15.33:0/1480673362 learned_addr learned my addr 172.21.15.33:0/1480673362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761affd640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 -- 172.21.15.33:0/1480673362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 msgr2=0x5611d2d36440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d36440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 -- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 msgr2=0x5611d2d39150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d39150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 -- 172.21.15.33:0/1480673362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d2a72620 con 0x5611d2d39800 2024-04-23T18:58:10.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f761b7fe640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7614011010 tx=0x7f7614004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2159eb90a823ce server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f75feff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614022070 con 0x5611d2d39800 2024-04-23T18:58:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f761401d8b0 con 0x5611d2d39800 2024-04-23T18:58:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76140044e0 con 0x5611d2d39800 2024-04-23T18:58:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7614036050 con 0x5611d2d39800 2024-04-23T18:58:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f7624053740 10 client.18738 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.386+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5611d2d48190 con 0x5611d2d39800 2024-04-23T18:58:11.097 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76140044e0 con 0x5611d2d39800 2024-04-23T18:58:11.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f7614026740 con 0x5611d2d39800 2024-04-23T18:58:11.125 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 10 client.18738 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:11.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 10 client.18738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:58:11.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 20 client.18738 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:11.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2d495f0 0x5611d2d4ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:11.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 4 client.18738 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-23T18:58:11.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611d2ca1390 con 0x5611d2d495f0 2024-04-23T18:58:11.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f761affd640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2d495f0 0x5611d2d4ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f761affd640 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2d495f0 0x5611d2d4ba00 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e1079e67f56401 server_cookie=209ce300358342c0 in_seq=0 out_seq=0 2024-04-23T18:58:11.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.387+0000 7f75feff5640 10 client.18738 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:11.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f75feff5640 1 -- 172.21.15.33:0/1480673362 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611d2ca1390 con 0x5611d2d495f0 2024-04-23T18:58:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f75feff5640 10 client.18738 handle_command_reply: tid=0 2024-04-23T18:58:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 client.18738 shutdown 2024-04-23T18:58:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 msgr2=0x5611d2d369a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7614011010 tx=0x7f7614004290 comp rx=0 tx=0).stop 2024-04-23T18:58:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 shutdown_connections 2024-04-23T18:58:11.390 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2d495f0 0x5611d2d4ba00 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d39800 0x5611d2d369a0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d2ef40 0x5611d2d36440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f75feff5640 0 client.18738 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:11.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 --2- 172.21.15.33:0/1480673362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d39150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 >> 172.21.15.33:0/1480673362 conn(0x5611d2d2ab30 msgr2=0x5611d2d30860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 shutdown_connections 2024-04-23T18:58:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.388+0000 7f7624053740 1 -- 172.21.15.33:0/1480673362 wait complete. 2024-04-23T18:58:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.389+0000 7f7624053740 20 client.18738 trim_cache size 0 max 16384 2024-04-23T18:58:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 Processor -- start 2024-04-23T18:58:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 -- start start 2024-04-23T18:58:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2ef40 0x5611d2d38c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:11.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d51310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1ac670 con 0x5611d2d2e620 2024-04-23T18:58:11.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1aef50 con 0x5611d2d2ef40 2024-04-23T18:58:11.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.403+0000 7f7624053740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1acd40 con 0x5611d2d39800 2024-04-23T18:58:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39194/0 (socket says 172.21.15.33:39194) 2024-04-23T18:58:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 -- 172.21.15.33:0/437141451 learned_addr learned my addr 172.21.15.33:0/437141451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761b7fe640 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d51310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:11.734 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 -- 172.21.15.33:0/437141451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 msgr2=0x5611d2d51310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:11.742 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d51310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.751 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 -- 172.21.15.33:0/437141451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2ef40 msgr2=0x5611d2d38c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:11.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2ef40 0x5611d2d38c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 -- 172.21.15.33:0/437141451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611d2d36860 con 0x5611d2d2e620 2024-04-23T18:58:11.779 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f761affd640 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f760c0bcd50 tx=0x7f760c03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6249a17f96ee9a6b server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f7600ff9640 1 -- 172.21.15.33:0/437141451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c049040 con 0x5611d2d2e620 2024-04-23T18:58:11.798 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f7600ff9640 1 -- 172.21.15.33:0/437141451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c0c1400 con 0x5611d2d2e620 2024-04-23T18:58:11.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.404+0000 7f7600ff9640 1 -- 172.21.15.33:0/437141451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c03b4f0 con 0x5611d2d2e620 2024-04-23T18:58:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- 172.21.15.33:0/437141451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 msgr2=0x5611d2d452f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f760c0bcd50 tx=0x7f760c03b460 comp rx=0 tx=0).stop 2024-04-23T18:58:11.948 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- 172.21.15.33:0/437141451 shutdown_connections 2024-04-23T18:58:12.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d51310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.023 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2ef40 0x5611d2d38c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- 172.21.15.33:0/437141451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2e620 0x5611d2d452f0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- 172.21.15.33:0/437141451 >> 172.21.15.33:0/437141451 conn(0x5611d2d2ab30 msgr2=0x5611d2d319d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:12.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- 172.21.15.33:0/437141451 shutdown_connections 2024-04-23T18:58:12.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- 172.21.15.33:0/437141451 wait complete. 2024-04-23T18:58:12.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- start start 2024-04-23T18:58:12.083 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 Processor -- start 2024-04-23T18:58:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d41be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:12.102 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d3d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:12.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1ad410 con 0x5611d2d2ef40 2024-04-23T18:58:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1ae880 con 0x5611d2d2e620 2024-04-23T18:58:12.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f761c1adae0 con 0x5611d2d39800 2024-04-23T18:58:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.405+0000 7f7624053740 10 client.0 _ll_register_callbacks cb 0x7f7614003a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-23T18:58:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39208/0 (socket says 172.21.15.33:39208) 2024-04-23T18:58:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 -- 172.21.15.33:0/2945501703 learned_addr learned my addr 172.21.15.33:0/2945501703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761b7fe640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d3d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:12.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761affd640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d41be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 -- 172.21.15.33:0/2945501703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 msgr2=0x5611d2d3d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d3d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 -- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 msgr2=0x5611d2d41be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-23T18:58:12.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d41be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 -- 172.21.15.33:0/2945501703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611d2d3dd30 con 0x5611d2d2ef40 2024-04-23T18:58:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f761a7fc640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7610005fd0 tx=0x7f7610012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c319ef00afb16 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f75feff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-23T18:58:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610004240 con 0x5611d2d2ef40 2024-04-23T18:58:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 8542+0+0 (secure 0 0 0) 0x7f76100043e0 con 0x5611d2d2ef40 2024-04-23T18:58:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f760c0c1400 con 0x5611d2d2ef40 2024-04-23T18:58:12.527 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f76100035f0 con 0x5611d2d2ef40 2024-04-23T18:58:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f7624053740 10 client.18744 fetch_fsmap learned FSMap version 49 2024-04-23T18:58:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.406+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5611d2dafc20 con 0x5611d2d2ef40 2024-04-23T18:58:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f760c0c1400 con 0x5611d2d2ef40 2024-04-23T18:58:12.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 7144+0+0 (secure 0 0 0) 0x7f761001c980 con 0x5611d2d2ef40 2024-04-23T18:58:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 10 client.18744 fetch_fsmap finished waiting for FSMap version 49 2024-04-23T18:58:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-23T18:58:12.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 20 client.18744 populate_metadata read hostname 'smithi033' 2024-04-23T18:58:12.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2db03d0 0x5611d2db27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:12.728 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 4 client.18744 mds_command: new command op to 24551 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-23T18:58:12.737 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 --> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611d2c9b200 con 0x5611d2db03d0 2024-04-23T18:58:12.745 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f761affd640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2db03d0 0x5611d2db27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:12.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f761affd640 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2db03d0 0x5611d2db27f0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a04050ad3f3255 server_cookie=4bdc8e6e4aa4ca35 in_seq=0 out_seq=0 2024-04-23T18:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:01.407+0000 7f75feff5640 10 client.18744 ms_handle_connect on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:12.767 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:01 smithi033 ceph-mon[29274]: pgmap v1025: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-23T18:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.204+0000 7f75feff5640 1 -- 172.21.15.33:0/2945501703 <== mds.0 v2:172.21.15.106:6838/830372376 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611d2c9b200 con 0x5611d2db03d0 2024-04-23T18:58:12.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.204+0000 7f75feff5640 10 client.18744 handle_command_reply: tid=0 2024-04-23T18:58:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.204+0000 7f7624053740 1 client.18744 shutdown 2024-04-23T18:58:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 msgr2=0x5611d2d3cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7610005fd0 tx=0x7f7610012820 comp rx=0 tx=0).stop 2024-04-23T18:58:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 shutdown_connections 2024-04-23T18:58:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:6838/830372376,v1:172.21.15.106:6839/830372376] conn(0x5611d2db03d0 0x5611d2db27f0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611d2d39800 0x5611d2d3d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611d2d2ef40 0x5611d2d3cff0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f75feff5640 0 client.18744 ms_handle_reset on v2:172.21.15.106:6838/830372376 2024-04-23T18:58:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 --2- 172.21.15.33:0/2945501703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5611d2d2e620 0x5611d2d41be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 >> 172.21.15.33:0/2945501703 conn(0x5611d2d2ab30 msgr2=0x5611d2d442e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 shutdown_connections 2024-04-23T18:58:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 1 -- 172.21.15.33:0/2945501703 wait complete. 2024-04-23T18:58:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.205+0000 7f7624053740 20 client.18744 trim_cache size 0 max 16384 2024-04-23T18:58:13.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-04-23T18:58:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-04-23T18:58:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-04-23T18:58:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-04-23T18:58:13.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:09 smithi106 ceph-mon[32716]: pgmap v1030: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:13.172 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:11 smithi106 ceph-mon[32716]: pgmap v1031: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:13.183 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:11 smithi106 ceph-mon[32716]: osdmap e152: 12 total, 12 up, 12 in 2024-04-23T18:58:13.195 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:09 smithi122 ceph-mon[32546]: pgmap v1030: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:13.305 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:11 smithi122 ceph-mon[32546]: pgmap v1031: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:13.316 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:11 smithi122 ceph-mon[32546]: osdmap e152: 12 total, 12 up, 12 in 2024-04-23T18:58:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 msgr2=0x7f76040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:13.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 0x7f76040a6280 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f761401f000 tx=0x7f761401d000 comp rx=0 tx=0).stop 2024-04-23T18:58:13.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 msgr2=0x7f761c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:13.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f760c005ab0 tx=0x7f760c003c80 comp rx=0 tx=0).stop 2024-04-23T18:58:13.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 shutdown_connections 2024-04-23T18:58:13.369 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f76040a3dc0 0x7f76040a6280 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f761c10c550 0x7f761c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761c103aa0 0x7f761c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 --2- 172.21.15.33:0/2415727324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f761c103180 0x7f761c1986a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:13.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 >> 172.21.15.33:0/2415727324 conn(0x7f761c077fd0 msgr2=0x7f761c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 shutdown_connections 2024-04-23T18:58:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-04-23T18:58:03.206+0000 7f7621336640 1 -- 172.21.15.33:0/2415727324 wait complete. 2024-04-23T18:58:13.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-mon[29274]: pgmap v1026: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T18:58:13.508 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: *** Caught signal (Segmentation fault) ** 2024-04-23T18:58:13.519 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: in thread 7fbd8a07d700 thread_name:md_log_replay 2024-04-23T18:58:13.586 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:58:13.595 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbd9b43ed20] 2024-04-23T18:58:13.606 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5647e540738f] 2024-04-23T18:58:13.617 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5647e540ab23] 2024-04-23T18:58:13.625 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 4: (MDLog::_replay_thread()+0x617) [0x5647e5465647] 2024-04-23T18:58:13.636 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5647e546ed31] 2024-04-23T18:58:13.647 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fbd9b4341ca] 2024-04-23T18:58:13.656 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 7: clone() 2024-04-23T18:58:13.665 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:03.578+0000 7fbd8a07d700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:58:13.674 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: in thread 7fbd8a07d700 thread_name:md_log_replay 2024-04-23T18:58:13.683 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:13.693 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:58:13.704 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbd9b43ed20] 2024-04-23T18:58:13.762 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5647e540738f] 2024-04-23T18:58:13.771 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5647e540ab23] 2024-04-23T18:58:13.780 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 4: (MDLog::_replay_thread()+0x617) [0x5647e5465647] 2024-04-23T18:58:13.789 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5647e546ed31] 2024-04-23T18:58:13.855 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fbd9b4341ca] 2024-04-23T18:58:13.865 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 7: clone() 2024-04-23T18:58:13.873 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:58:13.883 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:13.891 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 0> 2024-04-23T18:58:03.578+0000 7fbd8a07d700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:58:13.900 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: in thread 7fbd8a07d700 thread_name:md_log_replay 2024-04-23T18:58:13.909 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:13.918 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:58:13.926 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbd9b43ed20] 2024-04-23T18:58:13.935 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5647e540738f] 2024-04-23T18:58:13.943 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5647e540ab23] 2024-04-23T18:58:13.952 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 4: (MDLog::_replay_thread()+0x617) [0x5647e5465647] 2024-04-23T18:58:13.961 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5647e546ed31] 2024-04-23T18:58:14.016 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fbd9b4341ca] 2024-04-23T18:58:14.023 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 7: clone() 2024-04-23T18:58:14.031 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:58:14.038 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:14.107 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 0> 2024-04-23T18:58:03.578+0000 7fbd8a07d700 -1 *** Caught signal (Segmentation fault) ** 2024-04-23T18:58:14.115 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: in thread 7fbd8a07d700 thread_name:md_log_replay 2024-04-23T18:58:14.123 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:14.132 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-23T18:58:14.140 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbd9b43ed20] 2024-04-23T18:58:14.148 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2: /usr/bin/ceph-mds(+0x54938f) [0x5647e540738f] 2024-04-23T18:58:14.157 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5647e540ab23] 2024-04-23T18:58:14.165 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 4: (MDLog::_replay_thread()+0x617) [0x5647e5465647] 2024-04-23T18:58:14.174 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5647e546ed31] 2024-04-23T18:58:14.182 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fbd9b4341ca] 2024-04-23T18:58:14.191 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 7: clone() 2024-04-23T18:58:14.200 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-23T18:58:14.216 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[101656]: 2024-04-23T18:58:14.277 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:03 smithi033 podman[102787]: 2024-04-23 18:58:03.630880443 +0000 UTC m=+0.023501156 container died 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-23T18:58:14.286 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:04 smithi033 podman[102787]: 2024-04-23 18:58:04.351706878 +0000 UTC m=+0.744327635 container remove 9c618b5549a5ca60cb767197211363667479b4e3365fa9c4e551c2bfc1244460 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-23T18:58:14.295 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=139/n/a 2024-04-23T18:58:14.305 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:04 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-23T18:58:14.387 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-23T18:58:14.413 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-23T18:58:14.422 INFO:teuthology.task.exec:Executing custom commands... 2024-04-23T18:58:14.431 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T18:58:14.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-23T18:58:14.450 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:13 smithi106 ceph-mon[32716]: pgmap v1033: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:58:14.461 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:13 smithi122 ceph-mon[32546]: pgmap v1033: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:58:14.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:05 smithi033 ceph-mon[29274]: pgmap v1027: 97 pgs: 97 active+clean; 15 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T18:58:14.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:05 smithi033 ceph-mon[29274]: osdmap e151: 12 total, 12 up, 12 in 2024-04-23T18:58:14.516 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:07 smithi033 ceph-mon[29274]: pgmap v1029: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:14.525 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:09 smithi033 ceph-mon[29274]: pgmap v1030: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:14.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:11 smithi033 ceph-mon[29274]: pgmap v1031: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-04-23T18:58:14.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:11 smithi033 ceph-mon[29274]: osdmap e152: 12 total, 12 up, 12 in 2024-04-23T18:58:14.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:13 smithi033 ceph-mon[29274]: pgmap v1033: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 67 KiB/s wr, 37 op/s 2024-04-23T18:58:14.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-23T18:58:14.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-23T18:58:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 -- 172.21.15.33:0/2812199697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 msgr2=0x7fd6781035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:14.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/2812199697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd6781035e0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd668009ca0 tx=0x7fd66802a950 comp rx=0 tx=0).stop 2024-04-23T18:58:14.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 -- 172.21.15.33:0/2812199697 shutdown_connections 2024-04-23T18:58:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/2812199697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd67810c5d0 0x7fd67810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/2812199697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678103b20 0x7fd67810c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/2812199697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd6781035e0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:14.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 -- 172.21.15.33:0/2812199697 >> 172.21.15.33:0/2812199697 conn(0x7fd6780771d0 msgr2=0x7fd6780775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 -- 172.21.15.33:0/2812199697 shutdown_connections 2024-04-23T18:58:14.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.910+0000 7fd67d7f0640 1 -- 172.21.15.33:0/2812199697 wait complete. 2024-04-23T18:58:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 Processor -- start 2024-04-23T18:58:15.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 -- start start 2024-04-23T18:58:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd678103b20 0x7fd678101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:15.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67810c5d0 0x7fd6781018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:15.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd678112200 con 0x7fd678103200 2024-04-23T18:58:15.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd678112080 con 0x7fd678103b20 2024-04-23T18:58:15.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd67d7f0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd678112380 con 0x7fd67810c5d0 2024-04-23T18:58:15.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:15.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47466/0 (socket says 172.21.15.33:47466) 2024-04-23T18:58:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 -- 172.21.15.33:0/1850366080 learned_addr learned my addr 172.21.15.33:0/1850366080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd6767fc640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd678103b20 0x7fd678101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:15.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 -- 172.21.15.33:0/1850366080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67810c5d0 msgr2=0x7fd6781018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:15.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67810c5d0 0x7fd6781018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 -- 172.21.15.33:0/1850366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd678103b20 msgr2=0x7fd678101390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:15.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd678103b20 0x7fd678101390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.911+0000 7fd676ffd640 1 -- 172.21.15.33:0/1850366080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6781a6b60 con 0x7fd678103200 2024-04-23T18:58:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd676ffd640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd668005cc0 tx=0x7fd66802f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:15.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668034040 con 0x7fd678103200 2024-04-23T18:58:15.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6680308f0 con 0x7fd678103200 2024-04-23T18:58:15.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6781a6df0 con 0x7fd678103200 2024-04-23T18:58:15.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680047c0 con 0x7fd678103200 2024-04-23T18:58:15.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.912+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6781a7280 con 0x7fd678103200 2024-04-23T18:58:15.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.913+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd67804fce0 con 0x7fd678103200 2024-04-23T18:58:15.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.914+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd668037020 con 0x7fd678103200 2024-04-23T18:58:15.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.915+0000 7fd65ffff640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 0x7fd6600a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:15.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.915+0000 7fd6767fc640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 0x7fd6600a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:15.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.915+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 8542+0+0 (secure 0 0 0) 0x7fd6680f17e0 con 0x7fd678103200 2024-04-23T18:58:15.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.915+0000 7fd6767fc640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 0x7fd6600a62b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd678102890 tx=0x7fd66c00a4c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:14.919+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6680bdf80 con 0x7fd678103200 2024-04-23T18:58:15.549 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:14 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 3. 2024-04-23T18:58:15.568 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:14 smithi033 systemd[1]: Stopped Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:15.577 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:14 smithi033 systemd[1]: Starting Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T18:58:15.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.131+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd67802d630 con 0x7fd6600a3df0 2024-04-23T18:58:15.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.139+0000 7fd65ffff640 1 -- 172.21.15.33:0/1850366080 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fd67802d630 con 0x7fd6600a3df0 2024-04-23T18:58:15.606 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-23T00:00:00", "created": "2024-04-23T18:26:30", "first": "2024-04-23T18:27:00", "last": "2024-04-23T18:58:00", "last_pruned": "2024-04-23T18:58:01", "created_count": 32, "pruned_count": 26, "active": true} 2024-04-23T18:58:15.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 msgr2=0x7fd6600a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 0x7fd6600a62b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd678102890 tx=0x7fd66c00a4c0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 msgr2=0x7fd678100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:15.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd668005cc0 tx=0x7fd66802f650 comp rx=0 tx=0).stop 2024-04-23T18:58:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 shutdown_connections 2024-04-23T18:58:15.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fd6600a3df0 0x7fd6600a62b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd67810c5d0 0x7fd6781018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd678103b20 0x7fd678101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 --2- 172.21.15.33:0/1850366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678103200 0x7fd678100e50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 >> 172.21.15.33:0/1850366080 conn(0x7fd6780771d0 msgr2=0x7fd6780fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 shutdown_connections 2024-04-23T18:58:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:15.141+0000 7fd67d7f0640 1 -- 172.21.15.33:0/1850366080 wait complete. 2024-04-23T18:58:15.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-23T18:58:15.841 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:14 smithi033 podman[102926]: 2024-04-23 18:58:14.83810318 +0000 UTC m=+0.020099831 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T18:58:15.850 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:14 smithi033 podman[102926]: 2024-04-23 18:58:14.960662476 +0000 UTC m=+0.142659112 container create 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T18:58:15.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:15 smithi033 ceph-mon[29274]: pgmap v1034: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:58:15.868 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 podman[102926]: 2024-04-23 18:58:15.256128508 +0000 UTC m=+0.438125145 container init 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-23T18:58:15.876 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 podman[102926]: 2024-04-23 18:58:15.259002285 +0000 UTC m=+0.440998930 container start 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T18:58:15.885 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 ceph-mds[102979]: -- 172.21.15.33:0/3009572791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56533cc05a40 con 0x56533cbd8c00 2024-04-23T18:58:15.894 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: starting mds.j at 2024-04-23T18:58:15.950 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 bash[102926]: 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd 2024-04-23T18:58:15.958 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 18:58:15 smithi033 systemd[1]: Started Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T18:58:15.968 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:15 smithi122 ceph-mon[32546]: pgmap v1034: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:58:15.977 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:15 smithi106 ceph-mon[32716]: pgmap v1034: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-23T18:58:16.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 -- 172.21.15.33:0/3919160570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f241016a0 msgr2=0x7f0f2410e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:16.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 --2- 172.21.15.33:0/3919160570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f241016a0 0x7f0f2410e3f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009900 tx=0x7f0f1802a410 comp rx=0 tx=0).stop 2024-04-23T18:58:16.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 -- 172.21.15.33:0/3919160570 shutdown_connections 2024-04-23T18:58:16.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 --2- 172.21.15.33:0/3919160570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f2410e930 0x7f0f24110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 --2- 172.21.15.33:0/3919160570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f241016a0 0x7f0f2410e3f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 --2- 172.21.15.33:0/3919160570 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f24101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 -- 172.21.15.33:0/3919160570 >> 172.21.15.33:0/3919160570 conn(0x7f0f24077fd0 msgr2=0x7f0f240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:16.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 -- 172.21.15.33:0/3919160570 shutdown_connections 2024-04-23T18:58:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.075+0000 7f0f2af20640 1 -- 172.21.15.33:0/3919160570 wait complete. 2024-04-23T18:58:16.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 Processor -- start 2024-04-23T18:58:16.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 -- start start 2024-04-23T18:58:16.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f241a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:16.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f241016a0 0x7f0f241a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:16.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:16.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24114410 con 0x7f0f2410e930 2024-04-23T18:58:16.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24114290 con 0x7f0f24100d80 2024-04-23T18:58:16.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f2af20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f24114590 con 0x7f0f241016a0 2024-04-23T18:58:16.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f294bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f294bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47494/0 (socket says 172.21.15.33:47494) 2024-04-23T18:58:16.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f294bd640 1 -- 172.21.15.33:0/4165437137 learned_addr learned my addr 172.21.15.33:0/4165437137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:16.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f28cbc640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f241a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:16.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.076+0000 7f0f23fff640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f241016a0 0x7f0f241a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 -- 172.21.15.33:0/4165437137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f241016a0 msgr2=0x7f0f241a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f241016a0 0x7f0f241a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 -- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 msgr2=0x7f0f2419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:16.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 -- 172.21.15.33:0/4165437137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2419b010 con 0x7f0f24100d80 2024-04-23T18:58:16.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f294bd640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-23T18:58:16.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f28cbc640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f241a0690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e9f0 tx=0x7f0f1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:16.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400ccc0 con 0x7f0f24100d80 2024-04-23T18:58:16.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f140108e0 con 0x7f0f24100d80 2024-04-23T18:58:16.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2419b300 con 0x7f0f24100d80 2024-04-23T18:58:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f14018880 con 0x7f0f24100d80 2024-04-23T18:58:16.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.077+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f241a7610 con 0x7f0f24100d80 2024-04-23T18:58:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.078+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2419b640 con 0x7f0f24100d80 2024-04-23T18:58:16.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.079+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0f14010430 con 0x7f0f24100d80 2024-04-23T18:58:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.080+0000 7f0f21ffb640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 0x7f0f0c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:16.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.080+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 8542+0+0 (secure 0 0 0) 0x7f0f140d0bc0 con 0x7f0f24100d80 2024-04-23T18:58:16.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.080+0000 7f0f23fff640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 0x7f0f0c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:16.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.081+0000 7f0f23fff640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 0x7f0f0c0a6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005ab0 tx=0x7f0f18005a20 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:16.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.084+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f1409cc00 con 0x7f0f24100d80 2024-04-23T18:58:16.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.301+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 --> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0f24062390 con 0x7f0f0c0a3dc0 2024-04-23T18:58:16.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.323+0000 7f0f21ffb640 1 -- 172.21.15.33:0/4165437137 <== mgr.15357 v2:172.21.15.33:6800/3751231116 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0f24062390 con 0x7f0f0c0a3dc0 2024-04-23T18:58:16.707 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-04-23T18:58:16.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 msgr2=0x7f0f0c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:16.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 0x7f0f0c0a6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0f18005ab0 tx=0x7f0f18005a20 comp rx=0 tx=0).stop 2024-04-23T18:58:16.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 msgr2=0x7f0f241a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:16.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f241a0690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400e9f0 tx=0x7f0f1400eeb0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 shutdown_connections 2024-04-23T18:58:16.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f0f0c0a3dc0 0x7f0f0c0a6280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f2410e930 0x7f0f2419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f241016a0 0x7f0f241a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 --2- 172.21.15.33:0/4165437137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f24100d80 0x7f0f241a0690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:16.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 >> 172.21.15.33:0/4165437137 conn(0x7f0f24077fd0 msgr2=0x7f0f2410fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:16.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.325+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 shutdown_connections 2024-04-23T18:58:16.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:16.326+0000 7f0f2af20640 1 -- 172.21.15.33:0/4165437137 wait complete. 2024-04-23T18:58:16.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: from='client.18750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:16.980 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: daemon mds.j restarted 2024-04-23T18:58:16.991 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:58:17.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: osdmap e153: 12 total, 12 up, 12 in 2024-04-23T18:58:17.013 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6834/217544075,v1:172.21.15.33:6835/217544075] up:boot 2024-04-23T18:58:17.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:58:17.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:16 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:58:17.040 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-23T18:58:17.115 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T18:58:17.127 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: from='client.18750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:17.137 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: daemon mds.j restarted 2024-04-23T18:58:17.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:58:17.245 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: osdmap e153: 12 total, 12 up, 12 in 2024-04-23T18:58:17.254 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6834/217544075,v1:172.21.15.33:6835/217544075] up:boot 2024-04-23T18:58:17.262 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:58:17.271 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:16 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:58:17.277 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: from='client.18750 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:17.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: daemon mds.j restarted 2024-04-23T18:58:17.292 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T18:58:17.299 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: osdmap e153: 12 total, 12 up, 12 in 2024-04-23T18:58:17.308 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6834/217544075,v1:172.21.15.33:6835/217544075] up:boot 2024-04-23T18:58:17.316 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: fsmap cephfs:5 5 up:standby-replay 5 up:active 2024-04-23T18:58:17.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:16 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T18:58:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 -- 172.21.15.33:0/3627030928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc102240 msgr2=0x7fa9fc102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:17.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 --2- 172.21.15.33:0/3627030928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc102240 0x7fa9fc102620 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009900 tx=0x7fa9ec02a410 comp rx=0 tx=0).stop 2024-04-23T18:58:17.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 -- 172.21.15.33:0/3627030928 shutdown_connections 2024-04-23T18:58:17.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 --2- 172.21.15.33:0/3627030928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc107280 0x7fa9fc10ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:17.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 --2- 172.21.15.33:0/3627030928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102bf0 0x7fa9fc106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:17.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 --2- 172.21.15.33:0/3627030928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc102240 0x7fa9fc102620 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 -- 172.21.15.33:0/3627030928 >> 172.21.15.33:0/3627030928 conn(0x7fa9fc0790e0 msgr2=0x7fa9fc0794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:17.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 -- 172.21.15.33:0/3627030928 shutdown_connections 2024-04-23T18:58:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 -- 172.21.15.33:0/3627030928 wait complete. 2024-04-23T18:58:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.436+0000 7faa0286b640 1 Processor -- start 2024-04-23T18:58:17.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 -- start start 2024-04-23T18:58:17.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102240 0x7fa9fc19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:17.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:17.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc1985a0 0x7fa9fc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:17.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc1121e0 con 0x7fa9fc107280 2024-04-23T18:58:17.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc112060 con 0x7fa9fc102240 2024-04-23T18:58:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa0286b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9fc112360 con 0x7fa9fc1985a0 2024-04-23T18:58:17.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:17.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44660/0 (socket says 172.21.15.33:44660) 2024-04-23T18:58:17.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 -- 172.21.15.33:0/3600297455 learned_addr learned my addr 172.21.15.33:0/3600297455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:17.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7faa00e08640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc1985a0 0x7fa9fc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:17.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fbfff640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102240 0x7fa9fc19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:17.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 -- 172.21.15.33:0/3600297455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc1985a0 msgr2=0x7fa9fc198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:17.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc1985a0 0x7fa9fc198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:17.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 -- 172.21.15.33:0/3600297455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102240 msgr2=0x7fa9fc19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:17.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102240 0x7fa9fc19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:17.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.437+0000 7fa9fb7fe640 1 -- 172.21.15.33:0/3600297455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9fc1990c0 con 0x7fa9fc107280 2024-04-23T18:58:17.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7fa9fb7fe640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000d740 tx=0x7fa9f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0017070 con 0x7fa9fc107280 2024-04-23T18:58:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f000de50 con 0x7fa9fc107280 2024-04-23T18:58:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9fc1993b0 con 0x7fa9fc107280 2024-04-23T18:58:17.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f000f620 con 0x7fa9fc107280 2024-04-23T18:58:17.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.438+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9fc1a5600 con 0x7fa9fc107280 2024-04-23T18:58:18.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.439+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9fc199ad0 con 0x7fa9fc107280 2024-04-23T18:58:18.012 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: pgmap v1035: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T18:58:18.020 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='client.27854 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:18.070 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.081 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.092 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:58:18.103 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:58:18.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: pgmap v1035: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T18:58:18.187 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='client.27854 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:18.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.220 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:58:18.231 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:58:18.242 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.440+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9f001a070 con 0x7fa9fc107280 2024-04-23T18:58:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.441+0000 7fa9f97fa640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.441+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 8498+0+0 (secure 0 0 0) 0x7fa9f00d00b0 con 0x7fa9fc107280 2024-04-23T18:58:18.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.441+0000 7fa9fbfff640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.441+0000 7fa9fbfff640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec031440 tx=0x7fa9ec005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:18.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.444+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9f009c1b0 con 0x7fa9fc107280 2024-04-23T18:58:18.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.633+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9fc0691a0 con 0x7fa9fc107280 2024-04-23T18:58:18.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.635+0000 7fa9f97fa640 1 -- 172.21.15.33:0/3600297455 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+23127 (secure 0 0 0) 0x7fa9fc0691a0 con 0x7fa9fc107280 2024-04-23T18:58:18.383 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:58:18.445 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":153,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T18:58:15.287249+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":59,"snap_epoch":151,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T18:58:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 msgr2=0x7fa9e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:18.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec031440 tx=0x7fa9ec005b50 comp rx=0 tx=0).stop 2024-04-23T18:58:18.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 msgr2=0x7fa9fc19e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:18.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa9f000d740 tx=0x7fa9f000dc00 comp rx=0 tx=0).stop 2024-04-23T18:58:18.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 shutdown_connections 2024-04-23T18:58:18.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fa9e40a3d20 0x7fa9e40a61e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:18.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9fc1985a0 0x7fa9fc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:18.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9fc107280 0x7fa9fc19e930 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 --2- 172.21.15.33:0/3600297455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa9fc102240 0x7fa9fc19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:18.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 >> 172.21.15.33:0/3600297455 conn(0x7fa9fc0790e0 msgr2=0x7fa9fc100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:18.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 shutdown_connections 2024-04-23T18:58:18.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:17.637+0000 7faa0286b640 1 -- 172.21.15.33:0/3600297455 wait complete. 2024-04-23T18:58:18.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: pgmap v1035: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T18:58:18.644 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='client.27854 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-23T18:58:18.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.728 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T18:58:18.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T18:58:18.741 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T18:58:18.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:18 smithi106 ceph-mon[32716]: pgmap v1037: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 138 B/s wr, 12 op/s 2024-04-23T18:58:18.756 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:18 smithi106 ceph-mon[32716]: pgmap v1038: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:58:18.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:18 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/3600297455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:58:18.770 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:18 smithi122 ceph-mon[32546]: pgmap v1037: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 138 B/s wr, 12 op/s 2024-04-23T18:58:18.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:18 smithi122 ceph-mon[32546]: pgmap v1038: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:58:18.786 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:18 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/3600297455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:58:18.819 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 59, 'snap_epoch': 151, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T18:58:18.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T18:58:18.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:18 smithi033 ceph-mon[29274]: pgmap v1037: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 138 B/s wr, 12 op/s 2024-04-23T18:58:18.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:18 smithi033 ceph-mon[29274]: pgmap v1038: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T18:58:18.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:18 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/3600297455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T18:58:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 -- 172.21.15.33:0/2055786122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac107280 msgr2=0x7f35ac10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 --2- 172.21.15.33:0/2055786122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac107280 0x7f35ac10ee30 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009ca0 tx=0x7f35a402a8c0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 -- 172.21.15.33:0/2055786122 shutdown_connections 2024-04-23T18:58:19.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 --2- 172.21.15.33:0/2055786122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac107280 0x7f35ac10ee30 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 --2- 172.21.15.33:0/2055786122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac06b170 0x7f35ac106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 --2- 172.21.15.33:0/2055786122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac06a7c0 0x7f35ac06aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 -- 172.21.15.33:0/2055786122 >> 172.21.15.33:0/2055786122 conn(0x7f35ac076c50 msgr2=0x7f35ac077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 -- 172.21.15.33:0/2055786122 shutdown_connections 2024-04-23T18:58:19.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.017+0000 7f35b1bad640 1 -- 172.21.15.33:0/2055786122 wait complete. 2024-04-23T18:58:19.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 Processor -- start 2024-04-23T18:58:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 -- start start 2024-04-23T18:58:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:19.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 0x7f35ac1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:19.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac101cf0 0x7f35ac1a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:19.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1121e0 con 0x7f35ac06a7c0 2024-04-23T18:58:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac112060 con 0x7f35ac107280 2024-04-23T18:58:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35b1bad640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac112360 con 0x7f35ac101cf0 2024-04-23T18:58:19.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44686/0 (socket says 172.21.15.33:44686) 2024-04-23T18:58:19.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1693614338 learned_addr learned my addr 172.21.15.33:0/1693614338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:19.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35aaffd640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 0x7f35ac1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35abfff640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac101cf0 0x7f35ac1a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:19.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1693614338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac101cf0 msgr2=0x7f35ac1a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:19.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac101cf0 0x7f35ac1a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1693614338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 msgr2=0x7f35ac1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:19.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 0x7f35ac1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1693614338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1a6f90 con 0x7f35ac06a7c0 2024-04-23T18:58:19.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.018+0000 7f35aaffd640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 0x7f35ac1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:19.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35ab7fe640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:19.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c017070 con 0x7f35ac06a7c0 2024-04-23T18:58:19.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1a7280 con 0x7f35ac06a7c0 2024-04-23T18:58:19.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f359c009c30 con 0x7f35ac06a7c0 2024-04-23T18:58:19.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c00f620 con 0x7f35ac06a7c0 2024-04-23T18:58:19.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.019+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1a7820 con 0x7f35ac06a7c0 2024-04-23T18:58:19.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.020+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac103680 con 0x7f35ac06a7c0 2024-04-23T18:58:19.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.021+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f359c005030 con 0x7f35ac06a7c0 2024-04-23T18:58:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.022+0000 7f35a8ff9640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 0x7f35940a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:19.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.022+0000 7f35aaffd640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 0x7f35940a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.022+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 8498+0+0 (secure 0 0 0) 0x7f359c0d04a0 con 0x7f35ac06a7c0 2024-04-23T18:58:19.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.022+0000 7f35aaffd640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 0x7f35940a6210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005d70 tx=0x7f35a0005d00 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:19.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.026+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f359c09cb50 con 0x7f35ac06a7c0 2024-04-23T18:58:19.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.212+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35ac06b170 con 0x7f35ac06a7c0 2024-04-23T18:58:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.212+0000 7f35a8ff9640 1 -- 172.21.15.33:0/1693614338 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) ==== 93+0+10 (secure 0 0 0) 0x7f359c09c450 con 0x7f35ac06a7c0 2024-04-23T18:58:19.648 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T18:58:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 msgr2=0x7f35940a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 0x7f35940a6210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35a0005d70 tx=0x7f35a0005d00 comp rx=0 tx=0).stop 2024-04-23T18:58:19.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 msgr2=0x7f35ac101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f359c00d950 tx=0x7f359c00de10 comp rx=0 tx=0).stop 2024-04-23T18:58:19.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 shutdown_connections 2024-04-23T18:58:19.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f35940a3d50 0x7f35940a6210 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac101cf0 0x7f35ac1a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35ac107280 0x7f35ac1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 --2- 172.21.15.33:0/1693614338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35ac06a7c0 0x7f35ac101270 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:19.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 >> 172.21.15.33:0/1693614338 conn(0x7f35ac076c50 msgr2=0x7f35ac0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:19.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 shutdown_connections 2024-04-23T18:58:19.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:19.214+0000 7f35b1bad640 1 -- 172.21.15.33:0/1693614338 wait complete. 2024-04-23T18:58:19.875 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T18:58:19.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:19 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1693614338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:58:19.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:19 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1693614338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:58:19.899 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:19 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1693614338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T18:58:20.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 -- 172.21.15.33:0/619630608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec102b50 msgr2=0x7f66ec106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:20.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 --2- 172.21.15.33:0/619630608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec102b50 0x7f66ec106b90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009900 tx=0x7f66dc02a410 comp rx=0 tx=0).stop 2024-04-23T18:58:20.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 -- 172.21.15.33:0/619630608 shutdown_connections 2024-04-23T18:58:20.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 --2- 172.21.15.33:0/619630608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec107270 0x7f66ec10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 --2- 172.21.15.33:0/619630608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec102b50 0x7f66ec106b90 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 --2- 172.21.15.33:0/619630608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 0x7f66ec102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 -- 172.21.15.33:0/619630608 >> 172.21.15.33:0/619630608 conn(0x7f66ec0793e0 msgr2=0x7f66ec0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:20.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 -- 172.21.15.33:0/619630608 shutdown_connections 2024-04-23T18:58:20.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.006+0000 7f66f2e10640 1 -- 172.21.15.33:0/619630608 wait complete. 2024-04-23T18:58:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 Processor -- start 2024-04-23T18:58:20.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 -- start start 2024-04-23T18:58:20.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 0x7f66ec1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:20.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec19cf70 0x7f66ec19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec1121b0 con 0x7f66ec107270 2024-04-23T18:58:20.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec112030 con 0x7f66ec1021a0 2024-04-23T18:58:20.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f2e10640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec112330 con 0x7f66ec19cf70 2024-04-23T18:58:20.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:20.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44704/0 (socket says 172.21.15.33:44704) 2024-04-23T18:58:20.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66e3fff640 1 -- 172.21.15.33:0/1478614840 learned_addr learned my addr 172.21.15.33:0/1478614840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:20.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f13ad640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec19cf70 0x7f66ec19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:20.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.007+0000 7f66f0bac640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 0x7f66ec1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:20.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 -- 172.21.15.33:0/1478614840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec19cf70 msgr2=0x7f66ec19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:20.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec19cf70 0x7f66ec19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 -- 172.21.15.33:0/1478614840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 msgr2=0x7f66ec1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 0x7f66ec1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 -- 172.21.15.33:0/1478614840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec19da60 con 0x7f66ec107270 2024-04-23T18:58:20.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e3fff640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f66dc031a90 tx=0x7f66dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc03f030 con 0x7f66ec107270 2024-04-23T18:58:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66dc003cb0 con 0x7f66ec107270 2024-04-23T18:58:20.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec19dcf0 con 0x7f66ec107270 2024-04-23T18:58:20.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc035070 con 0x7f66ec107270 2024-04-23T18:58:20.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.008+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec1a54b0 con 0x7f66ec107270 2024-04-23T18:58:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.009+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec0691c0 con 0x7f66ec107270 2024-04-23T18:58:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.010+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f66dc042030 con 0x7f66ec107270 2024-04-23T18:58:20.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.011+0000 7f66e1ffb640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 0x7f66d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.011+0000 7f66f0bac640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 0x7f66d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.011+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 8498+0+0 (secure 0 0 0) 0x7f66dc0f0de0 con 0x7f66ec107270 2024-04-23T18:58:20.681 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:20 smithi106 ceph-mon[32716]: pgmap v1039: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T18:58:20.690 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:20 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1478614840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:58:20.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.012+0000 7f66f0bac640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 0x7f66d40a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005e10 tx=0x7f66e8005da0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:20.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.016+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66dc0bd430 con 0x7f66ec107270 2024-04-23T18:58:20.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.201+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f66ec102b50 con 0x7f66ec107270 2024-04-23T18:58:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.201+0000 7f66e1ffb640 1 -- 172.21.15.33:0/1478614840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v153) ==== 104+0+11 (secure 0 0 0) 0x7f66dc0bcd30 con 0x7f66ec107270 2024-04-23T18:58:20.788 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T18:58:20.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 msgr2=0x7f66d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:20.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 0x7f66d40a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f66e8005e10 tx=0x7f66e8005da0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 msgr2=0x7f66ec198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f66dc031a90 tx=0x7f66dc004270 comp rx=0 tx=0).stop 2024-04-23T18:58:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 shutdown_connections 2024-04-23T18:58:20.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f66d40a3d20 0x7f66d40a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66ec19cf70 0x7f66ec19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66ec107270 0x7f66ec198bf0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 --2- 172.21.15.33:0/1478614840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f66ec1021a0 0x7f66ec1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:20.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 >> 172.21.15.33:0/1478614840 conn(0x7f66ec0793e0 msgr2=0x7f66ec0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:20.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 shutdown_connections 2024-04-23T18:58:20.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:20.203+0000 7f66f2e10640 1 -- 172.21.15.33:0/1478614840 wait complete. 2024-04-23T18:58:20.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T18:58:21.021 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:20 smithi033 ceph-mon[29274]: pgmap v1039: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T18:58:21.063 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:20 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1478614840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:58:21.072 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:20 smithi122 ceph-mon[32546]: pgmap v1039: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-23T18:58:21.080 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:20 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1478614840' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T18:58:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 -- 172.21.15.33:0/2377975458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f090806b1f0 msgr2=0x7f0908106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 --2- 172.21.15.33:0/2377975458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f090806b1f0 0x7f0908106b90 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009900 tx=0x7f08fc02a410 comp rx=0 tx=0).stop 2024-04-23T18:58:21.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 -- 172.21.15.33:0/2377975458 shutdown_connections 2024-04-23T18:58:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 --2- 172.21.15.33:0/2377975458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0908107270 0x7f090810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 --2- 172.21.15.33:0/2377975458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f090806b1f0 0x7f0908106b90 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 --2- 172.21.15.33:0/2377975458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f090806a840 0x7f090806ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:21.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 -- 172.21.15.33:0/2377975458 >> 172.21.15.33:0/2377975458 conn(0x7f0908077eb0 msgr2=0x7f09080782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:21.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 -- 172.21.15.33:0/2377975458 shutdown_connections 2024-04-23T18:58:21.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.192+0000 7f090e831640 1 -- 172.21.15.33:0/2377975458 wait complete. 2024-04-23T18:58:21.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 Processor -- start 2024-04-23T18:58:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 -- start start 2024-04-23T18:58:21.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 0x7f090819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:21.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 0x7f0908198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0908112210 con 0x7f0908107270 2024-04-23T18:58:21.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0908112090 con 0x7f090806a840 2024-04-23T18:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f090e831640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0908112390 con 0x7f09081985f0 2024-04-23T18:58:21.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f09077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:21.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f09077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44714/0 (socket says 172.21.15.33:44714) 2024-04-23T18:58:21.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.193+0000 7f09077fe640 1 -- 172.21.15.33:0/1674805050 learned_addr learned my addr 172.21.15.33:0/1674805050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f090cdce640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 0x7f0908198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:21.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f0907fff640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 0x7f090819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:21.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 -- 172.21.15.33:0/1674805050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 msgr2=0x7f0908198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:21.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 0x7f0908198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:21.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 -- 172.21.15.33:0/1674805050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 msgr2=0x7f090819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 0x7f090819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:21.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 -- 172.21.15.33:0/1674805050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0908199110 con 0x7f0908107270 2024-04-23T18:58:21.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f090cdce640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 0x7f0908198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f0907fff640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 0x7f090819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T18:58:21.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09077fe640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f08fc032fd0 tx=0x7f08fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc03f030 con 0x7f0908107270 2024-04-23T18:58:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08fc004490 con 0x7f0908107270 2024-04-23T18:58:21.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09081993a0 con 0x7f0908107270 2024-04-23T18:58:21.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc035070 con 0x7f0908107270 2024-04-23T18:58:21.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.194+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a5590 con 0x7f0908107270 2024-04-23T18:58:21.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.195+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09080691c0 con 0x7f0908107270 2024-04-23T18:58:21.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.197+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f08fc042030 con 0x7f0908107270 2024-04-23T18:58:21.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.197+0000 7f09057fa640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:21.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.197+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 8498+0+0 (secure 0 0 0) 0x7f08fc0f1100 con 0x7f0908107270 2024-04-23T18:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.197+0000 7f0907fff640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:21.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.198+0000 7f0907fff640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 0x7f08f00a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009930 tx=0x7f08f8008040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:21.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.201+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08fc0bd850 con 0x7f0908107270 2024-04-23T18:58:21.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.390+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f090806b1f0 con 0x7f0908107270 2024-04-23T18:58:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.390+0000 7f09057fa640 1 -- 172.21.15.33:0/1674805050 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v153) ==== 100+0+11 (secure 0 0 0) 0x7f08fc0bd150 con 0x7f0908107270 2024-04-23T18:58:21.871 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T18:58:21.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 msgr2=0x7f08f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:21.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 0x7f08f00a61e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009930 tx=0x7f08f8008040 comp rx=0 tx=0).stop 2024-04-23T18:58:21.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 msgr2=0x7f090819e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:21.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f08fc032fd0 tx=0x7f08fc004270 comp rx=0 tx=0).stop 2024-04-23T18:58:21.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 shutdown_connections 2024-04-23T18:58:21.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f08f00a3d20 0x7f08f00a61e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f09081985f0 0x7f0908198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0908107270 0x7f090819e980 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 --2- 172.21.15.33:0/1674805050 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090806a840 0x7f090819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 >> 172.21.15.33:0/1674805050 conn(0x7f0908077eb0 msgr2=0x7f0908108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:22.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 shutdown_connections 2024-04-23T18:58:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:21.392+0000 7f090e831640 1 -- 172.21.15.33:0/1674805050 wait complete. 2024-04-23T18:58:22.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-23T18:58:22.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- 172.21.15.33:0/1954144141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614101110 msgr2=0x7f26141014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:22.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 --2- 172.21.15.33:0/1954144141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614101110 0x7f26141014f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f2610009900 tx=0x7f261002a410 comp rx=0 tx=0).stop 2024-04-23T18:58:22.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- 172.21.15.33:0/1954144141 shutdown_connections 2024-04-23T18:58:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 --2- 172.21.15.33:0/1954144141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614106190 0x7f261410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 --2- 172.21.15.33:0/1954144141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2614101ac0 0x7f2614105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 --2- 172.21.15.33:0/1954144141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614101110 0x7f26141014f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- 172.21.15.33:0/1954144141 >> 172.21.15.33:0/1954144141 conn(0x7f2614077fb0 msgr2=0x7f26140783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T18:58:22.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- 172.21.15.33:0/1954144141 shutdown_connections 2024-04-23T18:58:22.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- 172.21.15.33:0/1954144141 wait complete. 2024-04-23T18:58:22.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 Processor -- start 2024-04-23T18:58:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.264+0000 7f261b62a640 1 -- start start 2024-04-23T18:58:22.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614101110 0x7f261419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 0x7f2614198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:22.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f26141110f0 con 0x7f2614106190 2024-04-23T18:58:22.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2614110f70 con 0x7f2614101110 2024-04-23T18:58:22.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2614111270 con 0x7f26141985b0 2024-04-23T18:58:22.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:22.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44730/0 (socket says 172.21.15.33:44730) 2024-04-23T18:58:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 -- 172.21.15.33:0/2118716360 learned_addr learned my addr 172.21.15.33:0/2118716360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T18:58:22.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f26193c6640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614101110 0x7f261419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:22.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2619bc7640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 0x7f2614198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 -- 172.21.15.33:0/2118716360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 msgr2=0x7f2614198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:22.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 0x7f2614198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 -- 172.21.15.33:0/2118716360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614101110 msgr2=0x7f261419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T18:58:22.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614101110 0x7f261419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T18:58:22.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 -- 172.21.15.33:0/2118716360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26141990d0 con 0x7f2614106190 2024-04-23T18:58:22.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2619bc7640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 0x7f2614198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T18:58:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f2618bc5640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260400ccc0 con 0x7f2614106190 2024-04-23T18:58:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.265+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141993c0 con 0x7f2614106190 2024-04-23T18:58:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.266+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26040054d0 con 0x7f2614106190 2024-04-23T18:58:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.266+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604010430 con 0x7f2614106190 2024-04-23T18:58:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.266+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a5610 con 0x7f2614106190 2024-04-23T18:58:22.994 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:22 smithi122 ceph-mon[32546]: pgmap v1040: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-23T18:58:23.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:22 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1674805050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:58:23.012 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:22 smithi106 ceph-mon[32716]: pgmap v1040: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-23T18:58:23.021 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:22 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1674805050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:58:23.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.267+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26140691c0 con 0x7f2614106190 2024-04-23T18:58:23.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.268+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f26040046c0 con 0x7f2614106190 2024-04-23T18:58:23.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.269+0000 7f260a7fc640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 0x7f26000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T18:58:23.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.269+0000 7f26193c6640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 0x7f26000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T18:58:23.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.269+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 8498+0+0 (secure 0 0 0) 0x7f26040d0e10 con 0x7f2614106190 2024-04-23T18:58:23.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.269+0000 7f26193c6640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 0x7f26000a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2610031440 tx=0x7f2610005b50 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T18:58:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.273+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2604053ff0 con 0x7f2614106190 2024-04-23T18:58:23.207 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:22 smithi033 ceph-mon[29274]: pgmap v1040: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-23T18:58:23.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:22 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1674805050' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T18:58:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.510+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f261410a430 con 0x7f2614106190 2024-04-23T18:58:23.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.519+0000 7f260a7fc640 1 -- 172.21.15.33:0/2118716360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f260409d4c0 con 0x7f2614106190 2024-04-23T18:58:23.253 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T18:58:23.264 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T18:58:23.272 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 26, 2024-04-23T18:58:23.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:23.290 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 15357, 2024-04-23T18:58:23.298 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-04-23T18:58:23.308 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-04-23T18:58:23.317 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T18:58:23.359 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:23.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T18:58:23.376 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-04-23T18:58:23.384 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3751231116 2024-04-23T18:58:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:23.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-23T18:58:23.497 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-04-23T18:58:23.505 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3751231116 2024-04-23T18:58:23.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T18:58:23.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:23.539 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/3751231116", 2024-04-23T18:58:23.547 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-04-23T18:26:29.463120+0000", 2024-04-23T18:58:23.556 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701547738038271, 2024-04-23T18:58:23.564 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-04-23T18:58:23.574 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-04-23T18:58:23.641 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:23.650 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24991, 2024-04-23T18:58:23.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-04-23T18:58:23.667 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-23T18:58:23.801 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-23T18:58:23.810 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:23.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-23T18:58:23.824 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:58:23.831 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:58:23.838 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:58:23.845 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T18:58:23.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T18:58:23.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:23.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:23.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T18:58:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T18:58:23.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:24.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:24.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:58:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:58:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:24.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:24.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:24.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:24.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:24.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:24.191 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:24.268 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:24.293 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:24.302 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:24.398 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:24.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:24.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:24.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:24.453 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:58:24.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:58:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:24.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:24.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:24.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:24.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:24.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:24.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:24.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:24.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:24.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:24.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:24.706 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:58:24.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:58:24.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:24.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:24.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:24.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:58:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:24.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:24.816 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:24.850 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:24.859 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:24.869 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:24.976 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:24.986 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:24.996 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:25.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:25.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:25.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:25.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:25.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:25.049 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:58:25.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:58:25.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:25.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:25.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:25.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:25.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:25.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:25.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:25.309 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:25.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:25.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:25.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:25.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:25.357 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-23T18:58:25.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-23T18:58:25.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:25.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:25.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:25.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:25.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:25.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:25.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-23T18:58:25.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:25.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:25.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:25.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:25.669 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-23T18:58:25.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-23T18:58:25.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:25.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:25.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:25.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-23T18:58:25.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:25.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:25.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:25.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-23T18:58:25.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:25.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:25.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:25.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:25.887 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-23T18:58:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-23T18:58:26.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:26.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:26.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:26.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:26.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:26.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:26.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:26.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-23T18:58:26.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:26.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:26.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:26.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:26.159 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-23T18:58:26.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-23T18:58:26.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:26.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:26.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:26.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:26.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:26.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:26.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-23T18:58:26.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:26.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:26.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:26.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:26.401 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-23T18:58:26.409 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-23T18:58:26.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:26.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:26.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:26.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-23T18:58:26.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:26.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:26.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:26.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-23T18:58:26.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:26.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:26.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:26.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:26.695 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-23T18:58:26.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-23T18:58:26.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:26.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:26.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:26.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:26.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:26.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-23T18:58:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:27.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:27.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:27.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-23T18:58:27.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-23T18:58:27.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:27.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:27.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:27.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:27.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:27.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:27.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:27.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T18:58:27.170 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:27.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:27.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:27.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-23T18:58:27.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-23T18:58:27.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:27.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:27.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:27.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:27.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:27.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:27.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:27.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-23T18:58:27.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:27.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:27.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:27.524 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:27.535 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:27.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:27.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:27.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-23T18:58:27.635 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:58:27.645 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:58:27.656 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:58:27.667 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-23T18:58:27.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-23T18:58:27.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:27.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:27.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:27.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:27.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:27.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:27.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T18:58:27.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:27.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:27.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:27.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:27.961 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-23T18:58:27.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-23T18:58:27.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:28.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-23T18:58:28.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:28.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:28.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:28.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T18:58:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:58:28.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:28.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-23T18:58:28.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-23T18:58:28.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T18:58:28.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:28.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:28.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T18:58:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T18:58:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T18:58:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:28.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T18:58:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:58:28.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:28.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:28.417 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-23T18:58:28.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-23T18:58:28.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T18:58:28.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:28.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-23T18:58:28.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T18:58:28.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-23T18:58:28.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:28.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T18:58:28.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:28.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:28.666 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-23T18:58:28.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-23T18:58:28.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:28.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:28.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:28.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T18:58:28.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:28.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:28.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T18:58:28.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T18:58:28.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:28.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:28.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-23T18:58:28.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-23T18:58:28.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:58:28.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:28.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:28.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-23T18:58:28.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-23T18:58:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-23T18:58:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-23T18:58:29.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T18:58:29.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:29.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:29.188 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-23T18:58:29.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-23T18:58:29.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:29.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:29.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:29.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-23T18:58:29.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:29.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:29.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:29.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T18:58:29.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T18:58:29.322 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:29.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:29.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:29.422 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-23T18:58:29.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-23T18:58:29.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T18:58:29.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:29.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:29.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T18:58:29.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T18:58:29.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T18:58:29.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:29.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T18:58:29.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T18:58:29.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:29.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:29.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:29.609 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:58:29.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:58:29.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:29.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:29.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:29.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:29.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:29.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:29.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:29.798 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:29.805 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:29.812 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:29.820 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:29.828 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:29.837 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:29.846 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:29.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:29.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:29.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:29.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:29.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:30.000 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:58:30.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:58:30.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:30.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:30.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:30.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:30.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:30.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:30.098 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:30.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:30.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:30.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:30.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:30.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:30.156 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:58:30.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:58:30.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:30.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:30.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:30.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:58:30.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:30.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:30.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:30.385 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:30.395 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:30.404 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:30.416 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:30.431 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:30.440 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:30.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:30.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:30.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:30.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:30.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:30.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:58:30.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:58:30.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:30.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:30.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:30.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:30.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:30.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:30.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:30.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:30.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:30.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:30.738 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:30.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:30.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-23T18:58:30.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-23T18:58:30.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:58:30.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:30.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:30.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T18:58:30.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:30.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:30.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:30.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T18:58:30.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:30.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:30.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:30.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:30.948 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T18:58:30.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T18:58:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:31.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:31.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:31.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-23T18:58:31.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:31.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:31.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:31.186 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-23T18:58:31.198 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-23T18:58:31.210 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-23T18:58:31.221 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-23T18:58:31.230 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-23T18:58:31.240 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:31.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-23T18:58:31.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:31.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:31.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:31.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:31.391 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-23T18:58:31.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-23T18:58:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:31.451 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:31.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:31.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:31.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:31.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T18:58:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:31.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:31.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:31.594 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T18:58:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T18:58:31.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:31.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:31.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T18:58:31.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:31.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:31.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:31.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T18:58:31.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:31.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:31.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:31.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:31.883 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-23T18:58:31.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-23T18:58:31.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:31.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:31.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:31.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T18:58:32.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T18:58:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:32.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:32.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T18:58:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T18:58:32.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:32.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:32.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:32.152 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-23T18:58:32.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-23T18:58:32.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T18:58:32.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:32.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:32.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:58:32.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:32.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:32.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T18:58:32.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:32.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:32.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:32.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:32.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:32.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:32.439 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:32.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-23T18:58:32.460 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:58:32.472 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:58:32.481 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:58:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-23T18:58:32.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-23T18:58:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:58:32.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:32.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T18:58:32.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:32.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T18:58:32.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:32.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:32.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:32.738 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:32.748 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-23T18:58:32.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-23T18:58:32.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:32.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-23T18:58:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:32.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:32.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T18:58:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:32.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:32.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-23T18:58:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-23T18:58:33.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:33.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:33.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:33.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-23T18:58:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:33.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:33.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:33.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T18:58:33.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:33.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:33.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:33.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:33.240 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-23T18:58:33.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-23T18:58:33.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:33.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:33.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:33.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:33.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:33.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:33.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:33.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T18:58:33.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T18:58:33.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:33.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:33.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:33.466 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-23T18:58:33.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-23T18:58:33.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:33.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:33.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:33.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T18:58:33.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:33.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:33.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:33.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T18:58:33.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:33.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:33.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:33.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:33.635 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-23T18:58:33.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T18:58:33.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:58:33.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:33.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:33.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-23T18:58:33.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:33.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:33.857 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:33.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T18:58:33.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:33.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:33.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:33.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:33.901 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-23T18:58:33.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-23T18:58:33.916 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:33.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:33.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:33.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:34.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:34.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:34.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:34.106 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-23T18:58:34.115 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-23T18:58:34.124 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-23T18:58:34.133 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:34.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T18:58:34.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:34.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:34.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:34.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:34.188 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-23T18:58:34.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-23T18:58:34.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:34.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:34.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:34.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:34.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:34.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:34.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T18:58:34.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:34.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:34.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:34.504 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-23T18:58:34.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-23T18:58:34.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:34.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:34.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:34.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:34.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:34.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:34.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:34.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T18:58:34.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:34.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:34.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:34.775 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-23T18:58:34.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-23T18:58:34.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:34.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:34.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:34.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:34.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:34.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:34.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:34.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T18:58:34.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:34.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:35.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:35.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:35.061 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-23T18:58:35.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-23T18:58:35.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:35.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:35.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:35.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T18:58:35.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:35.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:35.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:35.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T18:58:35.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:35.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:35.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-23T18:58:35.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-23T18:58:35.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:35.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:35.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:35.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T18:58:35.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:35.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:35.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:35.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-23T18:58:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:35.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:35.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:35.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:35.515 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-23T18:58:35.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-23T18:58:35.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:35.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:35.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:35.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T18:58:35.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:35.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:35.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:35.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-23T18:58:35.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:35.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:35.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:35.779 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-23T18:58:35.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-23T18:58:35.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:35.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:35.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:35.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T18:58:35.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T18:58:35.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:36.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:36.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:36.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:36.033 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-23T18:58:36.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-23T18:58:36.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:36.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:36.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:36.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T18:58:36.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:36.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:36.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:36.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-23T18:58:36.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:36.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:36.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:36.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:36.276 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-23T18:58:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-23T18:58:36.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:36.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T18:58:36.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:36.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:36.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-23T18:58:36.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:36.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:36.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:36.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:36.593 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-23T18:58:36.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-23T18:58:36.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:36.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:36.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:36.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T18:58:36.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:36.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:36.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:36.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-23T18:58:36.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:36.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:36.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:36.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-23T18:58:36.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-23T18:58:36.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:36.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:36.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:36.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T18:58:36.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:36.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:36.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:36.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-23T18:58:36.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:36.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:37.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:37.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-23T18:58:37.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-23T18:58:37.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:37.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:37.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:37.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T18:58:37.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:37.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:37.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-23T18:58:37.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:37.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-23T18:58:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-23T18:58:37.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:37.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:37.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:37.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T18:58:37.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:37.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:37.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:37.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-23T18:58:37.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:37.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:37.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:37.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:37.581 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-23T18:58:37.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-23T18:58:37.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:37.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:37.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:37.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T18:58:37.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:37.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:37.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:37.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T18:58:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:37.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:37.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:37.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:37.868 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-23T18:58:37.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-23T18:58:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:37.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:37.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:37.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T18:58:37.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:37.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-23T18:58:37.964 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:37.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:37.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:38.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:38.121 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-23T18:58:38.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-23T18:58:38.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:38.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:38.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:38.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T18:58:38.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:38.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:38.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:38.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T18:58:38.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:38.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:38.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:38.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:38.251 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-23T18:58:38.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-23T18:58:38.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:38.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:38.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:38.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T18:58:38.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:38.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:38.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:38.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-23T18:58:38.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:38.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:38.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:38.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:38.471 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-23T18:58:38.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-23T18:58:38.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:38.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:38.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:38.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T18:58:38.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:38.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:38.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:38.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-23T18:58:38.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:38.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:38.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:38.700 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-23T18:58:38.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-23T18:58:38.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:38.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:38.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:38.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T18:58:38.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:38.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:38.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:38.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-23T18:58:38.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:38.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:38.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:38.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:38.933 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-23T18:58:38.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-23T18:58:38.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:38.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:38.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:38.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:38.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:38.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:39.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:39.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T18:58:39.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:39.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:39.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:39.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:39.218 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-23T18:58:39.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-23T18:58:39.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:39.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:39.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:39.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T18:58:39.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:39.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:39.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T18:58:39.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:39.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:39.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:39.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-23T18:58:39.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T18:58:39.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:39.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:39.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:39.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-23T18:58:39.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:39.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:39.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:39.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T18:58:39.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:39.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:39.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:39.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:39.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-23T18:58:39.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-23T18:58:39.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:39.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:39.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:39.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-23T18:58:39.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:39.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:39.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:39.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T18:58:39.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:39.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:39.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:39.916 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-23T18:58:39.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-23T18:58:39.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:39.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:39.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:39.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-23T18:58:40.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:40.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:40.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:40.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-23T18:58:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:40.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:40.146 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-23T18:58:40.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-23T18:58:40.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:40.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:40.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:40.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:40.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:40.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:40.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T18:58:40.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:40.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:40.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:40.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:40.379 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-23T18:58:40.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-23T18:58:40.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:40.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:40.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:40.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T18:58:40.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:40.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:40.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:40.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-23T18:58:40.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:40.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:40.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:40.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:40.632 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-23T18:58:40.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-23T18:58:40.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:40.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:40.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:40.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T18:58:40.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:40.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:40.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:40.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T18:58:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:40.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:40.828 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-23T18:58:40.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-23T18:58:40.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:40.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:41.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T18:58:41.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:41.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:41.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-23T18:58:41.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:41.049 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:41.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:41.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:41.071 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-23T18:58:41.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-23T18:58:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:41.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:41.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:41.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:41.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:41.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:41.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T18:58:41.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:41.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:41.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:41.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:41.356 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-23T18:58:41.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-23T18:58:41.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:41.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:41.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:41.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:41.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:41.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T18:58:41.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:41.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:41.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:41.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:41.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:58:41.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:58:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:41.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:41.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:41.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:41.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:41.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:41.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:41.766 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:41.864 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:41.916 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:41.924 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:41.933 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:41.942 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:41.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:41.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:41.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:41.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:41.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:42.003 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-23T18:58:42.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-23T18:58:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:42.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:42.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:42.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:42.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:42.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:42.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:42.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T18:58:42.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:42.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:42.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:42.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:42.329 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:58:42.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:58:42.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:42.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:42.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:42.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:42.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:42.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:42.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:42.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T18:58:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:42.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:42.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:42.602 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:58:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:58:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:42.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:42.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:42.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:58:42.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:42.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:42.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:42.695 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:42.705 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:42.716 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:42.821 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:42.863 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:42.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:42.881 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:42.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:42.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:42.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:42.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:42.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:42.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:58:42.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:58:42.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:42.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:42.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:42.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:43.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:43.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:43.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:43.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:43.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:43.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:43.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:43.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:43.202 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T18:58:43.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T18:58:43.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:43.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:43.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:43.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:43.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:43.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:43.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:43.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T18:58:43.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:43.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:43.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:43.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T18:58:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T18:58:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:43.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:43.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:43.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-23T18:58:43.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:43.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:43.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:43.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T18:58:43.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:43.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:43.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:43.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:43.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-23T18:58:43.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-23T18:58:43.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:43.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:43.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:43.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:58:43.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:43.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:43.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:43.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-23T18:58:43.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:43.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:43.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:43.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:43.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-23T18:58:43.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-23T18:58:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:44.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:44.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:44.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:58:44.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:44.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:44.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:44.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T18:58:44.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:44.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:44.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:44.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:44.207 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-23T18:58:44.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-23T18:58:44.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:44.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:44.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:44.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:44.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:44.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:44.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:44.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-23T18:58:44.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:44.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:44.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:44.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:44.481 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T18:58:44.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T18:58:44.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:44.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:44.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:44.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-23T18:58:44.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:44.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:44.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:44.687 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-23T18:58:44.723 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-23T18:58:44.732 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:44.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T18:58:44.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:44.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:44.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:44.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:44.786 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-23T18:58:44.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-23T18:58:44.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:44.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:44.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:44.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-23T18:58:44.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:44.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:44.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T18:58:45.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:45.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:45.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:45.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:45.080 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-23T18:58:45.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-23T18:58:45.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:45.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:45.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:45.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T18:58:45.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:45.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:45.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:45.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T18:58:45.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:45.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:45.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:45.368 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-23T18:58:45.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-23T18:58:45.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:45.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:45.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:45.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:45.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:45.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:45.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T18:58:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:45.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:45.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:45.509 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-23T18:58:45.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-23T18:58:45.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:45.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:45.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:45.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:45.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:45.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:45.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:45.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T18:58:45.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:45.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:45.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:45.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:45.739 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-23T18:58:45.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-23T18:58:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:45.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:45.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:45.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:45.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:45.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:45.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T18:58:45.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:46.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:46.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:46.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:46.071 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-23T18:58:46.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-23T18:58:46.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:46.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:46.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:46.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:46.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:46.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:46.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T18:58:46.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:46.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:46.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:46.373 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-23T18:58:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-23T18:58:46.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:46.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:46.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:46.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:46.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:46.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:46.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T18:58:46.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:46.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:46.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:46.710 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-23T18:58:46.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-23T18:58:46.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:46.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:46.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:46.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-23T18:58:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:46.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:46.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-23T18:58:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:46.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:46.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:46.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:46.965 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-23T18:58:46.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-23T18:58:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:46.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:46.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:47.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:47.010 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:47.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:47.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:47.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T18:58:47.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:47.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:47.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:47.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-23T18:58:47.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-23T18:58:47.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:47.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:47.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:47.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:47.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:47.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T18:58:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:47.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:47.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:47.360 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-23T18:58:47.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-23T18:58:47.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:47.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:47.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T18:58:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:47.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:47.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:47.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:47.700 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-23T18:58:47.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-23T18:58:47.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:47.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:47.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:47.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:47.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:47.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:47.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T18:58:47.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:47.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:47.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:47.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:48.002 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-23T18:58:48.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-23T18:58:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:48.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:48.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:48.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:48.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:48.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:48.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:48.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T18:58:48.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:48.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:48.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:48.283 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-23T18:58:48.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-23T18:58:48.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:48.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:48.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:48.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:48.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:48.358 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:48.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:48.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T18:58:48.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:48.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:48.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:48.425 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:48.553 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:48.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:48.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:48.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-23T18:58:48.632 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:58:48.642 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:58:48.651 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:58:48.661 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:58:48.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:58:48.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:48.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:48.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:48.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:48.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:48.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:48.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:48.763 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:48.869 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:48.928 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:48.940 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:48.952 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:48.966 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:48.978 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:48.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:49.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:49.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:49.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:49.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:49.060 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:58:49.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:58:49.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:49.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:49.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:49.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:49.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:49.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:49.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:49.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:49.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:49.352 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:49.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:49.388 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:58:49.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:58:49.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:49.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:49.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:49.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:58:49.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:49.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:49.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:58:49.615 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:58:49.690 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:58:49.699 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:58:49.711 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:58:49.722 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:58:49.734 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:58:49.748 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:58:49.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:49.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:49.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:49.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:49.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:49.821 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:58:49.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:58:49.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:49.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:49.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:49.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:49.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:50.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:50.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:50.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:50.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:50.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:50.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:50.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:50.081 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-23T18:58:50.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-23T18:58:50.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:50.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:50.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:50.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-23T18:58:50.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:50.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:50.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T18:58:50.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:50.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:50.332 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-23T18:58:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-23T18:58:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:58:50.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:50.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:58:50.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-23T18:58:50.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:50.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:50.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:50.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T18:58:50.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:50.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:50.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:50.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:58:50.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:50.666 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:58:50.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-23T18:58:50.691 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:58:50.700 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:58:50.712 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:58:50.723 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T18:58:50.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T18:58:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:58:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:50.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:50.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:58:50.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:50.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:50.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:50.987 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:51.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:51.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:51.037 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T18:58:51.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T18:58:51.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:51.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:51.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:51.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:51.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:51.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:51.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:51.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:51.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:51.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:51.336 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T18:58:51.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T18:58:51.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:51.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:51.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:51.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:51.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:51.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:51.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:51.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:51.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:51.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:51.704 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-23T18:58:51.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T18:58:51.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:51.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:51.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:51.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:51.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:51.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:51.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:51.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:51.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:51.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:51.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:51.899 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T18:58:51.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T18:58:52.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:52.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:52.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:52.094 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:52.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:52.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:52.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:52.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:52.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:52.255 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T18:58:52.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T18:58:52.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:52.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:52.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:52.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:52.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:52.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:52.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:52.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:52.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:52.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:52.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:52.588 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T18:58:52.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T18:58:52.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:52.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:52.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:52.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:52.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:52.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:52.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:52.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:52.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:52.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:52.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:52.921 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T18:58:52.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T18:58:52.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:52.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:52.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:52.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:52.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:52.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:53.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:53.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:53.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:53.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:53.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:53.204 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T18:58:53.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T18:58:53.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:53.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:53.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:53.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:53.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:53.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:53.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:53.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:53.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:53.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:53.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:53.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:53.443 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T18:58:53.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T18:58:53.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:53.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:53.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:53.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:53.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:53.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:53.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:53.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:53.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:53.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:53.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:53.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:53.648 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T18:58:53.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T18:58:53.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:53.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:53.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:53.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:53.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:53.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:53.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:53.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:53.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:53.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:53.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:53.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:53.929 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T18:58:53.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T18:58:53.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:53.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:53.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:53.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:54.060 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:54.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:54.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:54.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:54.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:54.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:54.186 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T18:58:54.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T18:58:54.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:54.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:54.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:54.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:54.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:54.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:54.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:54.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:54.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:54.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:54.543 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T18:58:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T18:58:54.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:54.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:54.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:54.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:54.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:54.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:54.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:54.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:54.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:54.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:54.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T18:58:54.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T18:58:54.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:54.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:54.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:54.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T18:58:54.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:54.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:54.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:54.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:54.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:54.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:54.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:54.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:54.957 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T18:58:55.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T18:58:55.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:55.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:55.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:55.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:55.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:55.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:55.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:55.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:55.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:55.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:55.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:55.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:55.245 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-23T18:58:55.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-23T18:58:55.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:55.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:55.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:55.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:55.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:55.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:55.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:55.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:55.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:55.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:55.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:55.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:55.586 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T18:58:55.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T18:58:55.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:55.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:55.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:55.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T18:58:55.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:55.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:55.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:55.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:55.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:55.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:55.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:55.845 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T18:58:55.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T18:58:55.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:55.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:55.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:55.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:55.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:55.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:56.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:56.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:56.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:56.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:56.115 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T18:58:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T18:58:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:56.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:56.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:56.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:56.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:56.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:56.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:56.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:56.412 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T18:58:56.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T18:58:56.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:56.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:56.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:56.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:56.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:56.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:56.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:56.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:56.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:56.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:56.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:56.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:56.600 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T18:58:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T18:58:56.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:56.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:56.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:56.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:56.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:56.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:56.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:56.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:56.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:56.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:56.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:56.917 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T18:58:56.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T18:58:56.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:56.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:56.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-23T18:58:57.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:57.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:57.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:57.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:57.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:57.145 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T18:58:57.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T18:58:57.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:57.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:57.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:57.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-23T18:58:57.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:57.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:57.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:57.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:57.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:57.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:57.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:57.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:57.394 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T18:58:57.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T18:58:57.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:58:57.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:57.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:57.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:58:57.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:57.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:57.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:57.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:57.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:57.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:57.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:57.667 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T18:58:57.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T18:58:57.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:57.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:57.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:57.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:57.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:57.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:57.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:57.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:57.814 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:57.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:57.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:57.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:57.967 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T18:58:58.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T18:58:58.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:58.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:58.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:58.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:58.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:58.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:58.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:58.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:58.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:58.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:58.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:58.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:58.201 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T18:58:58.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T18:58:58.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:58.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:58.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:58.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:58.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:58.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:58.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:58.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:58.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:58.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:58.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:58.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:58.437 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T18:58:58.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T18:58:58.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:58.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:58.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:58.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:58.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:58.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:58.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:58.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:58.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:58.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:58.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:58.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:58.708 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T18:58:58.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T18:58:58.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:58.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:58.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:58.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:58:58.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:58.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:58.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:58.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:58.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:59.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:59.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:59.041 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T18:58:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T18:58:59.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:59.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:59.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:59.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:59.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:59.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:59.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:59.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:59.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:59.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:59.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:59.324 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T18:58:59.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T18:58:59.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:59.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:59.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:59.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:59.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:59.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:59.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:59.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:59.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:59.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:59.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:59.533 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T18:58:59.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T18:58:59.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:59.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:58:59.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:58:59.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:58:59.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:58:59.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:58:59.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:58:59.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:58:59.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:58:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:58:59.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:58:59.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:58:59.923 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T18:58:59.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T18:58:59.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:58:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:00.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:00.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:00.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:00.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:00.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:00.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:00.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:00.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:00.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:00.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:00.295 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T18:59:00.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T18:59:00.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:00.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:00.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:00.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T18:59:00.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:00.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:00.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:00.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:00.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:00.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T18:59:00.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T18:59:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:00.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:00.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:00.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:59:00.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:00.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:00.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:00.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:00.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:00.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:00.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:00.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:00.877 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T18:59:00.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T18:59:00.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:00.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:00.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:01.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-23T18:59:01.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:01.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:01.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:01.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:01.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:01.219 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T18:59:01.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T18:59:01.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:01.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:01.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:01.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-23T18:59:01.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:01.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:01.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:01.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:01.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:01.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:01.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:01.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:01.446 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T18:59:01.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T18:59:01.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:01.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:01.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:01.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:01.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:01.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:01.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:01.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:01.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:01.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:01.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:01.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:01.772 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T18:59:01.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T18:59:01.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:01.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:01.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:01.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T18:59:01.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:02.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:02.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:02.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:02.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:02.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:02.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:02.113 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T18:59:02.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T18:59:02.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:02.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:02.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:02.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:02.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:02.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:02.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:02.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:02.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:02.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:02.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:02.403 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T18:59:02.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T18:59:02.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:02.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:02.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:02.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:02.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:02.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:02.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:02.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:02.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:02.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:02.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:02.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:02.741 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T18:59:02.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T18:59:02.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:02.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:02.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:02.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:02.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:02.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:02.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:02.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:02.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:02.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:02.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:02.991 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T18:59:03.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T18:59:03.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:03.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:03.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:03.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T18:59:03.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:03.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:03.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:03.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:03.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:03.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:03.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:03.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:03.270 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T18:59:03.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T18:59:03.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:03.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:03.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:03.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:59:03.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:03.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:03.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:03.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:03.652 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T18:59:03.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T18:59:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:03.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T18:59:03.807 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:03.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:03.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:03.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:03.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:03.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:03.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:03.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:04.007 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-23T18:59:04.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-23T18:59:04.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:04.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:04.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:04.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:04.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:04.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:04.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:04.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:04.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:04.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:04.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:04.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:04.366 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-23T18:59:04.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-23T18:59:04.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:04.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:04.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:04.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:04.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:04.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:04.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:04.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:04.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:04.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:04.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:04.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:04.594 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-23T18:59:04.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-23T18:59:04.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:04.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:04.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:04.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:04.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:04.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:04.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-23T18:59:04.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:04.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:04.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:04.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:04.762 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-23T18:59:04.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-23T18:59:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:04.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:04.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:04.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-23T18:59:04.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:04.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:05.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:05.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:05.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:05.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:05.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:05.069 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T18:59:05.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T18:59:05.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:05.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:05.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:05.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:05.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:05.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:05.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:05.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:05.399 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:05.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:05.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:05.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:05.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:05.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:05.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:05.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-23T18:59:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-23T18:59:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-23T18:59:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-23T18:59:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T18:59:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T18:59:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T18:59:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T18:59:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:07.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:07.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:07.742 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T18:59:07.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T18:59:07.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:07.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:07.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:07.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-23T18:59:07.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:07.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:07.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:07.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:07.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:07.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:07.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:07.953 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T18:59:08.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T18:59:08.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:08.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:08.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:08.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:08.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:08.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:08.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:08.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:08.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:08.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:08.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:08.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-23T18:59:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-23T18:59:08.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:08.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:08.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:08.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-23T18:59:08.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:08.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:08.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:08.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:08.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:08.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:08.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:08.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:08.582 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T18:59:08.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T18:59:08.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:08.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:08.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:08.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-23T18:59:08.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:08.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:08.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:08.886 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-23T18:59:08.913 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:08.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:08.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:08.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:08.989 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T18:59:09.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T18:59:09.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:09.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:09.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T18:59:09.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T18:59:09.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T18:59:09.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:09.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:09.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:09.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:09.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:09.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:09.373 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-23T18:59:09.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-23T18:59:09.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:09.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:09.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:09.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:09.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:09.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:09.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:09.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:09.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:09.710 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:09.723 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:09.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:09.747 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:09.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-23T18:59:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:09.792 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:09.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-23T18:59:09.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-23T18:59:09.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:09.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:09.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:09.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:09.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:10.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:10.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:10.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-23T18:59:10.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:10.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:10.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:10.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:10.103 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:10.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:10.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:10.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:10.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:10.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:10.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:10.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:10.312 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:10.321 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:10.382 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:10.394 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:10.407 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:10.415 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:10.430 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:10.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:10.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:10.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:10.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:10.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:10.504 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:10.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:10.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:10.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:10.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:10.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:10.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:10.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:10.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:10.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:10.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:10.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:10.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:10.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:10.941 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:10.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:10.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:10.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:10.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:11.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:11.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-23T18:59:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-23T18:59:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-23T18:59:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T18:59:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-23T18:59:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-23T18:59:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-23T18:59:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T18:59:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-23T18:59:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-23T18:59:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-23T18:59:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-23T18:59:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-23T18:59:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-23T18:59:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T18:59:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T18:59:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-23T18:59:12.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-23T18:59:12.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:12.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:12.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:12.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:12.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:12.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:12.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T18:59:12.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:12.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:12.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:12.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:12.983 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T18:59:12.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T18:59:13.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:13.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:13.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T18:59:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:13.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T18:59:13.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:13.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:13.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:13.265 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-23T18:59:13.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-23T18:59:13.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:13.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:13.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-23T18:59:13.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:13.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:13.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:13.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T18:59:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:13.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:13.640 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:13.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-23T18:59:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:13.691 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:13.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:13.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:13.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:13.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:13.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:13.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:13.993 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:14.005 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:14.018 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:14.041 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:14.056 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:14.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:14.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:14.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:14.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:14.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:14.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:14.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:14.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:14.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:14.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:14.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:14.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:14.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:14.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:14.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:14.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:14.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:14.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:14.420 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:14.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:14.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:14.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:14.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:14.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:14.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:14.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:14.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:14.617 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:14.627 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:14.638 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:14.648 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:14.657 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:14.666 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:14.676 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:14.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:14.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:14.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:14.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:14.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:14.887 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:14.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:14.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:14.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:14.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:14.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:14.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:15.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:15.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:15.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:15.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:15.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:15.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:15.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:15.122 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-23T18:59:15.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-23T18:59:15.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:15.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:15.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T18:59:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:15.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:15.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:15.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:15.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:15.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:15.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:15.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:15.414 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-23T18:59:15.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-23T18:59:15.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:15.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:15.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:15.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-23T18:59:15.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:15.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:15.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:15.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:15.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:15.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:15.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:15.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:15.678 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T18:59:15.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T18:59:15.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:15.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:15.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:15.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T18:59:15.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:15.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:15.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:15.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:15.917 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:15.927 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:15.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:15.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:15.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-23T18:59:15.983 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-23T18:59:15.994 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-23T18:59:16.004 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:16.013 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-23T18:59:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-23T18:59:16.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:16.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:16.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:16.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-23T18:59:16.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:16.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:16.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:16.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T18:59:16.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:16.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:16.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:16.287 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-23T18:59:16.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-23T18:59:16.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:16.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:16.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:16.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-23T18:59:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:16.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:16.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T18:59:16.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:16.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:16.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:16.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:16.527 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-23T18:59:16.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-23T18:59:16.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:16.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:16.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:16.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:16.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:16.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:16.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-23T18:59:16.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:16.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:16.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:16.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:16.783 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T18:59:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T18:59:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-23T18:59:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-23T18:59:16.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:16.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:16.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T18:59:16.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:16.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:16.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:16.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:16.992 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:17.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:17.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:17.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:17.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:17.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:17.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:17.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:17.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:17.112 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:17.122 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:17.132 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:17.142 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:17.154 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:17.164 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:17.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:17.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:17.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:17.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:17.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:17.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:17.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:17.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:17.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:17.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:17.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:17.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:17.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:17.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:17.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:17.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:17.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:17.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:17.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:17.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:17.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:17.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:17.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:17.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:17.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:17.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:17.715 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:17.725 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:17.734 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:17.743 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:17.752 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:17.800 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:17.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:17.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:17.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:17.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:17.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:17.951 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:17.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:17.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:17.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:17.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:18.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:18.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:18.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:18.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:18.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:18.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:18.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:18.246 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-23T18:59:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-23T18:59:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:18.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:18.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:18.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:18.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:18.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-23T18:59:18.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:18.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:18.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:18.469 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-23T18:59:18.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-23T18:59:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:18.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:18.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:18.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-23T18:59:18.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:18.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:18.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:18.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-23T18:59:18.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:18.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:18.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:18.651 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T18:59:18.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T18:59:18.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:18.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:18.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:18.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-23T18:59:18.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:18.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:18.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:18.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:59:18.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:18.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:18.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:18.898 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-23T18:59:18.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-23T18:59:18.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:18.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:18.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:18.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T18:59:19.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T18:59:19.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-23T18:59:19.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:19.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T18:59:19.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:19.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:19.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-23T18:59:19.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-23T18:59:19.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:19.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:19.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:19.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:19.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:19.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-23T18:59:19.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:19.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:19.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:19.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:19.424 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-23T18:59:19.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-23T18:59:19.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:19.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:19.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:19.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-23T18:59:19.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:19.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:19.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:19.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T18:59:19.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:19.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:19.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:19.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:19.739 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:19.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:19.756 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:19.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-23T18:59:19.773 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:19.782 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:19.793 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:19.804 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:19.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:19.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:19.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:19.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:19.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:19.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:19.962 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:20.007 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:20.019 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:20.029 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:20.043 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:20.050 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:20.058 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:20.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:20.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:20.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:20.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:20.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:20.106 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:20.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:20.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:20.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:20.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:20.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:20.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:20.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:20.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:20.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:20.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:20.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:20.438 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:20.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:20.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:20.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:20.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:20.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:20.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:20.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:20.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:20.665 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:20.673 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:20.680 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:20.689 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:20.695 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:20.703 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:20.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:20.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:20.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:20.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:20.764 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:20.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:20.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:20.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:20.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:20.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:20.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:21.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:21.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:21.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:21.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:21.059 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:21.067 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:21.076 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:21.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:21.094 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:21.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-23T18:59:21.121 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:21.177 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:21.227 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:21.236 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:21.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:21.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:21.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:21.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:21.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:21.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:21.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:21.374 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:21.387 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:21.399 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:21.414 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:21.427 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:21.502 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:21.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:21.580 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:21.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:21.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:21.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:21.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:21.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:21.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:21.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:21.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:21.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:21.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:21.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:21.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:21.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:21.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:21.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:21.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:21.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:21.897 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:21.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:21.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:21.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:21.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:21.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:21.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:22.004 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:22.017 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:22.027 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:22.042 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:22.054 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:22.110 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:22.164 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:22.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:22.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:22.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:22.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:22.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:22.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:22.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:22.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:22.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:22.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:22.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:22.472 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:22.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:22.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-23T18:59:22.524 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:22.535 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:22.555 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-23T18:59:22.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-23T18:59:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:22.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:22.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:22.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-23T18:59:22.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:22.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:22.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:22.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T18:59:22.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:22.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:22.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:22.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:22.876 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-23T18:59:22.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-23T18:59:22.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:22.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:22.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:59:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-23T18:59:22.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:23.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T18:59:23.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:23.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:23.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:23.114 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:23.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:23.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:23.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:23.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:23.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:23.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:23.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:23.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:23.188 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:23.220 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:23.230 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:23.277 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:23.284 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:23.292 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:23.339 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:23.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:23.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:23.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:23.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:23.387 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:23.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:23.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:23.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:23.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:23.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:23.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:23.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:23.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:23.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:23.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:23.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:23.772 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:23.858 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:23.913 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:23.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:23.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:23.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:23.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:23.968 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:23.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:23.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:24.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:24.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:24.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:24.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:24.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:24.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:24.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:24.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:24.186 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:24.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:24.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:24.212 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:24.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-23T18:59:24.230 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:24.239 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:24.248 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:24.257 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-23T18:59:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-23T18:59:24.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:24.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:24.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-23T18:59:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:24.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:24.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:24.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-23T18:59:24.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T18:59:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:24.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:24.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:24.542 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:24.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:24.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:24.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:24.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:24.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:24.821 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:24.830 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:24.840 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:24.852 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:24.861 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:24.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:24.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:24.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:24.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:24.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:24.988 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:25.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:25.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:25.061 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:25.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:25.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:25.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:25.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:25.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:25.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:25.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:25.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:25.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:25.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:25.178 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:25.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:25.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:25.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:25.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:25.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:25.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:25.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:25.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:25.395 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:25.403 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:25.410 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:25.422 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:25.437 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:25.446 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:25.453 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:25.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:25.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:25.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:25.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:25.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:25.590 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:25.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:25.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:25.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:25.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:25.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:25.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:25.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:25.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:25.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:25.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:25.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:25.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:25.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:25.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-23T18:59:25.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-23T18:59:25.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:25.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:25.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:25.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:25.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:25.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:26.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:26.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-23T18:59:26.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T18:59:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:26.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:26.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:26.059 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-23T18:59:26.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-23T18:59:26.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:26.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:26.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:26.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-23T18:59:26.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:26.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:26.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:26.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-23T18:59:26.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:26.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:26.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-23T18:59:26.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-23T18:59:26.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:26.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:26.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-23T18:59:26.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:26.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:26.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:26.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T18:59:26.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:26.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:26.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:26.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:26.535 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-23T18:59:26.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-23T18:59:26.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:26.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:26.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:26.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:26.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:26.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:26.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:26.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-23T18:59:26.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:26.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:26.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:26.729 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-23T18:59:26.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-23T18:59:26.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:26.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-23T18:59:26.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:26.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T18:59:26.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T18:59:26.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:26.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:27.033 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:27.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-23T18:59:27.052 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:27.139 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:27.152 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:27.162 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:27.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:27.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:27.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:27.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:27.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:27.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:27.301 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:27.310 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:27.319 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:27.386 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:27.395 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:27.405 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:27.457 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:27.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:27.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:27.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:27.526 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:27.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:27.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:27.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:27.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:27.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:27.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:27.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:27.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:27.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:27.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:27.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:27.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:27.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:27.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:27.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:27.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:27.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:27.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:27.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:27.903 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:27.972 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:27.990 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:28.006 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:28.054 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:28.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:28.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:28.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:28.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:28.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:28.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:28.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:28.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:28.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:28.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:28.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:28.350 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:28.360 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:28.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:28.377 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:28.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-23T18:59:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:28.404 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:28.463 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:28.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:28.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:28.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:28.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:28.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:28.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:28.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:28.684 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:28.694 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:28.701 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:28.709 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:28.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:28.727 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:28.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:28.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:28.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:28.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:28.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:28.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:29.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:29.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:29.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:29.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:29.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:29.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:29.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:29.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:29.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:29.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:29.221 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:29.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:29.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:29.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:29.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:29.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:29.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:29.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:29.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:29.397 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:29.406 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:29.425 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:29.434 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:29.446 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:29.478 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:29.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:29.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:29.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:29.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:29.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:29.574 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:29.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:29.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:29.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:29.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:29.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:29.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:29.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:29.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:29.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:29.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:29.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:29.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:29.798 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:29.806 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:29.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:29.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:29.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-23T18:59:29.925 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:29.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:29.940 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:29.948 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:29.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:29.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:29.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:29.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:29.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:29.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:30.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:30.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:30.034 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:30.042 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:30.087 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:30.095 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:30.102 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:30.143 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:30.150 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:30.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:30.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:30.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:30.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:30.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:30.204 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:30.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:30.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:30.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:30.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:30.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:30.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:30.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:30.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:30.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:30.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:30.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:30.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:30.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:30.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:30.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:30.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:30.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:30.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:30.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:30.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:30.495 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:30.557 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:30.566 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:30.576 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:30.630 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:30.640 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:30.650 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:30.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:30.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:30.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:30.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:30.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:30.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:30.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:30.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:30.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:30.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:30.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:30.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:30.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:30.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:30.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:30.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:30.956 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:30.965 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:30.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:30.983 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T18:59:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:31.030 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:31.038 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:31.047 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-23T18:59:31.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-23T18:59:31.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:31.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:31.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:31.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:31.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:31.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:31.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:31.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T18:59:31.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:31.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:31.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:31.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:31.238 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:31.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:31.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:31.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:31.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:31.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:31.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:31.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:31.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:31.506 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:31.514 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:31.528 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:31.535 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:31.544 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:31.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:31.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:31.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:31.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:31.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:31.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:31.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:31.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:31.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:31.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:31.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:31.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:31.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:31.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:31.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:31.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:31.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:31.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:31.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:31.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:31.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:31.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:32.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:32.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:32.040 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:32.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:32.057 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:32.072 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:32.083 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:32.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:32.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:32.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:32.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:32.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:32.194 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:32.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:32.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:32.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:32.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:32.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:32.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:32.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:32.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:32.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:32.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:32.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:32.408 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-23T18:59:32.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T18:59:32.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:32.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:32.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:32.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:32.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:32.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:32.586 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-23T18:59:32.595 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-23T18:59:32.603 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-23T18:59:32.611 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:32.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-23T18:59:32.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:32.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:32.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:32.696 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:32.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:32.762 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:32.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-23T18:59:32.780 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:32.800 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:32.817 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:32.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:32.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:32.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:32.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:32.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:32.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:32.914 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:32.923 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:32.978 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:32.987 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:32.996 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:33.016 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:33.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:33.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:33.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:33.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:33.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:33.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:33.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:33.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:33.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:33.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:33.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:33.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:33.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:33.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:33.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:33.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:33.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:33.300 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:33.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:33.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:33.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:33.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:33.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:33.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:33.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:33.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:33.391 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:33.398 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:33.453 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:33.461 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:33.468 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:33.500 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:33.510 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:33.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:33.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:33.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:33.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:33.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:33.595 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:33.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:33.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:33.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:33.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:33.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:33.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:33.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:33.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:33.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:33.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:33.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:33.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:33.766 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:33.804 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:33.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:33.819 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:33.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-23T18:59:33.834 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:33.893 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:33.901 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:33.909 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:33.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:33.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:33.983 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:33.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:34.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:34.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:34.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:34.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:34.039 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:34.079 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:34.190 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:34.231 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:34.239 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:34.248 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:34.283 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:34.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:34.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:34.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:34.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:34.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:34.397 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:34.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:34.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:34.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:34.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:34.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:34.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:34.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:34.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:34.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:34.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:34.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:34.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:34.575 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:34.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:34.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:34.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:34.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:34.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:34.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:34.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:34.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:34.720 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:34.727 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:34.737 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:34.744 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:34.752 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:34.779 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:34.787 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:34.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:34.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:34.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:34.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:34.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:34.886 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:34.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:34.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:34.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:34.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:34.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:34.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:34.973 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:34.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:34.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:35.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:35.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:35.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:35.045 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:35.054 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:35.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:35.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:35.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-23T18:59:35.139 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:35.149 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:35.158 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:35.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:35.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:35.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:35.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:35.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:35.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:35.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:35.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:35.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:35.303 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:35.312 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:35.360 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:35.374 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:35.381 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:35.389 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:35.397 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:35.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:35.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:35.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:35.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:35.491 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:35.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:35.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:35.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:35.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:35.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:35.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:35.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:35.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:35.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:35.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:35.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:35.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:35.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:35.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:35.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:35.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:35.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:35.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:35.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:35.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:35.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:35.875 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:35.938 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:35.948 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:35.958 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:35.976 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:35.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:36.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:36.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:36.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:36.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:36.053 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:36.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:36.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:36.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:36.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:36.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:36.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:36.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:36.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:36.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:36.224 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:36.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:36.268 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T18:59:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T18:59:36.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:36.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:36.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T18:59:36.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:36.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:36.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:36.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:36.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:36.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:36.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:36.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:36.504 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-23T18:59:36.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-23T18:59:36.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:59:36.527 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:36.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:36.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T18:59:36.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-23T18:59:36.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:36.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:36.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-23T18:59:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T18:59:36.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:36.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:36.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:36.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:36.721 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:36.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-23T18:59:36.736 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:36.763 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:36.770 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:36.778 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-23T18:59:36.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-23T18:59:36.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:36.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:36.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:36.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:36.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:36.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:36.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:36.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T18:59:36.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:36.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:37.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:37.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T18:59:37.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T18:59:37.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:37.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:37.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:37.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:37.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:37.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:37.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:37.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:37.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:37.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:37.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:37.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:37.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:37.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:37.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:37.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:37.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:37.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:37.349 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:37.358 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:37.370 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:37.381 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:37.391 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:37.481 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:37.491 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:37.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:37.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:37.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:37.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:37.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:37.576 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:37.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:37.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:37.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:37.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:37.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:37.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:37.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:37.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:37.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:37.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:37.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:37.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:37.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:37.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:37.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:37.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:37.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:37.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:37.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:37.974 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:37.982 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:37.991 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:38.000 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:38.068 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:38.078 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:38.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:38.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:38.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:38.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:38.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:38.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:38.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:38.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:38.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:38.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:38.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:38.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:38.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:38.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:38.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:38.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:38.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:38.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:38.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:38.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:38.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-23T18:59:38.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-23T18:59:38.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:38.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:38.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:38.476 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-23T18:59:38.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:38.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:38.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:38.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-23T18:59:38.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:38.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:38.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:38.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:38.617 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T18:59:38.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T18:59:38.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:38.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:38.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:38.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T18:59:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:38.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:38.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:38.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-23T18:59:38.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:38.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:38.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:38.774 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:38.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:38.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-23T18:59:38.904 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:38.913 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:38.922 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:38.930 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-23T18:59:38.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-23T18:59:38.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:38.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:38.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:38.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:39.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:39.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:39.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:39.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:39.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:39.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:39.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:39.132 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-23T18:59:39.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-23T18:59:39.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:39.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:39.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:39.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:39.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:39.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:39.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:39.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T18:59:39.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T18:59:39.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:39.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:39.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:39.392 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:39.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:39.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:39.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:39.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:39.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:39.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:39.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:39.513 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:39.522 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:39.531 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:39.545 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:39.601 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:39.610 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:39.619 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:39.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:39.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:39.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:39.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:39.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:39.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:39.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:39.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:39.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:39.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:39.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:39.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:39.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:39.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:39.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:39.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:39.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:39.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:40.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:40.033 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:40.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:40.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:40.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:40.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:40.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:40.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:40.137 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:40.144 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:40.189 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:40.198 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:40.208 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:40.215 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:40.223 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:40.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:40.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:40.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:40.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:40.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:40.320 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:40.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:40.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:40.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:40.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:40.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:40.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:40.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:40.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:40.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:40.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:40.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:40.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:40.542 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-23T18:59:40.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-23T18:59:40.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:40.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:40.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:40.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:40.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:40.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:40.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:40.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:40.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:40.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:40.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:40.746 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T18:59:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T18:59:40.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:40.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:40.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:40.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-23T18:59:40.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:40.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:40.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:40.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:40.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:40.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:40.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:40.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:40.976 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-23T18:59:40.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-23T18:59:40.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:59:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:41.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-23T18:59:41.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:41.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:41.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:41.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:41.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:41.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:41.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:41.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:41.206 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T18:59:41.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T18:59:41.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:41.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:41.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:41.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T18:59:41.312 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:41.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:41.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:41.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T18:59:41.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:41.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:41.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:41.479 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T18:59:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T18:59:41.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-23T18:59:41.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:41.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:41.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:41.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T18:59:41.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:41.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:41.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:41.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-23T18:59:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-23T18:59:41.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:41.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:41.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:41.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-23T18:59:41.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:41.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:41.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:41.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:41.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:41.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:41.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:41.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:41.943 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T18:59:41.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T18:59:41.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:41.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:41.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-23T18:59:42.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:42.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:42.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-23T18:59:42.044 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-23T18:59:42.053 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:42.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:42.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:42.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:42.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:42.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:42.218 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T18:59:42.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T18:59:42.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:42.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:42.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:42.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T18:59:42.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T18:59:42.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T18:59:42.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:42.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:42.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:42.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:42.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:42.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:42.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:42.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:42.450 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:42.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-23T18:59:42.467 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:42.476 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:42.486 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:42.495 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:42.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:42.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:42.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:42.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:42.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:42.621 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:42.630 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:42.640 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:42.726 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:42.735 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:42.745 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:42.756 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:42.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:42.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:42.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:42.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:42.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:42.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:42.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:42.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:42.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:42.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:42.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:42.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:42.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:43.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:43.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:43.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:43.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:43.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:43.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:43.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:43.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:43.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:43.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:43.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:43.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:43.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:43.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:43.247 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:43.256 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:43.316 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:43.325 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:43.333 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:43.343 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:43.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:43.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:43.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:43.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:43.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:43.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:43.433 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:43.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:43.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:43.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:43.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:43.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:43.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:43.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:43.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:43.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:43.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:43.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-23T18:59:43.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-23T18:59:43.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:43.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:43.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:43.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T18:59:43.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:43.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:43.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:43.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:43.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:43.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:43.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:43.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:43.894 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-23T18:59:43.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T18:59:43.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:43.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:43.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:43.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:43.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:44.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:44.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:44.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:44.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:44.059 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:44.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:44.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:44.155 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-23T18:59:44.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-23T18:59:44.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:44.185 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:44.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:44.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:44.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:44.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:44.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:44.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:44.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:44.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:44.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:44.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:44.339 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:44.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:44.417 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:44.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-23T18:59:44.442 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:44.452 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:44.461 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:44.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-23T18:59:44.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-23T18:59:44.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:44.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:44.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:44.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:44.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:44.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:44.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:44.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:44.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:44.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:44.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:44.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:44.758 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T18:59:44.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T18:59:44.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:44.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:44.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:44.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:44.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:44.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:44.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:44.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:44.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:44.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:44.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:44.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:45.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:45.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:45.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:45.084 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:45.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:45.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:45.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:45.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:45.185 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:45.194 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:45.203 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:45.212 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:45.221 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:45.276 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:45.285 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:45.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:45.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:45.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:45.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:45.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:45.415 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:45.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:45.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:45.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:45.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:45.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:45.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:45.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:45.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:45.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:45.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:45.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:45.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:45.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:45.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:45.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:45.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:45.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:45.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:45.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:45.736 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:45.744 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:45.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:45.807 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:45.815 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:45.823 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:45.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:45.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:45.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:45.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:45.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:45.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:46.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:46.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:46.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:46.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:46.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:46.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:46.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:46.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:46.132 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:46.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T18:59:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T18:59:46.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:46.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:46.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:46.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:46.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:46.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:46.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:46.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:46.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:46.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:46.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:46.442 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T18:59:46.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T18:59:46.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:46.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:46.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:46.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:46.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:46.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:46.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:46.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:46.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:46.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:46.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:46.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:46.724 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:46.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-23T18:59:46.743 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:46.807 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:46.816 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:46.826 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:46.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:46.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:46.916 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:46.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:46.933 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:46.974 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:46.982 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:46.989 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:47.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:47.088 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:47.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:47.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:47.124 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:47.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:47.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:47.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:47.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:47.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:47.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:47.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:47.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:47.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:47.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:47.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:47.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:47.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:47.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:47.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:47.495 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:47.502 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:47.520 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:47.530 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:47.599 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:47.608 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:47.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:47.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:47.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:47.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:47.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:47.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:47.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:47.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:47.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:47.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:47.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:47.829 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:47.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:47.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:47.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:47.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:47.939 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:47.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:47.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:47.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-23T18:59:47.995 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:48.005 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:48.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:48.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:48.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:48.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:48.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:48.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:48.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:48.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:48.174 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:48.183 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:48.193 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:48.202 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:48.232 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:48.241 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:48.250 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:48.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:48.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:48.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:48.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:48.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:48.326 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:48.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:48.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:48.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:48.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:48.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:48.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:48.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:48.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:48.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:48.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:48.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:48.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:48.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:48.542 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:48.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:48.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:48.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:48.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:48.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:48.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:48.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:48.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:48.733 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:48.740 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:48.749 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:48.778 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:48.785 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:48.794 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:48.803 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:48.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:48.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:48.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:48.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:48.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:48.874 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:48.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:48.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:48.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:48.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:48.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:49.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:49.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:49.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:49.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:49.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:49.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:49.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:49.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:49.161 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-23T18:59:49.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-23T18:59:49.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:49.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:49.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:49.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-23T18:59:49.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:49.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:49.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:49.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T18:59:49.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:49.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:49.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:49.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:49.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:49.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:49.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-23T18:59:49.441 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:49.449 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:49.457 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:49.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:49.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:49.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:49.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:49.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:49.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:49.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:49.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:49.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:49.633 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:49.641 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:49.648 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:49.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:49.706 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:49.715 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:49.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:49.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:49.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:49.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:49.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:49.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:49.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:49.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:49.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:49.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:49.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:49.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:49.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:50.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:50.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:50.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:50.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:50.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:50.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:50.117 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:50.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:50.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:50.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:50.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:50.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:50.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:50.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:50.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:50.240 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:50.249 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:50.286 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:50.294 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:50.303 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:50.312 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:50.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:50.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:50.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:50.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:50.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:50.409 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:50.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:50.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:50.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:50.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:50.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:50.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:50.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:50.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:50.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:50.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:50.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:50.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:50.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:50.673 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-23T18:59:50.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-23T18:59:50.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:50.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:50.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:50.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:50.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:50.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:50.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:50.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:50.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:50.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:50.864 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-23T18:59:50.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-23T18:59:50.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:50.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:50.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:50.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-23T18:59:50.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:51.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:51.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:51.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:51.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:51.087 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-23T18:59:51.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-23T18:59:51.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:51.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:51.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:51.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:51.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:51.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:51.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:51.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:51.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:51.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:51.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:51.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:51.282 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-23T18:59:51.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-23T18:59:51.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:51.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:51.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:51.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:51.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:51.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:51.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:51.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:51.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:51.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:51.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:51.519 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-23T18:59:51.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-23T18:59:51.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T18:59:51.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:51.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:51.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:51.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:51.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:51.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:51.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:51.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:51.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:51.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:51.754 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-23T18:59:51.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-23T18:59:51.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:51.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:51.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:51.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:51.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:51.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:51.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:51.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:51.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:51.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:51.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:52.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:52.036 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-23T18:59:52.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-23T18:59:52.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:52.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:52.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:52.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:52.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:52.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:52.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:52.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:52.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:52.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:52.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:52.247 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-23T18:59:52.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-23T18:59:52.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:52.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:52.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:52.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:52.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:52.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:52.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:52.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:52.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:52.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:52.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:52.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:52.486 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-23T18:59:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-23T18:59:52.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T18:59:52.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:52.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:52.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:52.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T18:59:52.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-23T18:59:52.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:52.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:52.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:52.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:52.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:52.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:52.784 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-23T18:59:52.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-23T18:59:52.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:52.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:52.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:52.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:52.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:52.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:52.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:52.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:52.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:53.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:53.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:53.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:53.025 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-23T18:59:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-23T18:59:53.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:53.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:53.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:53.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:53.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:53.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:53.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:53.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:53.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:53.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:53.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:53.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:53.247 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-23T18:59:53.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-23T18:59:53.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:53.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:53.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:53.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:53.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:53.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:53.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:53.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:53.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:53.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:53.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:53.447 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:53.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:53.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:53.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:53.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-23T18:59:53.568 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:53.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:53.591 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:53.640 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-23T18:59:53.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-23T18:59:53.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:53.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:53.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:53.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:53.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:53.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:53.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:53.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T18:59:53.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:53.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:53.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:53.849 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-23T18:59:53.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-23T18:59:53.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:53.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:53.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:53.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:53.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:53.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:54.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:54.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-23T18:59:54.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:54.032 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:54.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:54.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:54.105 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:54.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:54.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:54.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:54.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:54.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:54.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:54.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:54.214 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:54.223 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:54.232 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:54.305 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:54.314 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:54.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:54.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:54.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:54.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:54.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:54.453 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:54.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:54.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:54.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:54.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:54.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:54.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:54.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:54.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:54.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:54.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:54.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:54.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:54.763 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:54.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:54.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:54.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:54.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:54.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:54.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:54.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:54.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:54.888 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:54.895 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:54.955 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:54.965 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:54.973 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:54.980 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:55.042 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:55.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:55.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:55.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:55.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:55.090 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:55.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:55.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:55.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:55.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:55.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:55.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:55.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:55.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:55.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:55.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:55.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:55.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:55.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:55.349 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:55.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:55.367 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:55.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-23T18:59:55.410 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:55.421 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:55.430 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:55.439 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:55.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:55.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:55.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:55.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:55.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:55.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:55.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:55.588 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:55.596 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:55.605 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:55.676 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:55.685 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:55.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:55.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:55.759 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:55.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:55.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:55.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:55.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:55.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:55.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:55.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:55.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:55.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:55.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:55.960 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:55.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:56.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:56.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:56.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:56.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:56.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:56.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:56.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:56.134 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:56.152 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:56.206 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:56.215 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:56.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:56.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:56.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:56.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:56.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:56.332 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:56.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:56.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:56.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:56.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:56.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:56.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:56.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:56.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:56.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:56.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:56.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:56.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:56.537 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:56.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:56.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:56.620 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:56.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-23T18:59:56.637 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:56.710 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:56.719 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:56.728 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:56.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:56.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:56.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:56.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:56.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:56.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:56.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:56.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:56.859 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:56.890 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:56.899 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:56.906 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:56.913 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:56.921 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:56.985 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:56.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:57.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:57.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:57.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:57.028 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:57.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:57.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:57.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:57.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:57.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:57.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:57.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:57.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:57.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:57.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:57.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:57.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:57.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:57.282 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:57.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:57.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:57.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:57.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:57.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:57.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:57.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:57.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:57.434 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:57.442 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:57.451 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:57.460 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:57.469 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:57.531 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:57.541 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:57.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:57.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:57.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:57.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:57.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:57.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:57.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:57.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:57.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:57.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:57.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:57.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:57.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:57.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:57.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:57.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:57.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:57.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:57.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:57.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:57.888 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:57.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-23T18:59:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:57.950 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:57.959 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:57.968 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-23T18:59:58.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-23T18:59:58.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:58.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:58.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:58.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T18:59:58.146 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:58.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:58.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:58.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:58.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:58.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:58.261 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T18:59:58.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T18:59:58.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T18:59:58.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:58.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:58.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-23T18:59:58.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:58.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:58.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:58.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:58.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:58.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:58.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:58.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:58.489 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T18:59:58.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T18:59:58.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:58.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:58.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T18:59:58.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:58.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:58.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:58.627 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:58.705 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:58.715 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:58.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:58.732 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:58.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:58.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:58.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:58.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:58.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:58.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T18:59:58.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T18:59:58.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:58.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:58.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:58.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:58.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:58.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:58.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:58.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:58.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:58.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:58.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:58.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:59.005 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T18:59:59.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T18:59:59.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T18:59:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:59.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T18:59:59.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:59.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:59.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T18:59:59.155 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T18:59:59.164 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T18:59:59.241 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T18:59:59.252 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T18:59:59.261 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T18:59:59.271 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T18:59:59.280 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T18:59:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:59.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:59.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:59.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:59.368 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T18:59:59.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T18:59:59.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:59.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:59.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T18:59:59.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T18:59:59.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:59.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:59.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T18:59:59.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:59.583 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:59.625 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T18:59:59.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:59.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T18:59:59.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-23T18:59:59.660 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T18:59:59.689 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T18:59:59.699 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T18:59:59.708 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-23T18:59:59.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-23T18:59:59.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T18:59:59.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T18:59:59.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T18:59:59.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T18:59:59.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T18:59:59.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T18:59:59.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T18:59:59.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T18:59:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T18:59:59.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T18:59:59.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T18:59:59.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T18:59:59.947 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-23T18:59:59.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-23T19:00:00.015 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:00.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:00.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:00.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:00.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:00.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:00.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:00.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T19:00:00.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:00.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:00.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:00.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:00.249 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-23T19:00:00.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-23T19:00:00.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:00.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:00.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:00.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:00.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:00.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:00.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:00.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T19:00:00.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:00.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:00.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:00.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-23T19:00:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-23T19:00:00.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:00.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:00.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:00.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:00.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:00.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:00.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T19:00:00.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:00.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:00.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:00.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:00.718 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-23T19:00:00.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-23T19:00:00.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:00.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:00.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:00.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:00.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:00.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:00.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:00.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T19:00:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:00.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:00.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:00.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:00.961 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-23T19:00:01.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-23T19:00:01.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:01.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:01.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:01.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:01.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:01.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:01.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:01.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:01.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:01.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:01.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:01.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:01.285 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-23T19:00:01.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-23T19:00:01.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:01.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:01.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:01.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:01.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:01.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:01.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:01.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:01.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:01.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:01.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:01.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:01.532 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-23T19:00:01.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-23T19:00:01.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:01.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:01.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:01.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T19:00:01.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:01.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:01.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:01.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:01.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:01.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:01.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:01.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:01.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T19:00:01.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T19:00:01.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:01.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:01.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:01.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:01.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:01.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:01.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:01.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:02.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:02.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:02.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:02.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:00:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:00:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:02.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:02.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:02.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-23T19:00:02.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-23T19:00:02.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:02.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:02.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:02.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:02.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:02.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:02.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:02.361 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-23T19:00:02.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-23T19:00:02.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:02.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:02.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:02.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-23T19:00:02.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:02.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:02.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:02.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:02.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:02.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:02.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:02.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:02.593 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-23T19:00:02.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-23T19:00:02.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:02.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:02.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:02.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:02.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:02.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:02.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:02.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:02.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:02.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:02.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:02.843 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:02.898 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-23T19:00:02.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-23T19:00:02.913 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:02.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:02.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:03.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:03.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:03.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:03.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:03.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:03.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:03.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:03.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:03.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:03.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:03.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:03.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:03.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:03.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:03.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:03.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:03.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:03.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:03.352 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:03.361 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:03.369 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:03.378 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:03.388 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:03.397 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:03.465 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:03.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:03.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:03.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:03.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:03.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:03.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:03.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:03.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:03.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:03.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:03.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:03.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:03.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:03.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:03.776 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:03.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:03.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:03.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:03.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:03.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:03.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:03.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:03.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:03.964 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:03.973 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:03.982 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:03.992 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:04.002 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:04.011 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:04.020 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:04.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:04.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:04.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:04.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:04.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:04.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:04.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:04.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:04.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:04.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:04.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:04.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:04.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:04.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:04.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:04.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:04.337 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-23T19:00:04.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-23T19:00:04.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:04.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:04.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:04.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:04.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:04.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:04.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:04.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:04.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:04.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:04.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:04.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-23T19:00:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-23T19:00:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:04.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:04.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:04.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:04.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:04.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:04.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:04.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:04.960 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:04.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:04.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:04.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:04.998 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-23T19:00:05.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-23T19:00:05.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:05.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:05.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:05.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T19:00:05.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:05.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:05.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:05.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:05.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:05.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:05.277 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:05.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:05.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:05.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:05.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-23T19:00:05.320 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:05.346 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:05.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:05.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:05.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:05.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:05.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:05.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-23T19:00:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:07.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:07.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:07.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:07.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:07.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:07.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:07.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:07.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:07.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:07.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:07.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:07.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:07.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-23T19:00:07.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-23T19:00:07.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:07.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:07.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-23T19:00:07.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:07.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:07.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:08.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T19:00:08.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:08.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:08.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:08.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:08.235 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-23T19:00:08.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-23T19:00:08.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:08.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:08.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:08.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:08.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:08.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-23T19:00:08.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:08.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:08.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:08.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-23T19:00:08.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-23T19:00:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:08.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:08.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:08.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:00:08.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:08.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:08.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:08.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T19:00:08.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:08.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:08.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:08.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:08.652 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-23T19:00:08.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T19:00:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:08.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:08.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:08.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:08.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:08.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:08.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:08.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T19:00:08.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:08.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:08.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:08.931 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:08.941 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:08.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:08.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-23T19:00:08.999 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:09.008 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:09.070 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:09.079 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-23T19:00:09.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-23T19:00:09.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:00:09.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:09.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:09.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-23T19:00:09.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:09.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:09.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:09.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:09.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:09.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:09.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:09.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:09.331 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-23T19:00:09.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-23T19:00:09.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:09.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:09.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:09.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:09.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:09.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:09.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:09.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:09.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:09.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:09.568 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:00:09.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:00:09.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:09.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:09.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:09.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:00:09.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:09.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:09.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:09.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:09.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:09.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:09.815 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:09.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:09.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:09.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:09.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:09.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:09.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:09.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:10.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:10.039 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:10.056 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:10.155 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:10.165 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:10.174 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:10.185 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:10.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:10.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:10.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:10.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:10.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:10.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:10.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:10.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:10.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:10.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:10.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:10.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:10.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:10.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:10.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:10.530 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:10.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:10.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:10.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:10.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:10.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:10.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:10.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:10.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:10.689 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:10.791 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:10.798 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:10.808 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:10.816 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:10.825 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:10.834 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:10.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:10.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:10.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:10.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:10.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:10.891 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:10.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:10.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:11.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:11.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:11.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:11.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-23T19:00:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-23T19:00:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-23T19:00:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-23T19:00:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:11.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-23T19:00:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-23T19:00:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-23T19:00:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T19:00:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:00:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 25010, 2024-04-23T19:00:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-04-23T19:00:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701547738038271, 2024-04-23T19:00:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-23T19:00:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-23T19:00:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:00:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:00:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:00:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T19:00:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:12.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:12.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:12.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:12.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:12.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:12.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:12.781 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:12.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:12.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:12.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:12.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:12.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:12.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:12.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:12.968 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:12.993 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:13.002 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:13.029 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:13.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:13.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:13.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:13.137 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:13.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:13.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:13.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:13.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:13.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:13.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:13.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:13.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:13.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:13.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:13.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:13.324 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-23T19:00:13.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-23T19:00:13.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:13.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:13.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:13.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:13.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:13.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:13.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:13.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-23T19:00:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-23T19:00:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-23T19:00:13.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:13.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-23T19:00:13.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:13.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:13.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-23T19:00:13.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:13.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:13.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-23T19:00:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-23T19:00:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:13.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:13.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:13.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-23T19:00:14.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:14.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:14.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:14.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:14.163 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-23T19:00:14.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-23T19:00:14.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:14.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:14.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:14.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:14.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:14.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-23T19:00:14.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:14.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:14.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:14.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:14.401 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-23T19:00:14.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-23T19:00:14.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:14.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-23T19:00:14.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:14.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:14.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:14.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-23T19:00:14.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:14.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:14.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:14.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:14.602 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-23T19:00:14.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-23T19:00:14.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:14.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:14.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:14.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:14.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:14.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:14.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:14.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-23T19:00:14.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:14.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-23T19:00:14.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-23T19:00:14.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:14.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:14.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:14.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:14.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:14.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:14.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:14.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T19:00:14.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:14.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:14.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:14.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:14.978 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-23T19:00:14.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-23T19:00:14.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:15.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:15.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:15.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:15.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:15.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:15.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:15.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-23T19:00:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:15.177 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:15.184 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:15.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:15.199 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-23T19:00:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:15.220 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:15.227 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-23T19:00:15.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-23T19:00:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:15.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:15.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:15.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:15.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:15.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:15.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:15.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T19:00:15.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:15.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:15.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:15.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:15.464 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-23T19:00:15.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-23T19:00:15.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:15.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:15.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:15.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-23T19:00:15.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:15.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:15.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:15.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T19:00:15.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T19:00:15.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:15.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:15.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:15.689 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-23T19:00:15.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-23T19:00:15.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:00:15.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:15.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:15.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:00:15.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T19:00:15.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T19:00:15.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:15.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T19:00:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T19:00:15.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:15.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:15.935 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-23T19:00:15.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-23T19:00:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:00:15.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:15.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-23T19:00:15.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:00:16.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-23T19:00:16.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:16.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T19:00:16.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:16.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:16.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:16.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:16.257 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-23T19:00:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-23T19:00:16.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:16.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:16.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:16.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T19:00:16.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:16.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:16.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:16.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T19:00:16.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T19:00:16.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:16.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:16.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:16.424 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-23T19:00:16.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-23T19:00:16.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:00:16.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:16.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:16.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-23T19:00:16.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-23T19:00:16.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-23T19:00:16.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:16.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-23T19:00:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T19:00:16.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:16.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:16.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:16.567 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-23T19:00:16.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-23T19:00:16.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:16.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:16.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:16.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-23T19:00:16.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:16.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:16.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:16.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T19:00:16.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T19:00:16.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:16.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:16.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:16.773 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-23T19:00:16.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-23T19:00:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:00:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:16.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:00:16.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T19:00:16.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T19:00:16.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:16.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T19:00:16.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T19:00:16.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:17.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:17.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:17.019 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:17.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:17.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:17.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:17.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:17.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:17.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:17.205 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:17.215 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:17.226 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:17.269 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:17.278 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:17.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:17.296 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:17.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:17.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:17.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:17.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:17.350 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:17.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:17.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:17.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:17.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:17.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:17.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:17.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:17.536 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:17.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:17.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:17.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:17.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:17.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:17.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:17.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:17.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:17.689 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:17.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:17.752 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:17.760 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:17.811 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:17.819 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:17.828 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:17.836 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:17.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:17.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:17.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:17.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:17.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:17.887 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:17.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:17.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:17.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:17.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:17.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:18.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:18.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:18.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:18.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:18.083 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:18.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:18.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-23T19:00:18.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-23T19:00:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:00:18.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:18.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:18.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:00:18.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:18.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:18.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:18.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T19:00:18.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:18.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:18.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:18.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:18.362 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T19:00:18.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T19:00:18.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:18.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:18.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-23T19:00:18.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:18.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:18.534 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-23T19:00:18.545 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-23T19:00:18.554 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-23T19:00:18.562 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-23T19:00:18.624 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-23T19:00:18.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:18.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-23T19:00:18.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:18.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:18.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-23T19:00:18.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-23T19:00:18.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:18.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:18.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:18.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:18.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:18.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:18.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T19:00:18.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:18.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:18.919 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:18.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:18.939 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:00:18.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:00:18.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:18.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:18.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:18.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:00:18.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:19.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:19.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:19.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T19:00:19.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:19.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:19.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:19.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:19.166 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-23T19:00:19.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-23T19:00:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:19.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:19.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:19.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:00:19.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:00:19.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:19.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:19.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T19:00:19.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T19:00:19.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:19.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:19.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:19.393 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-23T19:00:19.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-23T19:00:19.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:00:19.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:19.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:19.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:19.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:19.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:19.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:19.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T19:00:19.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:19.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:19.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:19.570 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:19.577 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:19.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-23T19:00:19.640 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:19.690 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:19.699 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:19.708 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-23T19:00:19.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-23T19:00:19.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:00:19.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:19.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:19.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T19:00:19.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:19.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:19.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:19.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T19:00:19.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:19.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:19.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:19.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:19.897 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-23T19:00:19.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-23T19:00:19.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:20.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:20.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:20.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-23T19:00:20.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:20.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:20.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:20.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T19:00:20.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:20.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:20.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:20.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:20.163 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-23T19:00:20.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-23T19:00:20.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:20.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:20.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-23T19:00:20.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:20.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:20.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:20.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T19:00:20.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:20.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:20.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:20.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:20.320 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-23T19:00:20.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-23T19:00:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:20.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:20.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:20.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:20.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:20.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:20.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T19:00:20.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T19:00:20.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:20.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:20.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:20.536 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-23T19:00:20.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-23T19:00:20.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:20.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:20.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:20.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:00:20.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:20.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:20.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:20.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T19:00:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:20.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:20.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:20.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:20.784 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-23T19:00:20.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T19:00:20.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:00:20.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:20.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:20.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-23T19:00:20.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:20.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:20.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:20.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T19:00:20.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:20.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:20.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:21.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:21.048 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-23T19:00:21.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-23T19:00:21.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:21.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:21.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:21.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:21.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:21.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:21.129 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-23T19:00:21.138 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-23T19:00:21.199 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-23T19:00:21.274 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T19:00:21.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:21.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:21.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:21.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:21.368 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-23T19:00:21.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-23T19:00:21.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:21.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:21.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:21.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:21.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:21.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:21.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T19:00:21.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:21.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:21.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:21.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:21.559 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-23T19:00:21.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-23T19:00:21.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:21.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:21.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:21.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:21.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:21.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:21.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:21.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T19:00:21.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:21.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:21.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:21.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:21.841 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-23T19:00:21.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-23T19:00:21.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:21.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:21.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:21.944 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:21.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:21.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:21.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T19:00:21.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:21.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:21.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:21.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-23T19:00:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-23T19:00:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:22.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:22.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:22.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T19:00:22.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:22.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:22.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:22.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:00:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:22.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:22.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:22.305 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-23T19:00:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-23T19:00:22.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:22.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:22.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:22.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T19:00:22.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:22.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:22.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:22.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-23T19:00:22.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:22.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:22.534 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-23T19:00:22.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-23T19:00:22.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:22.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:22.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T19:00:22.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:22.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:22.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:22.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-23T19:00:22.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:22.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:22.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:22.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:22.797 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-23T19:00:22.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-23T19:00:22.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:22.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:22.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:22.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T19:00:22.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:22.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:22.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:22.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:00:22.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:22.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:22.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:23.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-23T19:00:23.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-23T19:00:23.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:23.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:23.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T19:00:23.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:23.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:23.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:23.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-23T19:00:23.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:23.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:23.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:23.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:23.238 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-23T19:00:23.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-23T19:00:23.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:23.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:23.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:23.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T19:00:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:23.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:23.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:23.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-23T19:00:23.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:23.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:23.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:23.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:23.433 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-23T19:00:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-23T19:00:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:23.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:23.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:23.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T19:00:23.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:23.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:23.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:23.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-23T19:00:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:23.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:23.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:23.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:23.665 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-23T19:00:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-23T19:00:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:23.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:23.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T19:00:23.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:23.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:23.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:23.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-23T19:00:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:23.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:23.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:23.929 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-23T19:00:23.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-23T19:00:23.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:23.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:23.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T19:00:23.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:24.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:24.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-23T19:00:24.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:24.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:24.132 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:24.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:24.234 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-23T19:00:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-23T19:00:24.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:24.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T19:00:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:24.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:24.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:24.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-23T19:00:24.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:24.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:24.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:24.433 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-23T19:00:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-23T19:00:24.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:24.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:24.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:24.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T19:00:24.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:24.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:24.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:24.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:00:24.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:24.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:24.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:24.627 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-23T19:00:24.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-23T19:00:24.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:24.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:24.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:24.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T19:00:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:24.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:24.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:24.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-23T19:00:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:24.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:24.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:24.809 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-23T19:00:24.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-23T19:00:24.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:24.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:24.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:24.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T19:00:24.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:24.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:24.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:25.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:00:25.012 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:25.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:25.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:25.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:25.046 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-23T19:00:25.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-23T19:00:25.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:25.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:25.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:25.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T19:00:25.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:25.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:25.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:25.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-23T19:00:25.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:25.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:25.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:25.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:25.374 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-23T19:00:25.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-23T19:00:25.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:25.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:25.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:25.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T19:00:25.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:25.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:25.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:25.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-23T19:00:25.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:25.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:25.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:25.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:25.604 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-23T19:00:25.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-23T19:00:25.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:25.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:25.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:25.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T19:00:25.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:25.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:25.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:25.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-23T19:00:25.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:25.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:25.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:25.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:25.844 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-23T19:00:25.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-23T19:00:25.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:25.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:25.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:25.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:25.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:25.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:25.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T19:00:25.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:25.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:25.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:26.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:26.023 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-23T19:00:26.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-23T19:00:26.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:26.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:26.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:26.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:00:26.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:26.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:26.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:26.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T19:00:26.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:26.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:26.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:26.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:26.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-23T19:00:26.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T19:00:26.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:26.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:26.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-23T19:00:26.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:26.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:26.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:26.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T19:00:26.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:26.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:26.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:26.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:26.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-23T19:00:26.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-23T19:00:26.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:26.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:26.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:26.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-23T19:00:26.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:26.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:26.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:26.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T19:00:26.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:26.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:26.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:26.775 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-23T19:00:26.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-23T19:00:26.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:26.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:26.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:26.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-23T19:00:26.849 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:26.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-23T19:00:26.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:26.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:26.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:26.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-23T19:00:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-23T19:00:27.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:27.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:27.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:27.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:27.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:27.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:27.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:27.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T19:00:27.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:27.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:27.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:27.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:27.169 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-23T19:00:27.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-23T19:00:27.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:27.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:27.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:27.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:00:27.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:27.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:27.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:27.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-23T19:00:27.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:27.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:27.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:27.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:27.390 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-23T19:00:27.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-23T19:00:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:27.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:27.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:27.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T19:00:27.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:27.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:27.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:27.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T19:00:27.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:27.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:27.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:27.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:27.573 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-23T19:00:27.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-23T19:00:27.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:27.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:27.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:27.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:00:27.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:27.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:27.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:27.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-23T19:00:27.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:27.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:27.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:27.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:27.790 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-23T19:00:27.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-23T19:00:27.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:27.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:27.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:27.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:27.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:27.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T19:00:27.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:27.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:28.035 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-23T19:00:28.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-23T19:00:28.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:28.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:28.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:28.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:28.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:28.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T19:00:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:28.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:28.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:28.201 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:28.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:28.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:28.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:28.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:28.373 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:28.383 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:28.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:28.476 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:28.494 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:28.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:28.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:28.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:28.642 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-23T19:00:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-23T19:00:28.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:28.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:28.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:28.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:28.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T19:00:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:28.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:28.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:28.850 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:28.918 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:28.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:28.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:28.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:28.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:28.970 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T19:00:28.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:29.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:29.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:29.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:29.031 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:29.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:29.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:29.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:29.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:29.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:29.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:29.181 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:29.191 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:29.202 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:29.259 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:29.271 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:29.278 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:29.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:29.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:29.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:29.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:29.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:29.358 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:29.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:29.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:29.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:29.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:29.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:29.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:29.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:29.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:29.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:29.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:29.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:29.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:29.618 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T19:00:29.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T19:00:29.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:29.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:29.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:29.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:29.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:29.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:29.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:29.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T19:00:29.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:29.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:29.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:29.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:29.805 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T19:00:29.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T19:00:29.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:29.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:29.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-23T19:00:29.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:29.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:29.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:30.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T19:00:30.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:30.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:30.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:30.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:30.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-23T19:00:30.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-23T19:00:30.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:30.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:30.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:30.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:30.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:30.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:30.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:30.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-23T19:00:30.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:30.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:30.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:30.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:30.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-23T19:00:30.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-23T19:00:30.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:30.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:30.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:30.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:30.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:30.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:30.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:30.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T19:00:30.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:30.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:30.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:30.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:30.737 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-23T19:00:30.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-23T19:00:30.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:30.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:30.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:30.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:30.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:30.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:30.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:30.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-23T19:00:30.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:30.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:30.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:30.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T19:00:30.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T19:00:31.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:31.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:31.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:31.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-23T19:00:31.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:31.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:31.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:31.089 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-23T19:00:31.121 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-23T19:00:31.130 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:31.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T19:00:31.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:31.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:31.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:31.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:31.230 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-23T19:00:31.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-23T19:00:31.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:31.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:31.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:31.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-23T19:00:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:31.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:31.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:31.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T19:00:31.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:31.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:31.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:31.490 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-23T19:00:31.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-23T19:00:31.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:31.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:31.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:31.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T19:00:31.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:31.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:31.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:31.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T19:00:31.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:31.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:31.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:31.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:31.772 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-23T19:00:31.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-23T19:00:31.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:31.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:31.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:31.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:31.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:31.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:32.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T19:00:32.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:32.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:32.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:32.049 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-23T19:00:32.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-23T19:00:32.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:32.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:32.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:32.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:32.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:32.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:32.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T19:00:32.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:32.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:32.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:32.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:32.349 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-23T19:00:32.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-23T19:00:32.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:32.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:32.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:32.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:32.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:32.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:32.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:32.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T19:00:32.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:32.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:32.609 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-23T19:00:32.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-23T19:00:32.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:32.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:32.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:32.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:32.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:32.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T19:00:32.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:32.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:32.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:32.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:32.812 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-23T19:00:32.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-23T19:00:32.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:32.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:32.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:32.903 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:32.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:32.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:32.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:32.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T19:00:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:32.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:33.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:33.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:33.054 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-23T19:00:33.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-23T19:00:33.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:33.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:33.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-23T19:00:33.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:33.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:33.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:33.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-23T19:00:33.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:33.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:33.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:33.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:33.355 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-23T19:00:33.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-23T19:00:33.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:33.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:33.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:33.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:33.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T19:00:33.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:33.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:33.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:33.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:33.574 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-23T19:00:33.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-23T19:00:33.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:33.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:33.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:33.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:33.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:33.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:33.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:33.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T19:00:33.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:33.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:33.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:33.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:33.875 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-23T19:00:33.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-23T19:00:33.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:33.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:33.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:33.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:33.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:33.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:34.023 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:34.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T19:00:34.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:34.051 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:34.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:34.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:34.078 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-23T19:00:34.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-23T19:00:34.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:34.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:34.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:34.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:34.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:34.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:34.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:34.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T19:00:34.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:34.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:34.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:34.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:34.336 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-23T19:00:34.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-23T19:00:34.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:34.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:34.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:34.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:34.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:34.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:34.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T19:00:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:34.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:34.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:34.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:34.614 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-23T19:00:34.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-23T19:00:34.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:34.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:34.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:34.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:34.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:34.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:34.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:34.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T19:00:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:34.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:34.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:34.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:34.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:34.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:34.874 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:34.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-23T19:00:34.894 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:34.905 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:34.914 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:34.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:34.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:34.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:35.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:35.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:35.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:35.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:35.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:35.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:35.058 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:35.088 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:35.097 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:35.141 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:35.149 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:35.157 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:35.165 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:35.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:35.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:35.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:35.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:35.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:35.266 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:35.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:35.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:35.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:35.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:35.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:35.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:35.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:35.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:35.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:35.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:35.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:35.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:35.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:35.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:35.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:35.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:35.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:35.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:35.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:35.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:35.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:35.625 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:35.634 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:35.725 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:35.732 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:35.740 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:35.746 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:35.753 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:35.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:35.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:35.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:35.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:35.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:35.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:35.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:35.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:35.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:35.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:36.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:36.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:36.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:36.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:36.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:36.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:36.111 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-23T19:00:36.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-23T19:00:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:36.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:36.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-23T19:00:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:36.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:36.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:36.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T19:00:36.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:36.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:36.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:36.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:36.395 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-23T19:00:36.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-23T19:00:36.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:36.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:36.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:36.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-23T19:00:36.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:36.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:36.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:36.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T19:00:36.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:36.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:36.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:36.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:36.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-23T19:00:36.687 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:36.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:36.706 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T19:00:36.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T19:00:36.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:36.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:36.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:36.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:36.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:36.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:36.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:36.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:36.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:36.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:36.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:36.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:36.952 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T19:00:36.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T19:00:36.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:37.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:37.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:37.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:37.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:37.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:37.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:37.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:37.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:37.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:37.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:37.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:37.215 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T19:00:37.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T19:00:37.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:37.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:37.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:37.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:37.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:37.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:37.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:37.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:37.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:37.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:37.382 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-23T19:00:37.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T19:00:37.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:37.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:37.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:37.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:37.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:37.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:37.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:37.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:37.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:37.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:37.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:37.603 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T19:00:37.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T19:00:37.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:37.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:37.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:37.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:37.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:37.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:37.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T19:00:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T19:00:37.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:37.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:37.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:37.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:37.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:38.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:38.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:38.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:38.062 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:38.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:38.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:38.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:38.098 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T19:00:38.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T19:00:38.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:38.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:38.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:38.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:38.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:38.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:38.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:38.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:38.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:38.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:38.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:38.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:38.445 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T19:00:38.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T19:00:38.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:38.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:38.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:38.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:38.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:38.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:38.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:38.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:38.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:38.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:38.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:38.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:38.737 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T19:00:38.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T19:00:38.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:38.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:38.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:38.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:38.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:38.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:38.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:38.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:38.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:38.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:38.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:38.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:38.930 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T19:00:38.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T19:00:39.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:39.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:39.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:39.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:39.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:39.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:39.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:39.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:39.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:39.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:39.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:39.249 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T19:00:39.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T19:00:39.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:39.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:39.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:39.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:39.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:39.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:39.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:39.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:39.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:39.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:39.553 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T19:00:39.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T19:00:39.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:39.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:39.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:39.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:39.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:39.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:39.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:39.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:39.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:39.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:39.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:39.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:39.762 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T19:00:39.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T19:00:39.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:39.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:39.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:39.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:39.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:39.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:39.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:40.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:40.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:40.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:40.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T19:00:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T19:00:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:40.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:40.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:40.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:40.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:40.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:40.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:40.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:40.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:40.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:40.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:40.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:40.351 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T19:00:40.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T19:00:40.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:40.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:40.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:40.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:00:40.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:40.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:40.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:40.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:40.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:40.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:40.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:40.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:40.620 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T19:00:40.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T19:00:40.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:40.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:40.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:40.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:40.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:40.816 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:40.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:40.903 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-23T19:00:40.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-23T19:00:40.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:40.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:40.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:41.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:41.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:41.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:41.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:41.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:41.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:41.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:41.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:41.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T19:00:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T19:00:41.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:41.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:41.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:00:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:41.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:41.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:41.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:41.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:41.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:41.439 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T19:00:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T19:00:41.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:41.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:41.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:41.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:41.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:41.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:41.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:41.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:41.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:41.700 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T19:00:41.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T19:00:41.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:41.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:41.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:41.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:41.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:41.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:41.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:41.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:41.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:41.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:41.914 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T19:00:41.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T19:00:41.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:41.983 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:41.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:42.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:42.018 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:42.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:42.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:42.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:42.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:42.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:42.184 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T19:00:42.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T19:00:42.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:42.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:42.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:42.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:42.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:42.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:42.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:42.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:42.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:42.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:42.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:42.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:42.458 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T19:00:42.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T19:00:42.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:42.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:42.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:42.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-23T19:00:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:42.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:42.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:42.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:42.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:42.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:42.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:42.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:42.644 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T19:00:42.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T19:00:42.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:42.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:42.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:42.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-23T19:00:42.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:42.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:42.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:42.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:42.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:42.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:42.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:42.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:42.866 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T19:00:42.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T19:00:42.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:42.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:42.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:42.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:42.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:42.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:42.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:43.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:43.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:43.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:43.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:43.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:43.123 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T19:00:43.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T19:00:43.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:43.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:43.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:43.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:43.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:43.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:43.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:43.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:43.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:43.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:43.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:43.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:43.314 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T19:00:43.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T19:00:43.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:43.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:43.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:43.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:43.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:43.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:43.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:43.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:43.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:43.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:43.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:43.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:43.573 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T19:00:43.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T19:00:43.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:43.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:43.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:43.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:43.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:43.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:43.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:43.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:43.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:43.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:43.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:43.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:43.856 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T19:00:43.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T19:00:43.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:43.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:43.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:43.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:43.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:43.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:44.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:44.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:44.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:44.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:44.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:44.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:44.057 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T19:00:44.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T19:00:44.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:44.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:44.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:44.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:44.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:44.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:44.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:44.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:44.306 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:44.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:44.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:44.329 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T19:00:44.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T19:00:44.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:44.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:44.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:44.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:44.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:44.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:44.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:44.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:44.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:44.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:44.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:44.582 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T19:00:44.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T19:00:44.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:44.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:44.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:44.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:44.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:44.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:44.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:44.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:44.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:44.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:44.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:44.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:44.769 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T19:00:44.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T19:00:44.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:44.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:44.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:44.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:44.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:44.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:44.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:44.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:44.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:45.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:45.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:45.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:45.029 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T19:00:45.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T19:00:45.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:45.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:45.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:45.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:45.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:45.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:45.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:45.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:45.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:45.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:45.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:45.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:45.322 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T19:00:45.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T19:00:45.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:45.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:45.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:45.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T19:00:45.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:45.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:45.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:45.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:45.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:45.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:45.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:45.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:45.541 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T19:00:45.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T19:00:45.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:45.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:45.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:45.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:45.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:45.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:45.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:45.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:45.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:45.787 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T19:00:45.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T19:00:45.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:45.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:45.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:45.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-23T19:00:45.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:45.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:45.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:45.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:45.973 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T19:00:46.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T19:00:46.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:46.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:46.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:46.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-23T19:00:46.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:46.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:46.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:46.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:46.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:46.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:46.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:46.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:46.180 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T19:00:46.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T19:00:46.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:46.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:46.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:46.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:46.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:46.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:46.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:46.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:46.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:46.366 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T19:00:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T19:00:46.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:46.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:46.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:46.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:00:46.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:46.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:46.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:46.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:46.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:46.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:46.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:46.556 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T19:00:46.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T19:00:46.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:46.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:46.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:46.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:46.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:46.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:46.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:46.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:46.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:46.863 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T19:00:46.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T19:00:46.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:46.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:46.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:47.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:47.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:47.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:47.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:47.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:47.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:47.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:47.294 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T19:00:47.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T19:00:47.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:47.320 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:47.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:47.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:47.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:47.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:47.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:47.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:47.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:47.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:47.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:47.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.524+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 msgr2=0x7f26000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:00:47.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.524+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 0x7f26000a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2610031440 tx=0x7f2610005b50 comp rx=0 tx=0).stop 2024-04-23T19:00:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.524+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 msgr2=0x7f261419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:00:47.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f260400e9f0 tx=0x7f260400eeb0 comp rx=0 tx=0).stop 2024-04-23T19:00:47.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 shutdown_connections 2024-04-23T19:00:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f26000a3d20 0x7f26000a61e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:00:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26141985b0 0x7f2614198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:00:47.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2614106190 0x7f261419e940 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:00:47.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 --2- 172.21.15.33:0/2118716360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2614101110 0x7f261419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:00:47.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 >> 172.21.15.33:0/2118716360 conn(0x7f2614077fb0 msgr2=0x7f261406a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:00:47.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 shutdown_connections 2024-04-23T19:00:47.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T18:58:22.525+0000 7f261b62a640 1 -- 172.21.15.33:0/2118716360 wait complete. 2024-04-23T19:00:47.702 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2024-04-23T19:00:47.711 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T19:00:47.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T19:00:47.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:47.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:47.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:47.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:00:47.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:47.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:47.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:47.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:47.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:47.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:47.947 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T19:00:47.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T19:00:47.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:47.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:48.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:48.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:00:48.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:48.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:48.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:48.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:48.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:48.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:48.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:48.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:48.121 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T19:00:48.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T19:00:48.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:48.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:48.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:48.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:00:48.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:48.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:48.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:48.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:48.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:48.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:48.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:48.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:48.389 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-23T19:00:48.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-23T19:00:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:48.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:48.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:48.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:48.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:48.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:48.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:48.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:48.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:48.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:48.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:48.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:48.631 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-23T19:00:48.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-23T19:00:48.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:48.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:48.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:48.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:48.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:48.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:48.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:48.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:48.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:48.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:48.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:48.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:48.923 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-23T19:00:48.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-23T19:00:48.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:48.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:49.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:49.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:49.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:49.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:49.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:49.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-23T19:00:49.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:49.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:49.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:49.153 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-23T19:00:49.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-23T19:00:49.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:49.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:49.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:49.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-23T19:00:49.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:49.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:49.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:49.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:49.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:49.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:49.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:49.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:49.399 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T19:00:49.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T19:00:49.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:49.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:49.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:49.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:49.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:49.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:49.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:49.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:49.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:49.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:49.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:49.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:49.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:49.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:49.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:49.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:49.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:49.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:49.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:49.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:49.784 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:49.794 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:49.803 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:49.812 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:49.846 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:49.855 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:49.865 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:49.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:49.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:49.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:49.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:50.005 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:50.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:50.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:50.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:50.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:50.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:50.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:50.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:50.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:50.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:50.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:50.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:50.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:50.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:50.181 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:50.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:50.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:50.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:50.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:50.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:50.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:50.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:50.342 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:50.352 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:50.454 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:50.464 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:50.474 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:50.481 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:50.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:50.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:50.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:50.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:50.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:50.597 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:50.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:50.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:50.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:50.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:50.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:50.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:50.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-23T19:00:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-23T19:00:50.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:50.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:50.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:50.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:50.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:50.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:51.012 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-23T19:00:51.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:51.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:51.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:51.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:51.086 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-23T19:00:51.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T19:00:51.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:51.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:51.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:51.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:51.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:51.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:51.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:51.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:51.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:51.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:51.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:51.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:51.314 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:00:51.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:00:51.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:51.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:51.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T19:00:51.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:51.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:51.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:51.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:51.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:51.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:51.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:51.587 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:00:51.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:00:51.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:51.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:51.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:51.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-23T19:00:51.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:51.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:51.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:51.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:51.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:51.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:51.818 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T19:00:51.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T19:00:51.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:51.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:51.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:51.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:51.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:51.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:51.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:51.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:51.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:51.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:51.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:51.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:52.054 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-23T19:00:52.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-23T19:00:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:52.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:52.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:52.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-23T19:00:52.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:52.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:52.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:52.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:52.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:52.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:52.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:52.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:52.245 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T19:00:52.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T19:00:52.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:52.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:52.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-23T19:00:52.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:52.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:52.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:52.400 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:52.409 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-23T19:00:52.418 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:52.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:52.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:52.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:52.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:52.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:52.502 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T19:00:52.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T19:00:52.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:52.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:52.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:52.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T19:00:52.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T19:00:52.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T19:00:52.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:52.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:52.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:52.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:52.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:52.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:52.736 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-23T19:00:52.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-23T19:00:52.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:52.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:52.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:52.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:52.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:52.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:52.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:52.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:52.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:52.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:52.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:52.942 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:52.951 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:52.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:52.995 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:53.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-23T19:00:53.012 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:53.022 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:53.031 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:53.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-23T19:00:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-23T19:00:53.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:53.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:53.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:53.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:53.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:53.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:53.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:53.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-23T19:00:53.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:53.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:53.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:53.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:53.322 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:53.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:53.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:53.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:53.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:53.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:53.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:53.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:53.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:53.474 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:53.485 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:53.496 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:53.507 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:53.517 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:53.561 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:53.571 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:53.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:53.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:53.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:53.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:53.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:53.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:53.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:53.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:53.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:53.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:53.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:53.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:53.807 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:53.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:53.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:53.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:53.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:53.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:53.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:53.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:53.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:53.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:53.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:54.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:54.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:54.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:54.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:54.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:54.086 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:54.093 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:54.100 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:54.200 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:54.209 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:54.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:54.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:54.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:54.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:54.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:54.306 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:54.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:54.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:54.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:54.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:54.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:54.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:54.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:54.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:54.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:54.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:54.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:54.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:54.521 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-23T19:00:54.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-23T19:00:54.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:54.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:54.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-23T19:00:54.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:54.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:54.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:54.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T19:00:54.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:54.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:54.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:54.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:54.796 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-23T19:00:54.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-23T19:00:54.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:54.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:54.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:54.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-23T19:00:54.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:54.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:54.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:54.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T19:00:55.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:55.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:55.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:55.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:55.065 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-23T19:00:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-23T19:00:55.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:55.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:55.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-23T19:00:55.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:55.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:55.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:55.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-23T19:00:55.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:55.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:55.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:55.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:55.315 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-23T19:00:55.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-23T19:00:55.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:55.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:55.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:55.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T19:00:55.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:55.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:55.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:55.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T19:00:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:55.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:55.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:55.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:55.574 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-23T19:00:55.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-23T19:00:55.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:55.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:55.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:55.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:00:55.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:55.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:55.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:55.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T19:00:55.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:55.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:55.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:55.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:55.805 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:00:55.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:00:55.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:55.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:55.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:55.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:00:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:55.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:55.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:55.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T19:00:55.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:55.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:56.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:56.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:56.021 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-23T19:00:56.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-23T19:00:56.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:56.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:56.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:56.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-23T19:00:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:56.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:56.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:56.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T19:00:56.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:56.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:56.252 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:56.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:56.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:56.279 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:56.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-23T19:00:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:00:56.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:00:56.316 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:56.372 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:56.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:56.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:56.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:56.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:56.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:56.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:56.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:56.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:56.511 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:56.520 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:56.529 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:56.538 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:56.551 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:56.560 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:56.569 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:56.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:56.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:56.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:56.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:56.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:56.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:56.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:56.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:56.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:56.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:56.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:56.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:56.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:56.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:56.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:56.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:56.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:56.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:56.928 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:56.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:56.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:57.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:57.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:57.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:00:57.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:57.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:57.095 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:57.103 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:57.112 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:57.120 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:57.129 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:57.138 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:57.147 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:57.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:57.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:57.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:57.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:57.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:57.319 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:00:57.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:00:57.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:57.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:57.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:57.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:57.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:57.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:57.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:57.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:57.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:57.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:57.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:57.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:57.567 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-23T19:00:57.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-23T19:00:57.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:57.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:57.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:57.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T19:00:57.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:57.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:57.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:57.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:57.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:57.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:57.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:57.736 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-23T19:00:57.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-23T19:00:57.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:57.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:57.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:57.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-23T19:00:57.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:57.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:57.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:57.906 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:57.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:57.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:57.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:57.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:57.987 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:00:57.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:00:58.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:58.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:58.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:58.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:00:58.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:58.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:58.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:58.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:58.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:58.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:58.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:58.203 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:58.212 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:00:58.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:58.263 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:00:58.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-23T19:00:58.295 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-23T19:00:58.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-23T19:00:58.317 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:00:58.410 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-23T19:00:58.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-23T19:00:58.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:00:58.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:58.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:58.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-23T19:00:58.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:58.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:58.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:58.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T19:00:58.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:58.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:58.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:58.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:58.639 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-23T19:00:58.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-23T19:00:58.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:58.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:58.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:58.740 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-23T19:00:58.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:58.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:58.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:58.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T19:00:58.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:58.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:58.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:58.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-23T19:00:58.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-23T19:00:58.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:58.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:58.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:58.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:59.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:59.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:59.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-23T19:00:59.078 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:59.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:59.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:59.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:59.117 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:00:59.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:00:59.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:00:59.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:59.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:00:59.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-23T19:00:59.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-23T19:00:59.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:59.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:59.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T19:00:59.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:59.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:59.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:59.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:59.395 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:00:59.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:00:59.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:00:59.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:59.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:59.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:00:59.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:59.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:59.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:00:59.504 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:00:59.513 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:00:59.523 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:00:59.593 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:00:59.603 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:00:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:00:59.645 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:00:59.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:59.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:59.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:59.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:59.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:59.735 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:00:59.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:00:59.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:00:59.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:00:59.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:00:59.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:00:59.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:00:59.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:00:59.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:00:59.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:00:59.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:00:59.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:00:59.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:00:59.943 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:00:59.953 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:00:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:00:59.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:00.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:00.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:00.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:00.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:00.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:00.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:00.080 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:00.091 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:00.202 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:00.211 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:00.220 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:00.292 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:00.303 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:00.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:00.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:00.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:00.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:00.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:00.419 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:00.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:00.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:00.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:00.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:00.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:00.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:00.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:00.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:00.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:00.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:00.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:00.684 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-23T19:01:00.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-23T19:01:00.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:00.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:00.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:00.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:00.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:00.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:00.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:00.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-23T19:01:00.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:00.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:00.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:00.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:00.936 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-23T19:01:00.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-23T19:01:00.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:00.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:00.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:01.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-23T19:01:01.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:01.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:01.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:01.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-23T19:01:01.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:01.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:01.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T19:01:01.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T19:01:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:01.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:01.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:01.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-23T19:01:01.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:01.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:01.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:01.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T19:01:01.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:01.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:01.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:01.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:01.394 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-23T19:01:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-23T19:01:01.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:01.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:01.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:01.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:01:01.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:01:01.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-23T19:01:01.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:01.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T19:01:01.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:01.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:01.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:01.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:01.651 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-23T19:01:01.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-23T19:01:01.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:01.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:01.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:01.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:01.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:01.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:01.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:01.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-23T19:01:01.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:01.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:01.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:01.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:01.907 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-23T19:01:01.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-23T19:01:01.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:01.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:01.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:01.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-23T19:01:01.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:01.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:02.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:02.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T19:01:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:02.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:02.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:02.099 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:02.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:02.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:02.140 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:02.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-23T19:01:02.252 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:02.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:02.273 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:02.282 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:02.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:02.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:02.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:02.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:02.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:02.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:02.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:02.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:02.493 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:02.502 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:02.511 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:02.520 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:02.618 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:02.627 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:02.636 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:02.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:02.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:02.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:02.743 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:02.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:02.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:02.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:02.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:02.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:02.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:02.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:02.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:02.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:02.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:02.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:02.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:02.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:02.940 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:02.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:02.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:03.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:03.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:03.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:03.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:03.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:03.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:03.126 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:03.137 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:03.147 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:03.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:03.209 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:03.217 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:03.224 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:03.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:03.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:03.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:03.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:03.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:03.359 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:03.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:03.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:03.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:03.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:03.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:03.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:03.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:03.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:03.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:03.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:03.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:03.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:03.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:03.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:03.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-23T19:01:03.688 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:03.761 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:03.771 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:03.780 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:03.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:03.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:03.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:03.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:03.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:03.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:03.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:03.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:03.875 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:03.884 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:03.954 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:03.987 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:03.997 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:04.049 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:04.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:04.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:04.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:04.083 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:04.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:04.102 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:04.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:04.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:04.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:04.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:04.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:04.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:04.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:04.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:04.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:04.357 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:04.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:04.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:04.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:04.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:04.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:04.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:04.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:04.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:04.438 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:04.447 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:04.499 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:04.547 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:04.556 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:04.582 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:04.592 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:04.601 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:04.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:04.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:04.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:04.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:04.644 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:04.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:04.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:04.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:04.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:04.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:04.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:04.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:04.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:04.873 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:04.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:04.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:04.900 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:04.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-23T19:01:04.918 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:04.926 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:04.933 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:04.941 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-23T19:01:04.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-23T19:01:04.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:04.995 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:05.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:05.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-23T19:01:05.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:05.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:05.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:05.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T19:01:05.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:05.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:05.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:05.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:05.153 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-23T19:01:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-23T19:01:05.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:05.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:05.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:05.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:01:05.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-23T19:01:05.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:05.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:05.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T19:01:05.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:05.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:05.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:05.366 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:05.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:05.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:05.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:05.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:05.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:05.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:05.495 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:05.525 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:05.535 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-23T19:01:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-23T19:01:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-23T19:01:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-23T19:01:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-23T19:01:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T19:01:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:07.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:07.708 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:07.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:07.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:07.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:07.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:07.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:07.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:07.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:07.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:07.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:07.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:07.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:07.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:07.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:07.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-23T19:01:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-23T19:01:08.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:08.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:08.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:08.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:08.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:08.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:08.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:08.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-23T19:01:08.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T19:01:08.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:08.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:08.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:08.169 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-23T19:01:08.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-23T19:01:08.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:08.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:08.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-23T19:01:08.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:08.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:08.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:08.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-23T19:01:08.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:08.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:08.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:08.430 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-23T19:01:08.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-23T19:01:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:08.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:08.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:08.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-23T19:01:08.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:08.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:08.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:08.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T19:01:08.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:08.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:08.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:08.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:08.667 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-23T19:01:08.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-23T19:01:08.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:08.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:08.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:08.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:08.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:08.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:08.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-23T19:01:08.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:08.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:08.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:08.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:08.891 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-23T19:01:08.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-23T19:01:08.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:08.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:08.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:08.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-23T19:01:08.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:08.944 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:08.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:08.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T19:01:08.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T19:01:08.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:08.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:09.039 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:09.074 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:09.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:09.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:09.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-23T19:01:09.124 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:09.135 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:09.144 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:09.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:09.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:09.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:09.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:09.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:09.281 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:09.315 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:09.325 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:09.335 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:09.400 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:09.408 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:09.417 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:09.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:09.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:09.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:09.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:09.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:09.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:09.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:09.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:09.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:09.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:09.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:09.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:09.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:09.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:09.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:09.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:09.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:09.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:09.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:09.701 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:09.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:09.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:09.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:09.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:09.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:09.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:09.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:09.829 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:09.893 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:09.902 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:09.934 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:09.953 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:09.962 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:09.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:09.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:10.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:10.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:10.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:10.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:10.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:10.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:10.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:10.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:10.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:10.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:10.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:10.241 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:10.269 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:10.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-23T19:01:10.286 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:10.295 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:10.304 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:10.355 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:10.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:10.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:10.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:10.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:10.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:10.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:10.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:10.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:10.498 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:10.507 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:10.516 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:10.524 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:10.534 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:10.543 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:10.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:10.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:10.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:10.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:10.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:10.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:10.754 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:10.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:10.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:10.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:10.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:10.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:10.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:10.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:10.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:10.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:10.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:10.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:10.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:10.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:10.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:10.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:11.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:11.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:11.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:11.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:11.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:11.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:11.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:11.068 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:11.078 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:11.087 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:11.096 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-23T19:01:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T19:01:12.742 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:12.754 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:12.809 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-23T19:01:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-23T19:01:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:12.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:12.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:12.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:13.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:13.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:13.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T19:01:13.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:13.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:13.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:13.064 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:13.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:13.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:13.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:13.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:13.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:13.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:13.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:13.313 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:13.322 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:13.331 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:13.348 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:13.358 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:13.366 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:13.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:13.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:13.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:13.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:13.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:13.515 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:13.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:13.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:13.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:13.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:13.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:13.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:13.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:13.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:13.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:13.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:13.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:13.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:13.905 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:13.919 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:13.926 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:13.933 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:13.940 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:13.947 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:13.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:13.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:14.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:14.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:14.127 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:14.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:14.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:14.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:14.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:14.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:14.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:14.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:14.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:14.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:14.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:14.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:14.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:14.307 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-23T19:01:14.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T19:01:14.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:14.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:14.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:14.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:14.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:14.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:14.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:14.450 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-23T19:01:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-23T19:01:14.468 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-23T19:01:14.476 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:14.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-23T19:01:14.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:14.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:14.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:14.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:14.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:14.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:14.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-23T19:01:14.704 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:14.713 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:14.722 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:14.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:14.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:14.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:14.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:14.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:14.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:14.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:14.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:14.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:14.862 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:14.897 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:14.906 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:14.915 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:14.978 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:14.987 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:14.997 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:15.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:15.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:15.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:15.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:15.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:15.052 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:15.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:15.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:15.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:15.087 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:15.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:15.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:15.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:15.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:15.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:15.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:15.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:15.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:15.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:15.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:15.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:15.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:15.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:15.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:15.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:15.480 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:15.489 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:15.553 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:15.562 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:15.571 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:15.580 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:15.589 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:15.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:15.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:15.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:15.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:15.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:15.639 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:15.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:15.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:15.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:15.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:15.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:15.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:15.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:15.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:15.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:15.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:15.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:15.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:15.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:15.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:15.924 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-23T19:01:15.939 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:15.946 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:15.954 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:16.008 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:16.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:16.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:16.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:16.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:16.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:16.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:16.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:16.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:16.183 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:16.190 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:16.198 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:16.205 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:16.212 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:16.221 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:16.230 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:16.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:16.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:16.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:16.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:16.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:16.392 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:16.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:16.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:16.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:16.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:16.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:16.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:16.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:16.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:16.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:16.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:16.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:16.762 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:16.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:16.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:16.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:16.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:16.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:16.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:16.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:16.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:16.922 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:16.937 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:16.954 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:16.973 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:16.982 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:16.993 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:17.002 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:17.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:17.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:17.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:17.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:17.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:17.254 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:17.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:17.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:17.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:17.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:17.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:17.319 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:17.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:17.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:17.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:17.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:17.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:17.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:17.488 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:17.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:17.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:17.587 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:17.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-23T19:01:17.604 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:17.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:17.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:17.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:17.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:17.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:17.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:17.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:17.830 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:17.898 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:17.905 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:17.914 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:17.926 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:17.935 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:17.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:17.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:17.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:17.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:17.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:18.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:18.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:18.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:18.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:18.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:18.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:18.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:18.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:18.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:18.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:18.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:18.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:18.254 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:18.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:18.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:18.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:18.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:18.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:18.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:18.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:18.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:18.417 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:18.442 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:18.451 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:18.461 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:18.471 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:18.481 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:18.491 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:18.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:18.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:18.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:18.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:18.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:18.549 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:18.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:18.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:18.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:18.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:18.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:18.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:18.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:18.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:18.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:18.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:18.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:18.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:18.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:01:18.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:01:18.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:01:18.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:18.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:18.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:01:18.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:19.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:19.084 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:19.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:19.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:19.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:19.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:19.195 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-23T19:01:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-23T19:01:19.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:01:19.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:19.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:19.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T19:01:19.262 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-23T19:01:19.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:19.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:19.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-23T19:01:19.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T19:01:19.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:19.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:19.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:19.527 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:19.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:19.546 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:19.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-23T19:01:19.570 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:19.580 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:19.589 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:19.600 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-23T19:01:19.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-23T19:01:19.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:19.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:19.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:19.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:19.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:19.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:19.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:19.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T19:01:19.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:19.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:19.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:19.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:19.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T19:01:19.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T19:01:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:19.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:19.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:19.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:19.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:19.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:20.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:20.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:20.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:20.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:20.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:20.087 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:20.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:20.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:20.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:20.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:20.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:20.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:20.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:20.161 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:20.169 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:20.227 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:20.266 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:20.273 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:20.330 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:20.339 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:20.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:20.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:20.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:20.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:20.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:20.390 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:20.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:20.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:20.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:20.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:20.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:20.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:20.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:20.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:20.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:20.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:20.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:20.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:20.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:20.688 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:20.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:20.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:20.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:20.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:20.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:20.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:20.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:20.773 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:20.833 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:20.896 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:20.951 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:20.960 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:20.972 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:20.982 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:20.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:21.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:21.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:21.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:21.031 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:21.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:21.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:21.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:21.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:21.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:21.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:21.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:21.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:21.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:21.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:21.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:21.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:21.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-23T19:01:21.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-23T19:01:21.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:21.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:21.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:21.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-23T19:01:21.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:21.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:21.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-23T19:01:21.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:21.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:21.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:21.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:21.523 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:01:21.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:01:21.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:01:21.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:21.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:21.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:01:21.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:21.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:21.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:21.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-23T19:01:21.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:21.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:21.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:21.756 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:21.765 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:21.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:21.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:21.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-23T19:01:21.828 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:21.838 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:21.848 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:21.861 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-23T19:01:21.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-23T19:01:21.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:21.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:21.897 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:21.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:21.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:21.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:21.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:22.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:22.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:22.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:22.087 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:22.095 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-23T19:01:22.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-23T19:01:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:22.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:22.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:22.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:22.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:22.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:22.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:22.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T19:01:22.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T19:01:22.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:22.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:22.344 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:22.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:22.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:22.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:22.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:22.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:22.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:22.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:22.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:22.423 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:22.433 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:22.444 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:22.453 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:22.463 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:22.527 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:22.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:22.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:22.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:22.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:22.678 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:22.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:22.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:22.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:22.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:22.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:22.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:22.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:22.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:22.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:22.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:22.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:22.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:22.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:22.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:22.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:22.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:22.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:22.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:22.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:22.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:23.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:23.016 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:23.025 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:23.034 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:23.047 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:23.110 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:23.181 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:23.190 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:23.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:23.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:23.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:23.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:23.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:23.304 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:23.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:23.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:23.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:23.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:23.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:23.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:23.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:23.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:23.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:23.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:23.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:23.600 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-23T19:01:23.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-23T19:01:23.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:23.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:23.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:23.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:23.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:23.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:23.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:23.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:23.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:23.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:23.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T19:01:23.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T19:01:23.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:23.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:23.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-23T19:01:23.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:23.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:23.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:23.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:23.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:24.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-23T19:01:24.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-23T19:01:24.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:01:24.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:24.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:24.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-23T19:01:24.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:24.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:24.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:24.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:24.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:24.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:24.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:24.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:01:24.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:01:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:24.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T19:01:24.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:24.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:24.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T19:01:24.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:24.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:24.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:24.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:24.554 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:01:24.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:01:24.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:24.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:24.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-23T19:01:24.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:24.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:24.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:24.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T19:01:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:24.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:24.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:24.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:24.798 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-23T19:01:24.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-23T19:01:24.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:24.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:24.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-23T19:01:24.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:24.865 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:24.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:24.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:24.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:25.020 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:25.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:25.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:25.096 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T19:01:25.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T19:01:25.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:25.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:25.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:25.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-23T19:01:25.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:25.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:25.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:25.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-23T19:01:25.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-23T19:01:25.209 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:25.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:25.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:25.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:25.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:25.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:25.428 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T19:01:25.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T19:01:25.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:25.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:25.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:25.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T19:01:25.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T19:01:25.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T19:01:25.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:25.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:25.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:25.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:25.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:25.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:25.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:25.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:25.748 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:25.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-23T19:01:25.768 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:25.779 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:25.790 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:25.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:25.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:25.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:25.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:25.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:25.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:25.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:25.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:25.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:25.944 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:26.017 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:26.027 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:26.086 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:26.096 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:26.108 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:26.117 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:26.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:26.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:26.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:26.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:26.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:26.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:26.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:26.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:26.199 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:26.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:26.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:26.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:26.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:26.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:26.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:26.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:26.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:26.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:26.475 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:26.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:26.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:26.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:26.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:26.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:26.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:26.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:26.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:26.683 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:26.691 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:26.720 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:26.729 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:26.739 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:26.748 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:26.757 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:26.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:26.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:26.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:26.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:26.826 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:26.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:27.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:27.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:27.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:27.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:27.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:27.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:27.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:27.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:27.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:27.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-23T19:01:27.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-23T19:01:27.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:27.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:27.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:27.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:01:27.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:27.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:27.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:27.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:27.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:27.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:27.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:27.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:27.547 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-23T19:01:27.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T19:01:27.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:27.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:27.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:27.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:27.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:27.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:27.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:27.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:27.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:27.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:27.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:27.880 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-23T19:01:27.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-23T19:01:27.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:27.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:27.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:27.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:27.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:27.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:27.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:28.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:28.059 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:28.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:28.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:28.198 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:28.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-23T19:01:28.219 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:28.233 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:28.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-23T19:01:28.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-23T19:01:28.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:28.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:28.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:28.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:28.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:28.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:28.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:28.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:28.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:28.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:28.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:28.537 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T19:01:28.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T19:01:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:28.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:28.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:28.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:28.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:28.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:28.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:28.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:28.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:28.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:28.817 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:28.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:28.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:28.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:28.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:28.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:28.959 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:28.967 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:28.981 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:28.994 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:29.009 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:29.020 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:29.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:29.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:29.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:29.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:29.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:29.241 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:29.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:29.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:29.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:29.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:29.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:29.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:29.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:29.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:29.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:29.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:29.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:29.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:29.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:29.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:29.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:29.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:29.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:29.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:29.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:29.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:29.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:29.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:29.735 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:29.747 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:29.759 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:29.771 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:29.785 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:29.846 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:29.912 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:29.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:29.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:30.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:30.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:30.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:30.040 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:30.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:30.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:30.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:30.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:30.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:30.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:30.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:30.130 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:30.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:30.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:30.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:30.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:30.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:30.357 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:01:30.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:01:30.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:30.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:30.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:30.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:30.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:30.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:30.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:30.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:30.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:30.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:30.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:30.599 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:01:30.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:01:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:30.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:30.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:30.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:30.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:30.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:30.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:30.739 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:30.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:30.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:30.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:30.787 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:30.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:30.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:30.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:30.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-23T19:01:30.928 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:30.982 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:30.991 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:31.000 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:31.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:31.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:31.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:31.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:31.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:31.060 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:31.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:31.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:31.089 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:31.099 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:31.109 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:31.199 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:31.208 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:31.218 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:31.267 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:31.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:31.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:31.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:31.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:31.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:31.329 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:31.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:31.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:31.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:31.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:31.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:31.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:31.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:31.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:31.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:31.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:31.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:31.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:31.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:31.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:31.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:31.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:31.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:31.699 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:31.707 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:31.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:31.853 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:31.862 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:31.872 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:31.927 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:31.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:31.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:31.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:31.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:31.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:31.981 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:31.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:31.999 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:32.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:32.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:32.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:32.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:32.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:32.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:32.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:32.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:32.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:32.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:32.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:32.272 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:32.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:32.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:32.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-23T19:01:32.308 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:32.324 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:32.333 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:32.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:32.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:32.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:32.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:32.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:32.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:32.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:32.624 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:32.636 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:32.647 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:32.660 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:32.672 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:32.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:32.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:32.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:32.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:32.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:32.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:32.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:32.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:32.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:32.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:32.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:32.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:32.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:32.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:32.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:32.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:32.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:33.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:33.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:33.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:33.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:33.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:33.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:33.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:33.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:33.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:33.299 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:33.311 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:33.322 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:33.336 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:33.347 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:33.358 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:33.369 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:33.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:33.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:33.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:33.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:33.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:33.614 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:33.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:33.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:33.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:33.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:33.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:33.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:33.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:33.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:33.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:33.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:33.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:33.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:33.874 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-23T19:01:33.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-23T19:01:33.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:33.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:34.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:34.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-23T19:01:34.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:34.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:34.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:34.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T19:01:34.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:34.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:34.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:34.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:34.167 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:34.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:34.188 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:34.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-23T19:01:34.268 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:34.278 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:34.289 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:34.349 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:34.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:34.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:34.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:34.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:34.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:34.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:34.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:34.501 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:34.511 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:34.517 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:34.526 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:34.590 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:34.598 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:34.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:34.612 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:34.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:34.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:34.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:34.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:34.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:34.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:34.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:34.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:34.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:34.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:34.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:34.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:34.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:34.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:34.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:34.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:34.985 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:34.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:35.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:35.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:35.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:35.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:35.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:35.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:35.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:35.125 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:35.137 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:35.147 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:35.157 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:35.267 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:35.296 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:35.306 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:35.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:35.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:35.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:35.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:35.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:35.446 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:35.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:35.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:35.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:35.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:35.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:35.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:35.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:35.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:35.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:35.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:35.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:35.690 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-23T19:01:35.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-23T19:01:35.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:35.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:35.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:35.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:35.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:35.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:35.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:35.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:35.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:35.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:35.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:35.972 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-23T19:01:36.032 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-23T19:01:36.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:36.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:36.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:36.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-23T19:01:36.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:36.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:36.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:36.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:36.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:36.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:36.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:36.223 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-23T19:01:36.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-23T19:01:36.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:36.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:36.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:36.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:36.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:36.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:36.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:36.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:36.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:36.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:36.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-23T19:01:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-23T19:01:36.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:36.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:36.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:36.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:36.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:36.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:36.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:36.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:36.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:36.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:36.928 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-23T19:01:36.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-23T19:01:36.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:01:36.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:36.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:36.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:37.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:37.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:37.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:37.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:37.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:37.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:37.219 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-23T19:01:37.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-23T19:01:37.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:37.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:37.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:37.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:37.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:37.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:37.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:37.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:37.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:37.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:37.457 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-23T19:01:37.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-23T19:01:37.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:37.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:37.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:37.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:37.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:37.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:37.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:37.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:37.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:37.601 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:37.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:37.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:37.706 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-23T19:01:37.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-23T19:01:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:37.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:37.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:37.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:37.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:37.875 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:37.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:37.902 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:37.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:37.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:37.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:37.936 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-23T19:01:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-23T19:01:38.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:38.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:38.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:38.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:38.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:01:38.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-23T19:01:38.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:38.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:38.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:38.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:38.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:38.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:38.196 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-23T19:01:38.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-23T19:01:38.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:38.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:38.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:38.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:38.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:38.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:38.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:38.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:38.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:38.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:38.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:38.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:38.460 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-23T19:01:38.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-23T19:01:38.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:38.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:38.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:38.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:38.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:38.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:38.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:38.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:38.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:38.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:38.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:38.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:38.667 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-23T19:01:38.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-23T19:01:38.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:38.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:38.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:38.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:38.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:38.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:38.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:38.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:38.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:38.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:38.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:38.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:38.901 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:38.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:38.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:38.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-23T19:01:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:39.037 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:39.057 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-23T19:01:39.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-23T19:01:39.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:39.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:39.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:39.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:39.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:39.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:39.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:39.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T19:01:39.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:39.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:39.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:39.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:39.295 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-23T19:01:39.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-23T19:01:39.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:39.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:39.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:39.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:39.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:39.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:39.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:39.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-23T19:01:39.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:39.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:39.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:39.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:39.530 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:39.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:39.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:39.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:39.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:39.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:39.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:39.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:39.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:39.683 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:39.691 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:39.708 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:39.771 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:39.780 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:39.789 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:39.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:39.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:39.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:39.880 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:39.982 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:39.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:40.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:40.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:40.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:40.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:40.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:40.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:40.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:40.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:40.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:40.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:40.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:40.184 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:40.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:40.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:40.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:40.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:40.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:40.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:40.322 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:40.331 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:40.338 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:40.423 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:40.432 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:40.440 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:40.505 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:40.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:40.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:40.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:40.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:40.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:40.587 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:40.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:40.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:40.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:40.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:40.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:40.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:40.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:40.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:40.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:40.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:40.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:40.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:40.779 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:40.786 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:40.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:40.865 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:40.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-23T19:01:40.880 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:40.887 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:40.894 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:40.902 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:40.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:40.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:40.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:40.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:40.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:41.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:41.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:41.088 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:41.141 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:41.151 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:41.159 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:41.169 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:41.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:41.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:41.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:41.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:41.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:41.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:41.288 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:41.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:41.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:41.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:41.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:41.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:41.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:41.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:41.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:41.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:41.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:41.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:41.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:41.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:41.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:41.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:41.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:41.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:41.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:41.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:41.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:41.763 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:41.824 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:41.833 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:41.852 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:41.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:41.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:41.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:41.990 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:42.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:42.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:42.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:42.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:42.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:42.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:42.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:42.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:42.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:42.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:42.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:42.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:42.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:42.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:42.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:42.297 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:42.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-23T19:01:42.323 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:42.393 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:42.402 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:42.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:42.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:42.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:42.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:42.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:42.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:42.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:42.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:42.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:42.528 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:42.537 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:42.546 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:42.595 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:42.604 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:42.613 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:42.639 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:42.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:42.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:42.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:42.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:42.731 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:42.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:42.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:42.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:42.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:42.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:42.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:42.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:42.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:42.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:42.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:42.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:42.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:42.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:42.913 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:42.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:42.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:42.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:42.995 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:43.004 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:43.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:43.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:43.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:43.040 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:43.047 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:43.097 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:43.111 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:43.119 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:43.126 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:43.184 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:43.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:43.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:43.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:43.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:43.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:43.278 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:43.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:43.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:43.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:43.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:43.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:43.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:43.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:43.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:43.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:43.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:43.592 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:43.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:43.612 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:43.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-23T19:01:43.630 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:43.638 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:43.647 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:43.656 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-23T19:01:43.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-23T19:01:43.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:43.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:43.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:43.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T19:01:43.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:43.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:43.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:43.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:43.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:43.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:43.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:43.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:01:43.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:01:43.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:01:43.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:43.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:43.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-23T19:01:44.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:44.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:44.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:44.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:44.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:44.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:44.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:44.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:44.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:44.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:44.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:44.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:44.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:44.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:44.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:44.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:44.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:44.313 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:44.322 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:44.330 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:44.359 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:44.368 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:44.376 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:44.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:44.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:44.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:44.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:44.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:44.473 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:44.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:44.490 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:44.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:44.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:44.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:44.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:44.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:44.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:44.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:44.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:44.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:44.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:44.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:44.714 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:44.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:44.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:44.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:44.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:44.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:44.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:44.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:44.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:44.837 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:44.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:44.893 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:44.901 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:44.909 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:44.916 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:44.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:45.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:45.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:45.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:45.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:45.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:45.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:45.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:45.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:45.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:45.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:45.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:45.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:45.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:45.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:45.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:45.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:45.257 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:45.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-23T19:01:45.275 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:45.284 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:45.293 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:45.346 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-23T19:01:45.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-23T19:01:45.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:45.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:45.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:45.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:45.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:45.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:45.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:45.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T19:01:45.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:45.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:45.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:45.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:45.556 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-23T19:01:45.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-23T19:01:45.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:45.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:45.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:45.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:45.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:45.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:45.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:45.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T19:01:45.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:45.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:45.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:45.790 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-23T19:01:45.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-23T19:01:45.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:45.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:45.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:45.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:45.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T19:01:45.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:45.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:46.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:46.055 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-23T19:01:46.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-23T19:01:46.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:46.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:46.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:46.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:46.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:46.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:46.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T19:01:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:46.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:46.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:46.310 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-23T19:01:46.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-23T19:01:46.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:46.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:46.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:46.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:46.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:46.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T19:01:46.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:46.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:46.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:46.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:46.526 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-23T19:01:46.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-23T19:01:46.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:46.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:46.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:46.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:46.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:46.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:46.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:46.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:46.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:46.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:46.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-23T19:01:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-23T19:01:46.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:46.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:46.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:46.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:46.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:46.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:46.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:46.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:46.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:46.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:46.952 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-23T19:01:46.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-23T19:01:46.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:46.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:47.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T19:01:47.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:47.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:47.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:47.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:47.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:47.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:47.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:47.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:47.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T19:01:47.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T19:01:47.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:47.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:47.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:47.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:47.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:47.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:47.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:47.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:47.390 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:01:47.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:01:47.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:47.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-23T19:01:47.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-23T19:01:47.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:47.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:47.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:47.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:47.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:47.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:47.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:47.596 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-23T19:01:47.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-23T19:01:47.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:47.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:47.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:47.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-23T19:01:47.678 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:47.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:47.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:47.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:47.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:47.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:47.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:47.794 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-23T19:01:47.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-23T19:01:47.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:47.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:47.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:47.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:47.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:47.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:47.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:47.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:47.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:47.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:47.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:47.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:47.980 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-23T19:01:47.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-23T19:01:48.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:48.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:48.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:48.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:48.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:48.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:48.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:48.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:48.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:48.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:48.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:48.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:48.273 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:48.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:48.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:48.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:48.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:48.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:48.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:48.462 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:48.471 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:48.480 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:48.553 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:48.563 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:48.570 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:48.578 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:48.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:48.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:48.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:48.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:48.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:48.622 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:48.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:48.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:48.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:48.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:48.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:48.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:48.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:48.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:48.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:48.862 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:48.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:48.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:48.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:48.890 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:48.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:48.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:48.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:48.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:48.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:48.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:48.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:49.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:49.056 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:49.064 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:49.125 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:49.133 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:49.140 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:49.149 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:49.158 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:49.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:49.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:49.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:49.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:49.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:49.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:49.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:49.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:49.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:49.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:49.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:49.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:49.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:49.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:49.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:49.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:49.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:49.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:49.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:49.459 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-23T19:01:49.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-23T19:01:49.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:49.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:49.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:49.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:49.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:49.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:49.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:49.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:49.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:49.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:49.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:49.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:49.704 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-23T19:01:49.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-23T19:01:49.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:49.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:49.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:49.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:49.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:49.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:49.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:49.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:49.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:49.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:49.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:49.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-23T19:01:49.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-23T19:01:49.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:49.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:49.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:50.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T19:01:50.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:50.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:50.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:50.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:50.083 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:50.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:50.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:50.157 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:50.166 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:50.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:50.216 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:50.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-23T19:01:50.236 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:50.245 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:50.253 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:50.263 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:50.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:50.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:50.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:50.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:50.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:50.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:50.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:50.406 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:50.415 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:50.424 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:50.478 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:50.486 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:50.494 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:50.501 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:50.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:50.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:50.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:50.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:50.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:50.544 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:50.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:50.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:50.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:50.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:50.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:50.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:50.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:50.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:50.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:50.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:50.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:50.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:50.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:50.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:50.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:50.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:50.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:50.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:50.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:50.945 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:50.953 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:50.995 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:51.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:51.014 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:51.023 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:51.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:51.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:51.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:51.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:51.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:51.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:51.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:51.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:51.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:51.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:51.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:51.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:51.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:51.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:51.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:51.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:51.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:51.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:51.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:51.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-23T19:01:51.395 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:51.405 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:51.466 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:51.474 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:51.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:51.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:51.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:51.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:51.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:51.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:51.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:51.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:51.589 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:51.598 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:51.606 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:51.617 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:51.668 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:51.683 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:51.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:51.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:51.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:51.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:51.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:51.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:51.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:51.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:51.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:51.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:51.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:51.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:51.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:51.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:51.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:51.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:51.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:51.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:52.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:52.009 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:52.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:52.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:52.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:52.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:52.084 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:52.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:52.098 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:52.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:52.113 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:52.120 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:52.127 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:52.192 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:52.200 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:52.221 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:52.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:52.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:52.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:52.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:52.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:52.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:52.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:52.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:52.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:52.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:52.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:52.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:52.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:52.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:52.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:52.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:52.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:52.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:52.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:52.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-23T19:01:52.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-23T19:01:52.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:52.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:52.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:52.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-23T19:01:52.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:52.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:52.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:52.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T19:01:52.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:52.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:52.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:52.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:52.742 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-23T19:01:52.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-23T19:01:52.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:52.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:52.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:52.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:52.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:52.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:52.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:52.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-23T19:01:52.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:52.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:52.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:52.961 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-23T19:01:52.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-23T19:01:52.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:53.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:53.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:53.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:01:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:53.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:53.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:53.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T19:01:53.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:53.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:53.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:53.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:53.168 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-23T19:01:53.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T19:01:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:53.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:53.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:53.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:53.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:53.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:53.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:53.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T19:01:53.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:53.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:53.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:53.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:53.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:53.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:53.412 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:53.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-23T19:01:53.430 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:53.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:53.492 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:53.501 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-23T19:01:53.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-23T19:01:53.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:01:53.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:53.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:53.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-23T19:01:53.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:53.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:53.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:53.677 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:53.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:53.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:53.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:53.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:53.720 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-23T19:01:53.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-23T19:01:53.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:53.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:53.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:53.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:53.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:53.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:53.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:53.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:53.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:53.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:53.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:53.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:53.982 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:01:53.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:01:53.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:01:54.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:54.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:54.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:01:54.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:54.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:54.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:54.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:54.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:54.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:54.201 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:54.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:54.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:54.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:54.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:54.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:54.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:54.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:54.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:54.312 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:54.319 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:54.345 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:54.352 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:54.359 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:54.366 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:54.411 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:54.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:54.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:54.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:54.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:54.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:54.455 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:54.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:54.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:54.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:54.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:54.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:54.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:54.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:54.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:54.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:54.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:54.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:54.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:54.702 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:54.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:54.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:54.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:54.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:54.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:54.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:54.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:54.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:54.829 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:54.856 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:54.864 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:54.873 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:54.882 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:54.943 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:54.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:54.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:54.970 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:54.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:54.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:54.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:55.005 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:55.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:55.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:55.087 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:55.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:55.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:55.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:55.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:55.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:55.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:55.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:55.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:55.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:55.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:55.295 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-23T19:01:55.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-23T19:01:55.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:55.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:55.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:55.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:55.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:55.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:55.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:55.475 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:55.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:55.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:55.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:55.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:55.578 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-23T19:01:55.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-23T19:01:55.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:55.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:55.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-23T19:01:55.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:55.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:55.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:55.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:55.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:55.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:55.822 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-23T19:01:55.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-23T19:01:55.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:55.900 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:55.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:01:55.914 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T19:01:55.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:55.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:55.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:55.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:55.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:55.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:56.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:56.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:56.056 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:56.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:56.118 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:01:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:56.132 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:56.177 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-04-23T19:01:56.184 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-23T19:01:56.192 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-04-23T19:01:56.199 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-04-23T19:01:56.205 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-04-23T19:01:56.213 INFO:teuthology.orchestra.run.smithi033.stdout: "restful", 2024-04-23T19:01:56.222 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_schedule" 2024-04-23T19:01:56.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:56.239 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-04-23T19:01:56.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:56.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-04-23T19:01:56.318 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:56.327 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:56.352 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:56.363 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:01:56.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:01:56.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:01:56.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:56.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:56.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:01:56.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:56.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:56.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:56.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-04-23T19:01:56.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:56.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:56.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:56.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:56.604 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:01:56.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:01:56.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:56.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:56.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:56.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:56.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:56.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:56.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:56.804 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:56.813 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:56.822 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:56.831 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:56.839 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:56.848 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:56.858 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:56.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:56.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:56.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:56.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:56.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:56.991 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:01:57.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:01:57.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:57.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:57.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:57.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:57.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:57.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:57.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:57.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:57.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:57.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:57.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:57.200 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:01:57.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:01:57.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:57.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:57.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:57.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:01:57.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:57.319 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:57.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:01:57.336 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:01:57.345 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:01:57.353 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:01:57.362 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:01:57.370 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:01:57.380 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:01:57.441 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:01:57.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:57.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:57.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:57.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:57.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:57.521 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:01:57.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:01:57.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:57.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:57.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:57.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:01:57.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:57.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:57.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:57.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:01:57.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:57.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:57.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:57.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:57.742 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-04-23T19:01:57.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-04-23T19:01:57.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:57.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:57.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:57.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:57.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:57.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:57.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:57.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-04-23T19:01:57.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:57.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:57.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:57.976 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-04-23T19:01:57.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-04-23T19:01:57.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:58.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:58.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:58.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-04-23T19:01:58.055 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:58.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:58.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:58.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-04-23T19:01:58.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:58.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:58.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:58.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:58.156 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-04-23T19:01:58.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-04-23T19:01:58.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:58.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:58.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:58.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:58.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:58.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:58.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:58.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-04-23T19:01:58.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:58.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:58.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:58.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:58.480 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-04-23T19:01:58.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-04-23T19:01:58.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:58.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:58.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:58.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:58.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:58.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:58.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-04-23T19:01:58.677 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:58.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:58.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:58.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:58.743 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-04-23T19:01:58.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-04-23T19:01:58.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:01:58.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:58.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:58.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-04-23T19:01:58.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:58.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:58.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:58.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-04-23T19:01:58.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:58.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:58.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:58.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:58.984 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-04-23T19:01:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-04-23T19:01:59.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:59.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:59.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:59.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:59.088 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:59.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:59.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-04-23T19:01:59.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:59.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:59.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:59.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:59.200 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-04-23T19:01:59.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-04-23T19:01:59.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:59.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:59.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:59.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:59.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:59.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:59.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:59.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-23T19:01:59.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:59.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:59.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:59.389 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-04-23T19:01:59.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-04-23T19:01:59.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:01:59.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:59.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:59.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:01:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:59.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:59.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:59.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-04-23T19:01:59.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:59.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:59.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:59.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:59.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:01:59.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:59.659 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:01:59.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-04-23T19:01:59.721 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:01:59.729 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:01:59.736 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:01:59.764 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-04-23T19:01:59.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-04-23T19:01:59.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:01:59.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:01:59.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:01:59.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:01:59.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:01:59.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:01:59.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:01:59.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-04-23T19:01:59.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:01:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:01:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:01:59.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:01:59.974 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-04-23T19:01:59.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-04-23T19:01:59.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:00.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:00.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:00.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-04-23T19:02:00.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:00.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:00.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:00.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-04-23T19:02:00.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T19:02:00.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:00.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:00.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:00.182 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-04-23T19:02:00.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-04-23T19:02:00.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:02:00.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:00.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:00.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:02:00.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T19:02:00.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T19:02:00.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:00.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-23T19:02:00.418 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T19:02:00.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:00.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:00.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:00.458 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-04-23T19:02:00.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-04-23T19:02:00.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:02:00.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:00.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:00.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-04-23T19:02:00.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:02:00.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-04-23T19:02:00.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:00.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-23T19:02:00.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:00.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:00.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:00.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:00.768 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-04-23T19:02:00.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-04-23T19:02:00.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:00.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:00.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:00.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-04-23T19:02:00.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:00.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:00.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:00.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-23T19:02:00.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-23T19:02:00.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:00.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:00.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:01.005 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-04-23T19:02:01.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-04-23T19:02:01.077 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:02:01.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:01.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-04-23T19:02:01.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-04-23T19:02:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-04-23T19:02:01.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:01.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-04-23T19:02:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-23T19:02:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:01.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:01.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:01.219 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-04-23T19:02:01.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-04-23T19:02:01.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:01.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:01.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:01.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-04-23T19:02:01.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:01.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:01.418 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-04-23T19:02:01.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-23T19:02:01.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:01.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:01.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:01.459 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-04-23T19:02:01.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-04-23T19:02:01.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:02:01.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:01.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:01.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:02:01.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-04-23T19:02:01.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-04-23T19:02:01.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:01.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-23T19:02:01.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-23T19:02:01.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:01.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:01.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:01.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:01.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:01.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:01.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:01.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:01.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:01.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:01.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:01.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:01.970 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:01.978 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:02.037 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:02.046 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:02.055 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:02.064 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:02.073 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:02.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:02.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:02.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:02.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:02.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:02.190 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:02.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:02.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:02.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:02.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:02.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:02.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:02.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:02.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:02.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:02.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:02.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:02.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:02.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:02.405 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:02.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:02.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:02.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:02.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:02.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:02.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:02.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:02.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:02.593 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:02.646 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:02.656 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:02.667 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:02.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:02.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:02.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:02.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:02.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:02.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:02.824 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:02.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:02.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:02.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:02.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:02.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:02.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:02.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:02.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:03.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:03.016 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:03.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:03.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:03.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:03.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-04-23T19:02:03.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-04-23T19:02:03.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:02:03.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:03.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:03.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:02:03.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:03.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:03.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:03.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-23T19:02:03.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:03.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:03.289 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:03.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:03.310 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T19:02:03.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T19:02:03.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:03.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:03.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:03.408 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-04-23T19:02:03.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:03.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:03.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:03.488 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-04-23T19:02:03.498 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-04-23T19:02:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-04-23T19:02:03.546 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-04-23T19:02:03.556 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-04-23T19:02:03.565 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:03.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-04-23T19:02:03.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:03.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:03.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:03.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:03.621 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-04-23T19:02:03.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-04-23T19:02:03.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:03.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:03.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:03.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:03.760 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:03.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:03.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:03.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-23T19:02:03.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:03.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:03.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:03.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:03.879 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:02:03.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:02:03.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:03.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:03.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:04.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:02:04.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:04.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:04.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:04.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-23T19:02:04.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-04-23T19:02:04.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-04-23T19:02:04.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:04.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:04.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:04.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:02:04.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:02:04.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:04.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:04.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-23T19:02:04.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-23T19:02:04.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:04.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:04.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:04.544 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-04-23T19:02:04.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-04-23T19:02:04.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:02:04.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:04.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:04.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:04.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:04.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:04.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:04.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-23T19:02:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:04.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:04.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:04.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:04.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:04.842 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-04-23T19:02:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:02:04.874 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:02:04.883 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:04.892 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-04-23T19:02:04.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-04-23T19:02:04.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:02:04.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:04.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:04.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T19:02:04.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:05.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:05.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:05.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-23T19:02:05.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:05.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:05.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:05.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:05.122 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-04-23T19:02:05.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-04-23T19:02:05.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:05.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:05.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:05.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-04-23T19:02:05.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:05.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:05.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:05.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-23T19:02:05.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:05.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:05.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:05.347 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-04-23T19:02:05.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-04-23T19:02:05.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:05.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:05.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:05.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-04-23T19:02:05.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:05.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:05.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:05.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-23T19:02:05.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:05.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:05.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:05.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-04-23T19:02:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-04-23T19:02:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-23T19:02:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-23T19:02:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-04-23T19:02:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-04-23T19:02:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:02:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-23T19:02:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-04-23T19:02:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-04-23T19:02:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:02:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-04-23T19:02:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-23T19:02:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-04-23T19:02:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-04-23T19:02:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-04-23T19:02:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-04-23T19:02:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-04-23T19:02:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-23T19:02:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-04-23T19:02:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-04-23T19:02:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-23T19:02:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-04-23T19:02:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-04-23T19:02:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-23T19:02:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-04-23T19:02:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-04-23T19:02:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-23T19:02:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-04-23T19:02:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-04-23T19:02:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-23T19:02:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:02:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-04-23T19:02:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-04-23T19:02:07.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:07.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:07.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:07.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-23T19:02:07.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:07.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:07.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:07.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-04-23T19:02:07.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:07.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:07.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:07.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:07.908 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-04-23T19:02:07.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-04-23T19:02:07.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:07.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:07.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:07.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-23T19:02:08.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:08.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:08.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:08.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-04-23T19:02:08.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:08.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:08.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:08.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:08.195 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-04-23T19:02:08.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-04-23T19:02:08.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:08.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:08.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:08.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-23T19:02:08.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:08.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:08.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:02:08.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:08.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:08.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:08.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-04-23T19:02:08.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-04-23T19:02:08.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:08.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:08.476 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:08.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-23T19:02:08.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:08.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:08.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:08.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-04-23T19:02:08.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:08.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:08.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:08.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:08.598 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-04-23T19:02:08.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-04-23T19:02:08.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:08.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:08.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:08.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-23T19:02:08.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:08.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:08.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:08.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-04-23T19:02:08.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:08.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:08.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:08.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:08.804 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-04-23T19:02:08.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-04-23T19:02:08.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:08.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:08.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:08.916 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-23T19:02:08.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:08.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:08.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:08.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-04-23T19:02:08.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:08.970 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:08.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:08.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:08.996 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-04-23T19:02:09.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-04-23T19:02:09.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:09.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:09.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-23T19:02:09.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:09.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:09.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:09.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-04-23T19:02:09.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:09.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:09.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:09.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:09.280 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-04-23T19:02:09.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-04-23T19:02:09.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:09.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:09.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:09.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-23T19:02:09.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:09.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:09.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:09.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-04-23T19:02:09.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:09.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:09.604 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-04-23T19:02:09.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-04-23T19:02:09.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:09.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:09.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:09.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-23T19:02:09.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:09.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:09.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:09.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-04-23T19:02:09.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:09.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:09.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:09.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:09.870 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-04-23T19:02:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-04-23T19:02:09.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:09.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:09.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:09.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-23T19:02:09.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:09.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:09.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:09.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:02:09.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:09.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:09.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:10.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:10.014 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-04-23T19:02:10.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-04-23T19:02:10.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:10.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:10.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:10.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-23T19:02:10.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:10.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:10.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:10.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-04-23T19:02:10.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:10.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:10.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:10.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:10.213 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-04-23T19:02:10.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-04-23T19:02:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:10.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:10.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:10.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-23T19:02:10.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:10.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:10.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:10.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-04-23T19:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:10.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:10.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:10.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:10.450 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-04-23T19:02:10.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-04-23T19:02:10.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:10.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:10.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:10.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-23T19:02:10.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:10.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:10.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:10.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-04-23T19:02:10.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:10.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:10.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:10.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:10.677 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-04-23T19:02:10.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-04-23T19:02:10.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:10.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:10.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:10.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-23T19:02:10.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:10.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:10.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:10.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-04-23T19:02:10.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:10.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:10.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:10.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:10.979 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-04-23T19:02:10.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-04-23T19:02:10.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:11.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:11.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:11.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-23T19:02:11.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:11.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:11.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:11.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-04-23T19:02:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-04-23T19:02:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-04-23T19:02:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-04-23T19:02:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-04-23T19:02:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-04-23T19:02:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:02:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-23T19:02:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-04-23T19:02:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-04-23T19:02:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-04-23T19:02:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-23T19:02:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-04-23T19:02:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-04-23T19:02:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-04-23T19:02:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-23T19:02:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-04-23T19:02:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-04-23T19:02:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-04-23T19:02:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-04-23T19:02:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-04-23T19:02:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-04-23T19:02:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-23T19:02:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-04-23T19:02:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-04-23T19:02:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:02:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-04-23T19:02:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-04-23T19:02:12.740 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-04-23T19:02:12.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:12.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:12.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-23T19:02:12.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:12.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:12.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-23T19:02:12.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:12.960 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:12.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:12.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:12.987 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-04-23T19:02:12.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-04-23T19:02:13.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:13.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:13.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:02:13.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:13.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:13.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:13.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-04-23T19:02:13.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:13.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:13.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:13.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:13.197 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-04-23T19:02:13.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-04-23T19:02:13.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:13.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:13.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:13.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:13.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:13.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:13.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:13.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-23T19:02:13.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:13.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:13.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:13.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:13.450 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-04-23T19:02:13.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-04-23T19:02:13.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:13.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:13.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:13.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:13.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:13.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:13.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-23T19:02:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:13.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:13.694 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:13.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:13.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:13.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:13.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:13.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:13.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:13.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:13.805 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:13.814 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:13.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:13.885 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:13.895 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:13.970 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:14.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:14.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:14.051 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:14.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:14.074 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-04-23T19:02:14.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-04-23T19:02:14.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:14.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:14.115 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:14.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:14.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:14.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:14.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:14.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-23T19:02:14.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:14.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:14.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:14.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:14.369 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:14.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:14.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:14.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:14.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:14.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:14.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:14.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:14.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:14.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-23T19:02:14.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:14.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:14.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:14.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:14.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:14.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:14.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:14.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:14.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:14.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:14.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:14.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:14.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:14.737 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:14.746 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:14.753 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:14.761 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:14.770 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:14.839 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:14.848 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:14.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:14.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:14.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:14.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:14.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:14.970 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:14.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:14.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:15.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:15.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:15.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:15.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:15.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:15.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:15.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:15.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:15.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:15.182 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-23T19:02:15.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-23T19:02:15.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:15.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:15.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:15.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:15.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:15.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-23T19:02:15.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:15.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:15.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:15.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:15.412 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-23T19:02:15.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-23T19:02:15.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:15.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:15.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:15.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-04-23T19:02:15.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:15.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:15.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:15.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-23T19:02:15.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:15.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:15.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:15.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:15.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-04-23T19:02:15.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-04-23T19:02:15.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:15.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:15.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:15.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:15.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:15.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:15.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:15.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-04-23T19:02:15.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:15.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:15.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:15.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-04-23T19:02:15.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-04-23T19:02:15.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:15.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:15.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:15.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:15.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:16.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:16.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:16.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-23T19:02:16.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:16.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:16.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:16.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:16.135 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-04-23T19:02:16.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-04-23T19:02:16.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:16.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:16.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:16.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:16.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:16.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:16.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-04-23T19:02:16.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:16.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:16.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:16.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:16.408 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-04-23T19:02:16.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-04-23T19:02:16.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:16.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-04-23T19:02:16.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:16.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:16.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:16.489 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-04-23T19:02:16.498 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-04-23T19:02:16.577 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:16.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-04-23T19:02:16.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:16.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:16.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:16.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:16.735 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-04-23T19:02:16.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-04-23T19:02:16.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:16.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:16.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:16.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-04-23T19:02:16.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-23T19:02:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:16.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:16.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:16.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:16.969 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-04-23T19:02:16.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-04-23T19:02:17.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:17.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:17.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:17.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-23T19:02:17.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:17.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:17.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-23T19:02:17.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:17.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:17.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:17.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:17.189 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-04-23T19:02:17.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-04-23T19:02:17.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:17.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:17.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:17.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:17.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:17.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:17.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:17.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-23T19:02:17.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:17.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:17.371 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-04-23T19:02:17.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-04-23T19:02:17.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:17.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:17.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:17.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:17.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:17.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:17.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:17.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-23T19:02:17.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:17.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:17.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:17.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:17.588 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-04-23T19:02:17.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-04-23T19:02:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:17.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:17.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:17.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:17.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:17.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-23T19:02:17.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:17.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:17.866 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-04-23T19:02:17.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-04-23T19:02:17.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:17.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:17.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:17.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:17.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:17.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:17.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:18.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-23T19:02:18.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:18.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:18.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:18.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:18.134 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-04-23T19:02:18.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-04-23T19:02:18.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:18.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:18.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:18.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:18.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:18.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:18.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:18.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-23T19:02:18.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:18.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:18.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:18.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:18.401 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-04-23T19:02:18.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-04-23T19:02:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:18.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:18.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:18.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-04-23T19:02:18.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:18.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:18.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:18.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-04-23T19:02:18.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:18.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:18.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:18.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:18.588 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-04-23T19:02:18.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-04-23T19:02:18.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:18.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:18.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:18.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:18.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:18.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:18.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:18.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-23T19:02:18.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:18.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:18.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:18.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-04-23T19:02:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-04-23T19:02:18.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:18.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:18.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:18.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:19.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:19.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:19.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:19.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-23T19:02:19.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:19.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:19.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:19.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:19.145 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-04-23T19:02:19.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-04-23T19:02:19.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:19.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:19.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:19.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:19.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:19.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-23T19:02:19.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:19.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:19.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:19.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:19.415 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-04-23T19:02:19.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-04-23T19:02:19.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:19.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:19.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:19.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:19.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:19.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:19.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:19.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-04-23T19:02:19.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:19.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:19.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:19.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:19.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-04-23T19:02:19.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-04-23T19:02:19.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:19.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:19.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:19.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:19.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:19.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:19.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:19.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-23T19:02:19.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:19.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:19.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:19.865 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-04-23T19:02:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-04-23T19:02:19.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:19.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:20.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:20.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:20.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:20.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:20.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:20.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-23T19:02:20.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:20.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:20.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:20.098 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:20.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:20.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:20.180 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:20.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-04-23T19:02:20.204 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:02:20.238 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:02:20.249 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:20.308 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:20.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:20.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:20.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:20.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:20.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:20.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:20.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:20.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:20.397 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:20.405 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:20.418 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:20.472 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:20.480 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:20.538 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:20.554 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:20.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:20.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:20.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:20.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:20.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:20.671 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:20.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:20.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:20.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:20.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:20.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:20.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:20.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:20.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:20.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:20.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:20.915 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:20.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:20.999 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:21.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:21.019 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:21.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:21.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:21.055 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:21.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:21.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:21.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:21.100 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:21.163 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:21.172 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:21.234 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:21.245 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:21.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:21.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:21.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:21.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:21.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:21.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:21.374 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:21.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:21.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:21.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:21.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:21.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:21.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:21.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:21.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:21.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:21.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:21.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:21.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:21.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:21.626 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-04-23T19:02:21.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-04-23T19:02:21.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:21.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:21.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:21.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-04-23T19:02:21.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:21.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:21.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:21.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-23T19:02:21.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:21.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:21.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:21.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:21.838 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-04-23T19:02:21.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-04-23T19:02:21.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:21.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:21.918 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:21.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-04-23T19:02:21.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:21.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:21.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:21.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-23T19:02:21.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:21.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:22.055 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:22.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:22.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:22.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:22.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-04-23T19:02:22.224 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:02:22.234 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:02:22.245 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:22.253 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-23T19:02:22.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-23T19:02:22.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:22.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:22.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:22.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:22.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:22.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:22.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:22.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:22.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:22.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:22.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:22.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-04-23T19:02:22.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-23T19:02:22.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:22.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:22.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:22.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:22.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:22.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:22.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:22.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:22.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:22.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:22.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:22.799 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-23T19:02:22.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-23T19:02:22.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:22.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:22.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:22.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:22.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:22.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:22.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:22.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:22.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:22.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:22.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:22.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:22.967 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-04-23T19:02:23.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-04-23T19:02:23.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:23.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:23.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:23.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:23.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:23.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:23.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:23.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:23.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:23.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:23.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:23.210 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-23T19:02:23.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-23T19:02:23.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:23.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:23.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:23.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:23.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:23.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:23.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:23.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:23.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:23.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:23.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:23.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:23.452 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-04-23T19:02:23.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-23T19:02:23.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:23.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:23.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:23.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:23.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:23.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:23.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:23.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:23.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:23.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:23.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:23.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:23.692 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-23T19:02:23.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-23T19:02:23.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:23.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:23.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:23.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:23.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:23.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:23.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:23.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:23.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:23.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:23.908 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-23T19:02:23.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-23T19:02:23.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:23.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:23.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:24.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:24.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:24.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:24.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:24.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:24.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:24.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-23T19:02:24.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-23T19:02:24.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:24.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:24.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:24.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:24.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:24.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:24.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:24.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:24.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:24.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:24.372 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-23T19:02:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-23T19:02:24.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:24.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:24.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:24.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:24.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:24.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:24.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:24.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:24.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:24.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:24.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-23T19:02:24.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-23T19:02:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:24.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:24.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:24.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:24.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:24.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:24.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:24.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:24.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:24.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:24.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:24.992 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-23T19:02:25.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-23T19:02:25.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:25.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:25.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:25.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:25.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:25.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:25.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:25.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:25.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:25.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:25.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:25.241 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-23T19:02:25.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-23T19:02:25.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:25.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:25.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:25.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:25.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:25.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:25.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:25.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:25.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:25.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:25.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:25.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:25.431 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-23T19:02:25.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-23T19:02:25.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:25.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:25.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:25.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:25.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:25.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:25.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:25.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:25.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:25.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:25.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:25.637 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-04-23T19:02:25.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-23T19:02:25.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:25.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:25.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:25.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:02:25.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:25.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:25.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:25.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:25.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:25.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:25.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:25.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:25.816 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-23T19:02:25.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-23T19:02:25.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:25.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:25.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:25.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:25.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:26.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:26.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:26.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:26.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:26.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:26.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:26.123 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-04-23T19:02:26.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-04-23T19:02:26.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:26.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:26.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:26.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:26.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:26.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:26.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:26.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:26.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:26.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:26.404 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-04-23T19:02:26.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-23T19:02:26.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:26.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:26.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:26.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:02:26.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:26.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:26.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:26.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:26.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:26.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:26.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:26.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:26.618 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-23T19:02:26.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-23T19:02:26.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:26.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:26.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:26.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:26.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:26.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:26.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:26.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:26.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:26.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:26.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:26.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:26.822 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-23T19:02:26.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-23T19:02:26.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:26.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:26.906 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:26.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:26.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:26.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:26.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:26.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:27.001 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-23T19:02:27.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-23T19:02:27.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:27.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:27.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:27.094 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:27.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:27.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:27.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:27.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:27.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:27.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:27.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:27.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:27.213 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-23T19:02:27.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-23T19:02:27.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:27.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:27.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:27.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:27.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:27.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:27.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:27.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:27.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:27.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:27.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:27.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:27.505 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-23T19:02:27.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-23T19:02:27.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:27.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:27.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:27.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-04-23T19:02:27.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:27.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:27.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:27.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:27.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:27.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:27.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:27.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:27.703 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-04-23T19:02:27.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-23T19:02:27.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:27.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:27.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:27.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-04-23T19:02:27.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:27.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:27.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:27.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:27.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:27.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:27.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:27.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:27.867 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-04-23T19:02:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-23T19:02:27.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:27.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:27.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:27.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:27.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:27.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:27.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:28.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:28.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:28.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:28.080 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-23T19:02:28.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-23T19:02:28.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:28.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:28.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:28.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:28.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:28.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:28.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:28.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:28.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:28.273 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-23T19:02:28.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-23T19:02:28.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:28.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:28.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:28.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:28.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:28.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:28.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:28.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:28.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:28.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:28.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:28.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:28.518 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-23T19:02:28.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-23T19:02:28.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:28.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:28.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:28.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:28.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:28.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:28.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:28.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:28.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:28.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:28.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:28.766 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-23T19:02:28.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-23T19:02:28.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:28.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:28.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:28.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:28.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:28.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:28.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:28.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:28.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:28.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:28.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.022 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-23T19:02:29.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-23T19:02:29.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:29.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:29.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:29.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:29.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:29.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:29.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:29.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:29.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:29.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.213 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-23T19:02:29.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-23T19:02:29.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:29.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:29.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:29.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:29.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:29.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:29.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:29.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:29.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:29.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:29.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.431 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-23T19:02:29.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-23T19:02:29.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:29.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:29.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:29.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:29.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:29.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:29.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:29.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:29.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:29.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:29.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.595 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-23T19:02:29.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-23T19:02:29.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:29.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:29.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:29.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:29.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:29.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:29.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:29.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:29.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:29.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:29.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.792 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-04-23T19:02:29.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-23T19:02:29.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:29.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:29.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:29.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:29.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:29.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:29.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:29.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:29.917 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:29.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:29.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:29.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:29.974 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-23T19:02:29.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-23T19:02:29.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:30.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:30.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:30.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-23T19:02:30.028 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:30.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:30.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:30.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:30.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:30.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:30.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:30.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:30.204 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-23T19:02:30.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-23T19:02:30.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:30.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:30.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:30.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:30.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:30.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:30.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:30.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:30.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:30.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:30.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:30.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:30.395 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-23T19:02:30.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-23T19:02:30.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:30.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:30.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:30.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-04-23T19:02:30.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:30.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:30.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:30.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:30.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:30.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:30.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:30.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:30.572 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-23T19:02:30.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-23T19:02:30.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:30.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:30.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:30.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-04-23T19:02:30.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:30.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:30.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:30.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:30.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:30.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:30.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:30.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:30.814 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-04-23T19:02:30.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-23T19:02:30.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:30.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:30.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:30.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:30.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:30.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:30.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:31.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:31.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:31.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:31.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:31.043 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-23T19:02:31.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-23T19:02:31.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:31.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:31.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:31.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-04-23T19:02:31.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:31.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:31.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:31.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:31.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:31.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:31.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:31.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:31.321 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-04-23T19:02:31.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-23T19:02:31.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:31.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:31.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:31.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:31.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:31.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:31.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:31.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:31.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:31.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:31.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-04-23T19:02:31.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-23T19:02:31.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:31.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:31.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:31.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:31.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:31.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:31.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:31.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:31.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:31.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:31.734 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-04-23T19:02:31.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-23T19:02:31.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:31.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:31.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:31.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:31.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:31.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:31.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:31.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:31.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:31.902 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:31.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:31.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:31.931 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-23T19:02:31.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-23T19:02:32.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:32.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:32.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:32.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:02:32.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:32.098 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:32.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:32.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:32.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:32.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:32.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:32.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:32.161 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-23T19:02:32.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-23T19:02:32.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:32.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:32.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:32.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:02:32.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:32.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:32.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:32.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:32.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:32.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:32.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:32.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:32.368 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-23T19:02:32.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-23T19:02:32.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:32.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:32.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:32.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:02:32.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:32.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:32.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:32.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:32.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:32.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:32.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:32.609 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-04-23T19:02:32.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-04-23T19:02:32.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:32.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:32.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:32.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:32.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:32.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:32.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:32.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:32.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:32.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:32.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:32.784 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-04-23T19:02:32.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-04-23T19:02:32.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:32.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:32.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:32.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:32.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:32.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:32.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:32.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:32.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:32.895 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:32.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:32.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:32.973 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-04-23T19:02:32.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-04-23T19:02:32.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:32.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:33.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:33.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:33.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:33.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:33.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:33.083 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-04-23T19:02:33.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:33.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:33.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:33.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:33.149 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-04-23T19:02:33.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-04-23T19:02:33.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:33.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:33.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:33.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-04-23T19:02:33.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:33.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:33.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:33.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:33.306 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:33.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:33.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:33.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:33.350 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T19:02:33.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T19:02:33.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:33.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:33.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:33.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:33.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:33.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:33.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:33.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:33.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:33.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:33.552 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:33.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:33.571 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:33.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:33.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:33.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:33.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:33.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:33.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:33.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:33.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:33.713 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:33.723 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:33.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:33.751 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:33.801 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:33.810 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:33.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:33.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:33.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:33.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:33.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:33.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:33.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:33.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:33.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:33.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:34.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:34.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:34.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:34.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:34.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:34.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:34.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:34.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:34.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:34.114 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:34.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:34.131 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:34.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:34.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:34.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:34.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:34.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:34.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:34.319 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:34.328 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:34.336 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:34.344 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:34.357 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:34.366 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:34.376 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:34.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:34.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:34.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:34.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:34.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:34.431 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:34.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:34.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:34.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:34.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:34.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:34.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:34.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:34.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:34.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:34.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:34.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:34.633 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-04-23T19:02:34.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-04-23T19:02:34.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:34.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:34.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:34.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:34.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:34.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:34.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:34.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-04-23T19:02:34.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:34.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:34.878 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:34.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:34.892 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-04-23T19:02:34.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-04-23T19:02:34.906 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:34.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:34.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:34.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:34.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:34.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:35.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:35.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:35.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:35.075 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:35.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:35.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:35.097 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:02:35.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:02:35.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:35.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:35.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:35.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T19:02:35.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:35.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:35.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:35.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:35.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:35.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:35.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:35.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:35.327 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:02:35.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:02:35.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:35.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:35.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:35.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-04-23T19:02:35.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:35.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:35.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:35.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:35.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:35.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:35.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:35.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:35.542 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T19:02:35.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T19:02:35.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:35.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:35.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:35.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:35.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:35.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:35.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:35.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:35.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:35.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:35.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:35.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-04-23T19:02:35.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-04-23T19:02:35.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:35.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:35.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:35.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-04-23T19:02:36.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:36.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:36.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:36.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:36.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:36.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:36.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:36.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:36.176 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T19:02:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T19:02:36.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:36.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:36.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:36.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-04-23T19:02:36.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:36.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:36.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:36.523 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:36.549 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-04-23T19:02:36.558 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:36.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:36.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:36.586 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:36.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:36.671 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T19:02:36.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T19:02:36.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:36.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:36.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:37.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T19:02:37.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T19:02:37.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T19:02:37.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:37.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:37.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:37.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:37.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:37.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:37.697 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-04-23T19:02:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-04-23T19:02:37.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:37.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:37.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:38.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:38.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:38.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:38.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:38.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:38.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:38.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:38.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:38.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:38.496 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:38.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:38.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:38.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-23T19:02:38.671 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:02:38.678 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:02:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:38.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-04-23T19:02:38.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-04-23T19:02:38.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:38.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:38.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:38.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:39.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:39.140 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:39.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:39.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-04-23T19:02:39.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:39.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:39.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:39.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:39.478 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:39.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:39.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:39.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:39.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:39.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:39.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:40.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:40.140 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:40.234 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:40.302 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:40.347 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:40.497 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:40.548 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:40.610 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:40.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:40.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:40.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:40.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:40.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:40.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:40.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:40.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:41.028 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:41.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:41.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:41.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:41.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:41.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:41.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:41.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:41.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:41.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:41.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:41.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:41.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:41.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:41.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:41.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:41.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:41.965 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:42.005 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:42.094 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:42.209 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:42.270 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:42.354 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:42.423 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:42.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:42.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:42.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:42.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:42.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:42.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:42.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:42.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:42.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:42.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:42.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:42.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:42.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:43.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:43.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:43.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:43.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:43.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:43.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:43.330 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-04-23T19:02:43.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-04-23T19:02:43.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:43.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:43.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-04-23T19:02:43.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:43.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:43.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:43.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-23T19:02:43.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:44.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:44.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:44.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:44.290 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-04-23T19:02:44.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-04-23T19:02:44.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:44.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:44.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:44.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-04-23T19:02:44.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:44.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:44.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:44.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-23T19:02:44.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:44.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:44.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:45.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:45.146 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-04-23T19:02:45.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-04-23T19:02:45.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:45.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:45.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:45.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-04-23T19:02:45.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:45.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:45.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:45.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-04-23T19:02:45.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:45.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:45.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:45.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:45.881 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-04-23T19:02:45.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-04-23T19:02:46.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:46.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:46.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:46.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T19:02:46.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:46.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:46.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:46.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-23T19:02:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:46.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:46.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:46.504 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-04-23T19:02:46.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-04-23T19:02:46.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:46.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:46.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:46.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:02:46.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:46.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:47.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:47.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-23T19:02:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:47.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:47.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:47.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:47.408 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:02:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:02:47.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:47.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:47.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:47.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:02:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:47.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:47.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:47.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-04-23T19:02:47.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:47.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:48.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:48.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:48.097 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-04-23T19:02:48.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-04-23T19:02:48.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:48.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:48.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:48.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-04-23T19:02:48.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:48.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:48.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:48.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-23T19:02:48.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:48.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:48.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:48.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:48.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:48.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:48.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-04-23T19:02:48.987 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:02:49.040 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:02:49.091 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:49.120 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:49.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:49.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:49.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:49.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:49.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:49.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:49.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:49.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:49.798 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:49.807 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:49.817 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:49.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:49.912 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:50.068 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:50.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:50.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:50.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:50.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:50.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:50.376 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:02:50.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:02:50.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:50.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:50.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:50.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:50.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:50.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:50.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:50.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:51.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:51.114 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:51.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:51.142 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:02:51.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:02:51.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:51.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:51.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:51.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:02:51.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:51.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:51.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:51.745 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:51.809 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:51.851 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:51.909 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:51.982 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:52.043 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:52.067 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:02:52.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:52.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:52.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:52.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:52.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:52.465 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:02:52.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:02:52.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:02:52.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:52.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:52.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:02:52.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:52.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:52.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:53.016 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:53.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:53.078 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:53.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:53.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:53.180 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-04-23T19:02:53.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-04-23T19:02:53.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:53.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:53.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:53.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-04-23T19:02:53.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:53.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:53.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:53.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:53.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:53.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:54.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:54.145 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-04-23T19:02:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-04-23T19:02:54.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:54.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:54.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:54.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-04-23T19:02:54.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:54.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:54.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:54.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:54.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:54.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:55.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:55.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:55.131 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:02:55.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:02:55.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:55.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:55.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:55.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-04-23T19:02:55.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:55.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:55.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:55.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:02:55.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:55.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:55.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:55.823 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:02:55.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:55.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:02:56.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-04-23T19:02:56.107 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-04-23T19:02:56.143 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-04-23T19:02:56.151 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:02:56.158 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-04-23T19:02:56.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-04-23T19:02:56.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:02:56.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:56.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:56.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-04-23T19:02:56.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:56.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:56.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:56.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-23T19:02:56.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:56.880 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:56.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:56.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:57.079 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-04-23T19:02:57.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-04-23T19:02:57.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:57.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:57.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:57.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-04-23T19:02:57.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:57.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:57.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:57.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-23T19:02:57.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:57.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:57.708 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:57.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:57.831 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-04-23T19:02:57.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-04-23T19:02:57.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:57.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:57.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:57.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:57.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:58.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:58.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:58.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-04-23T19:02:58.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:58.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:58.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:58.521 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:02:58.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:02:58.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:02:58.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:58.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:02:58.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-04-23T19:02:58.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-04-23T19:02:58.902 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:58.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:02:58.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-23T19:02:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:02:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:02:59.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:02:59.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:02:59.300 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:02:59.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:02:59.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:02:59.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:02:59.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:02:59.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:02:59.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:02:59.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:02:59.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:02:59.665 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:02:59.700 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:02:59.748 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:02:59.794 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:02:59.841 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:02:59.938 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:02:59.963 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:00.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:00.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:00.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:00.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:00.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:00.328 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:00.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:00.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:00.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:00.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:00.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:00.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:00.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:00.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:00.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:00.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:00.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:00.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:00.913 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:00.983 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:01.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:01.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:01.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:01.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:01.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:01.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:01.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:01.370 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:01.407 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:01.433 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:01.488 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:01.548 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:01.588 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:01.663 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:01.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:01.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:01.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:01.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:01.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:01.938 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:02.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:02.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:02.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:02.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:02.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:02.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:02.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:02.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:02.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:02.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:02.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:02.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:02.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:02.501 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-04-23T19:03:02.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-04-23T19:03:02.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:02.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:02.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:02.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:02.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:02.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:02.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-04-23T19:03:02.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:02.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:02.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:02.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:02.958 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-04-23T19:03:02.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-04-23T19:03:03.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:03.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:03.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:03.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-04-23T19:03:03.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:03.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:03.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:03.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-04-23T19:03:03.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:03.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:03.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:03.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:03.872 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-04-23T19:03:03.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-04-23T19:03:04.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:04.087 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:04.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:04.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-04-23T19:03:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:04.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:04.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:04.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T19:03:04.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:04.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:04.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:04.744 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-04-23T19:03:04.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-04-23T19:03:04.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:04.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:04.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:05.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:03:05.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:03:05.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-04-23T19:03:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:05.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-23T19:03:05.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:05.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:05.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:05.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:05.549 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-04-23T19:03:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-04-23T19:03:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-04-23T19:03:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-04-23T19:03:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-04-23T19:03:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-04-23T19:03:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-23T19:03:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-04-23T19:03:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:07.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:07.766 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:07.849 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:07.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:07.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:07.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:07.971 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:08.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:08.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:08.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:08.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:08.249 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:08.257 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:08.335 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:08.373 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:08.470 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:08.509 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:08.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:08.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:08.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:08.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:08.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:08.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:08.892 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:09.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:09.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:09.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:09.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:09.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:09.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:09.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:09.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:09.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:09.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:09.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:09.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:09.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:09.935 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:10.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:10.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:10.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:10.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:10.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:10.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:10.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:10.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:10.457 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:10.487 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:10.496 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:10.506 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:10.515 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:10.525 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:10.533 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:10.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:10.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:10.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:10.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:10.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:10.685 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:10.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:10.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:10.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:11.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:11.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-04-23T19:03:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:12.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:12.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:13.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:13.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:13.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:13.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:13.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:13.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:13.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:13.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:13.492 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:13.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:13.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:13.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:13.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:13.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:13.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:14.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:14.011 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:14.020 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:14.098 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:14.165 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:14.237 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:14.306 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:14.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:14.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:14.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:14.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:14.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:14.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:14.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:14.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:14.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:14.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:14.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:14.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:14.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:14.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:14.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:15.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:15.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:15.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:15.227 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:15.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:15.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:15.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-04-23T19:03:15.257 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:15.264 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:15.278 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-04-23T19:03:15.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-04-23T19:03:15.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:15.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:15.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:15.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-04-23T19:03:15.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:15.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:15.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:15.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-23T19:03:15.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:15.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:15.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:16.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-04-23T19:03:16.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-04-23T19:03:16.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:16.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:16.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:16.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:03:16.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-04-23T19:03:16.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:16.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:16.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-23T19:03:16.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:16.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:16.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:16.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:16.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:16.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:16.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:16.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:16.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:16.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:16.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:16.923 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:16.957 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:16.978 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:16.986 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:17.067 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:17.095 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:17.154 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:17.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:17.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:17.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:17.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:17.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:17.476 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:17.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:17.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:17.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:17.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:17.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:17.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:18.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:18.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:18.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:18.211 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:18.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:18.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:18.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:18.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:18.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:18.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:18.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:18.675 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:18.684 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:18.765 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:18.839 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:18.900 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:18.972 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:19.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:19.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:19.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:19.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:19.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:19.075 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:19.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:19.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:19.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:19.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:19.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:19.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:19.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:19.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:19.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:19.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:19.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:19.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:19.726 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:19.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:19.776 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-04-23T19:03:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:19.994 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:20.022 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:20.030 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-04-23T19:03:20.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-04-23T19:03:20.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:20.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:20.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-04-23T19:03:20.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:20.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:20.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:20.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-04-23T19:03:20.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-23T19:03:20.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:20.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:20.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:20.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:21.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:21.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:21.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:21.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:21.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:21.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:21.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:21.416 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:21.539 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:21.589 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:21.630 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:21.699 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:21.765 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:21.798 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:21.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:21.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:22.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:22.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:22.120 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:22.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:22.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:22.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:22.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:22.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:22.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:22.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:22.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:22.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:22.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:22.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:22.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:22.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:22.822 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:22.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:22.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:22.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:22.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:23.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:23.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:23.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:23.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:23.166 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:23.182 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:23.191 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:23.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:23.212 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:23.221 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:23.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:23.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:23.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:23.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:23.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:23.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:23.551 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:23.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:23.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:23.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:23.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:23.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:23.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:23.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:24.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:24.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:24.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:24.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:24.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:24.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-04-23T19:03:24.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-04-23T19:03:24.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:24.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:24.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:24.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:24.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:24.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:24.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:24.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-04-23T19:03:24.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-23T19:03:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:24.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:24.699 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-04-23T19:03:24.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-04-23T19:03:24.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:24.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:24.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:24.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-04-23T19:03:24.883 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:24.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:25.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:25.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-04-23T19:03:25.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:25.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:25.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:25.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:25.387 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-04-23T19:03:25.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-04-23T19:03:25.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:03:25.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:25.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:25.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-04-23T19:03:25.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:25.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:25.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:25.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-04-23T19:03:26.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:26.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:26.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:26.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:26.246 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-04-23T19:03:26.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-04-23T19:03:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:26.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:26.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:26.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:26.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:26.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:26.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:26.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-04-23T19:03:26.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:26.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:26.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:26.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-04-23T19:03:27.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-04-23T19:03:27.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:27.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:27.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:27.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-04-23T19:03:27.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:27.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:27.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:27.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-23T19:03:27.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-23T19:03:27.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:27.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:27.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:27.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:27.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:27.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-04-23T19:03:27.930 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:27.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:28.071 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:28.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:28.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:28.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:28.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:28.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:28.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:28.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:28.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:28.475 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:28.532 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:28.557 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:28.625 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:28.711 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:28.747 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:28.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:28.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:28.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:28.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:29.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:29.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:29.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:29.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:29.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:29.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:29.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:29.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:29.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:29.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:29.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:29.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:29.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:29.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:29.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:30.052 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:30.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:30.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:30.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:30.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:30.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:30.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:30.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:30.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:30.321 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:30.382 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:30.427 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:30.549 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:30.603 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:30.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:30.710 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:30.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:30.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:30.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:30.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:30.987 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:31.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:31.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:31.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:31.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:31.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:31.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:31.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:31.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:31.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:31.712 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:31.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:31.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:31.775 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:31.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-04-23T19:03:31.901 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:31.948 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:32.033 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:32.065 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:32.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:32.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:32.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:32.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:32.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:32.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:32.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:32.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:32.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:32.754 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:32.809 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:32.877 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:32.924 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:32.966 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:33.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:33.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:33.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:33.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:33.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:33.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:33.374 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:33.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:33.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:33.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:33.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:33.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:33.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:33.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:33.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:33.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:33.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:33.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:33.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:33.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:33.851 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:33.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:33.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:34.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:34.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:34.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:34.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:34.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:34.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:34.420 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:34.479 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:34.514 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:34.594 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:34.647 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:34.725 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:34.789 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:34.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:34.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:35.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:35.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:35.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:35.160 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:35.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:35.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:35.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:35.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:35.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:35.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:35.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:35.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:35.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:35.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:35.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:35.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:35.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:35.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:36.151 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:36.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-04-23T19:03:36.172 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:36.374 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:36.423 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:36.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:36.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:36.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:36.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:36.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:36.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:36.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:36.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:36.912 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:36.945 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:37.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:37.067 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:37.132 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:37.263 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:37.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:37.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:37.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:37.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:37.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:37.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:37.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:37.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:37.740 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:37.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:37.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:37.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:37.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:38.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:38.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:38.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:38.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:38.366 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:38.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:38.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:38.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:38.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:38.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:38.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:38.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:38.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:38.772 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:38.781 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:38.878 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:38.948 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:39.114 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:39.184 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:39.226 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:39.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:39.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:39.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:39.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:39.559 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:39.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:39.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:39.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:39.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:39.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:39.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:39.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:40.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:40.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:40.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:40.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:40.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:40.414 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:40.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:40.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:40.535 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:40.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T19:03:40.628 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:40.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:40.780 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:40.789 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-04-23T19:03:40.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-04-23T19:03:40.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:40.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:40.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:41.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:41.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:41.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-23T19:03:41.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:41.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:41.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:41.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:41.533 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:41.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:41.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:41.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:41.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:41.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:41.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:41.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:41.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:42.006 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:42.118 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:42.181 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:42.209 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:42.259 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:42.268 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:42.277 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:42.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:42.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:42.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:42.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:42.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:42.557 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:42.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:42.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:42.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:42.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:42.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:43.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:43.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:43.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:43.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:43.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:43.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:43.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:43.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:43.261 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:43.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:43.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:43.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:43.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:43.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:43.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:43.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:43.759 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:43.815 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:43.857 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:43.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:43.985 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:44.043 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:44.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:44.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:44.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:44.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:44.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:44.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:44.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:44.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:44.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:44.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:44.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:44.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:44.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:44.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:44.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:45.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:45.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:45.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:45.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:45.303 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-04-23T19:03:45.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-04-23T19:03:45.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:45.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:45.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:45.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:45.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:45.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:45.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:45.887 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-04-23T19:03:45.964 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-04-23T19:03:45.987 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-04-23T19:03:46.081 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:46.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-04-23T19:03:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:46.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:46.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:46.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:46.384 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:46.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:46.534 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:46.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-04-23T19:03:46.660 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:46.700 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:46.798 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:46.866 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:46.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:46.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:47.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:47.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:47.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:47.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:47.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:47.325 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:47.342 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:47.351 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:47.405 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:47.508 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:47.559 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:47.606 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:47.814 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:47.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:47.943 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:48.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:48.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:48.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:48.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:48.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:48.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:48.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:48.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:48.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:48.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:48.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:48.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:48.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:48.796 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:48.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:48.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:48.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:49.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:49.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:49.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:49.060 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:49.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:49.075 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:49.111 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:49.169 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:49.257 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:49.306 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:49.356 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:49.439 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:49.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:49.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:49.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:49.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:49.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:49.768 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:49.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:49.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:49.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:49.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:49.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:49.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:50.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:50.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:50.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:50.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:50.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:50.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:50.484 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:50.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:50.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:50.693 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:50.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-04-23T19:03:50.800 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:50.809 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:50.827 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:50.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:50.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:50.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:50.916 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:51.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:51.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:51.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:51.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:51.374 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:51.467 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:51.485 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:51.503 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:51.512 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:51.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:51.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:51.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:51.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:51.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:51.910 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:52.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:52.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:52.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:52.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:52.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:52.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:52.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:52.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:52.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:52.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:52.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:52.601 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:52.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:52.620 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:52.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:52.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:52.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:52.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:52.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:52.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:52.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:52.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:52.963 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:53.014 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:53.024 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:53.033 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:53.041 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:53.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:53.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:53.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:53.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:53.241 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:53.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:53.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:53.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:53.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:53.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:53.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:53.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:53.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:53.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:53.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:53.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:53.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:53.919 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:53.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:03:53.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:53.985 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:03:53.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-04-23T19:03:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:03:54.184 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:03:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:03:54.344 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:03:54.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:03:54.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:54.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:54.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:54.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:03:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:54.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:54.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:54.780 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:54.860 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:54.914 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:54.976 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:54.985 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:54.994 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:55.071 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:55.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:55.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:55.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:55.352 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:55.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:55.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:03:55.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:03:55.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:55.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:55.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:55.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:55.916 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:55.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:56.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:56.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:56.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:56.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:56.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:56.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:56.636 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:03:56.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:03:56.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:03:56.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:56.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:56.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:03:57.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:57.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:57.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:03:57.242 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:03:57.314 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:03:57.407 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:03:57.521 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:03:57.564 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:03:57.683 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:03:57.813 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:03:57.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:57.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:57.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:58.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:58.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:03:58.117 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:03:58.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:03:58.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:58.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:03:58.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:03:58.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:58.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:58.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:58.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:58.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:58.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:58.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:58.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:58.958 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:03:59.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:03:59.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:03:59.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:59.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:59.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:03:59.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:03:59.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:59.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:59.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:03:59.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:03:59.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:59.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:03:59.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:03:59.753 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-04-23T19:03:59.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-04-23T19:03:59.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:03:59.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:03:59.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:03:59.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-04-23T19:03:59.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-04-23T19:03:59.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:03:59.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:03:59.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-04-23T19:03:59.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-23T19:03:59.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:03:59.960 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:00.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:00.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:00.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:00.254 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:00.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-04-23T19:04:00.453 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:00.485 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:00.553 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:00.616 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-04-23T19:04:00.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-04-23T19:04:00.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:00.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:00.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:00.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:00.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:00.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:00.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:00.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-23T19:04:01.020 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:01.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:01.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:01.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:01.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T19:04:01.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T19:04:01.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:01.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:01.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:01.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:04:01.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:01.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:01.775 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:01.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:01.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:01.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:01.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:02.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:02.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:02.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:02.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:02.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:02.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:02.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:02.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:02.626 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:02.708 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:02.761 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:02.791 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:02.800 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:02.808 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:02.878 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:02.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:02.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:03.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:03.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:03.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:03.212 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:03.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:03.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:03.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:03.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:03.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:03.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:03.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:03.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:03.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:03.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:03.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:03.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:03.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:03.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:04.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:04.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:04.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:04.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:04.336 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:04.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:04.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:04.459 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:04.497 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:04.554 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:04.613 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:04.637 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:04.685 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:04.728 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:04.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:04.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:04.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:04.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:05.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:05.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:05.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:05.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:05.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:05.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:05.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:05.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:05.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:05.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:05.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:05.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-04-23T19:04:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-04-23T19:04:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-04-23T19:04:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-04-23T19:04:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-04-23T19:04:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-04-23T19:04:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:04:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-04-23T19:04:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-04-23T19:04:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-04-23T19:04:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-04-23T19:04:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-04-23T19:04:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:07.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:07.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:07.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:04:07.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:07.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:07.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:07.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:07.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:08.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:08.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:08.158 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-04-23T19:04:08.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-04-23T19:04:08.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:08.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:08.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:08.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:04:08.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:08.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:08.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:08.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-23T19:04:08.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-23T19:04:08.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:08.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:08.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:08.774 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:08.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:09.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:09.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:09.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:09.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:09.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:09.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:09.411 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:09.421 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:09.429 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:09.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:09.448 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:09.456 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:09.465 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:09.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:09.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:09.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:09.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:09.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:09.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:09.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:09.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:09.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:10.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:10.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:10.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:10.181 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:10.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:10.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:10.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:10.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:10.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:10.437 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:10.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:10.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:10.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:10.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:10.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:10.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:10.815 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:10.875 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:10.911 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:10.993 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:11.057 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-04-23T19:04:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-04-23T19:04:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:12.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:12.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:12.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:12.873 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-23T19:04:12.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-23T19:04:12.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:13.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:13.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-04-23T19:04:13.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:13.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:13.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:13.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:13.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:13.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:13.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-04-23T19:04:13.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-04-23T19:04:13.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:04:13.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:13.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-04-23T19:04:13.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:13.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:13.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:14.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:14.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:14.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:14.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:14.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:14.234 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:04:14.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:04:14.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:14.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:14.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:14.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-04-23T19:04:14.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:14.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:14.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:14.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-23T19:04:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:14.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:14.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:14.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:15.063 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:04:15.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:04:15.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:15.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:15.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-04-23T19:04:15.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:15.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:15.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:15.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-23T19:04:15.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:15.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:15.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:15.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:15.791 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-04-23T19:04:15.876 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-04-23T19:04:15.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:15.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:16.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:16.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-04-23T19:04:16.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:16.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:16.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:16.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:16.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:16.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:16.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:16.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:16.248 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-04-23T19:04:16.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-04-23T19:04:16.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:16.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:16.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-04-23T19:04:16.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:16.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:16.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:16.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-04-23T19:04:16.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-04-23T19:04:16.778 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:16.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:16.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:16.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:16.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:16.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:16.994 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-04-23T19:04:17.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-04-23T19:04:17.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:17.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:17.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:17.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-04-23T19:04:17.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-04-23T19:04:17.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-04-23T19:04:17.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:17.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:17.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:17.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:17.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:17.333 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:17.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:17.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:17.517 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:17.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-23T19:04:17.604 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:17.611 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:17.674 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:17.703 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:17.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:17.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:17.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:18.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:18.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:18.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:18.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:18.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:18.458 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:18.520 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:18.527 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:18.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:18.608 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:18.679 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:18.747 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:18.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:18.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:18.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:19.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:19.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:19.182 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:19.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:19.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:19.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:19.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:19.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:19.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:19.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:19.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:19.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:19.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:19.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:19.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:19.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:19.764 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:19.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:19.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:19.971 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:20.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:20.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:20.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:20.271 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:20.340 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:20.411 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:20.463 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:20.520 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:20.620 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:20.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:20.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:20.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:20.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:21.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:21.136 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:21.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:21.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:21.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:21.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:21.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:21.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:21.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:21.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:21.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:21.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:21.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:21.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:21.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-04-23T19:04:21.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-04-23T19:04:21.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:22.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:22.028 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:22.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-04-23T19:04:22.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:22.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:22.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:22.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:22.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:22.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:22.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:22.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:22.410 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-04-23T19:04:22.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-04-23T19:04:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:22.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:22.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:22.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:22.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:22.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:22.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:22.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:22.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:22.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:22.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:22.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:22.840 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-04-23T19:04:22.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-04-23T19:04:22.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:23.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:23.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:23.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:23.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:23.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:23.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:23.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:23.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:23.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:23.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:23.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:23.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:23.746 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:23.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-04-23T19:04:23.819 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:23.905 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:23.984 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:24.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-04-23T19:04:24.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-04-23T19:04:24.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:24.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:24.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:24.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:04:24.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:24.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:24.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:24.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:24.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:24.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:24.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:24.697 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-04-23T19:04:24.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-04-23T19:04:24.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:24.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:24.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:25.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:25.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:25.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:25.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:25.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:25.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:25.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:25.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:25.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:25.445 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:25.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:25.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:25.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:25.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:25.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:25.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:25.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:26.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:26.387 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:26.408 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:26.421 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:26.432 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:26.447 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:26.465 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:26.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:26.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:26.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:26.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:26.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:26.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:26.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:26.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:26.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:26.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:27.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:27.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:27.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:27.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:27.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:27.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:27.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:27.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:27.547 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:27.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:27.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:27.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:27.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:27.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:27.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:28.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:28.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:28.143 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:28.208 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:28.236 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:28.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:28.308 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:28.317 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:28.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:28.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:28.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:28.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:28.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:28.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:28.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:28.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:28.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:28.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:28.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:28.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:28.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:28.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:28.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:28.998 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:29.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:29.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:29.096 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-04-23T19:04:29.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-04-23T19:04:29.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:29.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:29.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:29.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:29.455 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:29.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:29.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:29.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:29.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:29.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:29.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:29.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:29.688 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-04-23T19:04:29.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-04-23T19:04:29.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:29.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:29.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:29.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:30.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:30.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:30.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:30.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:30.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:30.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:30.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:30.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:30.513 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:30.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:30.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:30.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-04-23T19:04:30.657 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:30.667 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:30.758 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:30.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:30.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:30.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:31.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:31.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:31.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:31.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:31.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:31.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:31.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:31.315 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:31.324 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:31.333 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:31.352 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:31.420 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:31.472 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:31.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:31.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:31.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:31.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:31.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:31.977 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:32.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:32.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:32.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:32.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:32.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:32.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:32.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:32.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:32.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:32.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:32.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:32.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:32.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:32.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:32.678 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:32.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:32.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:32.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:32.912 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:33.007 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:33.023 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:33.065 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:33.120 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:33.177 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:33.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:33.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:33.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:33.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:33.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:33.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:33.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:33.740 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:33.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:33.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:33.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:33.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:33.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:33.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:33.936 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:33.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:34.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:34.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:34.043 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:34.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:34.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:34.352 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:34.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-04-23T19:04:34.397 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:34.405 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:34.414 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:34.424 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:34.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:34.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:34.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:34.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:34.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:34.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:34.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:34.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:34.731 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:34.781 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:34.833 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:34.886 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:34.959 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:34.993 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:35.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:35.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:35.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:35.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:35.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:35.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:35.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:35.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:35.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:35.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:35.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:35.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:35.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:35.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:35.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:35.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:35.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:35.704 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:35.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:35.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:35.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:35.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:35.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:36.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:36.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:36.156 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:36.327 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:36.336 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:36.348 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:36.382 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:36.419 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:36.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:36.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:36.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:36.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:36.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:36.693 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:36.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:36.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:36.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:36.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:37.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:37.098 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:37.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:37.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-04-23T19:04:37.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-04-23T19:04:37.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:37.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:37.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:37.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-04-23T19:04:37.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:37.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:37.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:37.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-23T19:04:37.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:37.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:37.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:37.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:38.017 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:38.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:38.037 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:38.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-04-23T19:04:38.056 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:38.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:38.115 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:38.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:38.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:38.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:38.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:38.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:38.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:38.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:38.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:38.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:38.699 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:38.738 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:38.794 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:38.845 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:38.935 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:38.984 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:39.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:39.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:39.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:39.365 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:39.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:39.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:39.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:39.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:39.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:39.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:39.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:39.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:39.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:39.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:39.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:39.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:39.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:39.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:39.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:40.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:40.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:40.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:40.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:40.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:40.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:40.477 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:40.514 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:40.561 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:40.635 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:40.664 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:40.758 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:40.782 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:40.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:40.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:41.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:41.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:41.188 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:41.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:41.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:41.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:41.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:41.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:41.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:41.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:41.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:41.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:41.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:41.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:41.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:41.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:41.742 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-04-23T19:04:41.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-04-23T19:04:41.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:41.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:41.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:41.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:42.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:42.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:42.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:42.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:42.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:42.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:42.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:42.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:42.484 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-04-23T19:04:42.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-04-23T19:04:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:42.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:42.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:42.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-04-23T19:04:42.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:42.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:43.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:43.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:43.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:43.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:43.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:43.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:43.303 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-04-23T19:04:43.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-04-23T19:04:43.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:43.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:43.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:43.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:43.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:43.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:43.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:43.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:43.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:43.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:44.002 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:44.089 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-04-23T19:04:44.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-04-23T19:04:44.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:44.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:44.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:44.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:44.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:44.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:44.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:44.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:44.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:44.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:44.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:44.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:44.839 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-04-23T19:04:44.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-04-23T19:04:44.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-04-23T19:04:44.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:45.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:45.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:45.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:45.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:45.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:45.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:45.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:45.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:45.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:45.488 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-04-23T19:04:45.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-04-23T19:04:45.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:45.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:45.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:45.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:45.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:46.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:46.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:46.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:46.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:46.168 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-04-23T19:04:46.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-04-23T19:04:46.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:46.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:46.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:47.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:47.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:47.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:47.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:47.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:47.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:47.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:47.895 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:47.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:47.990 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-04-23T19:04:47.999 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-04-23T19:04:48.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:48.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:48.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:48.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:04:48.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:48.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:48.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:48.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:48.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:48.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:48.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:48.898 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-04-23T19:04:48.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-04-23T19:04:48.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:04:49.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:49.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:49.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:49.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-04-23T19:04:49.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-04-23T19:04:49.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:49.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:49.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:49.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:49.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:49.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:49.753 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-04-23T19:04:49.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-04-23T19:04:49.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:49.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:50.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:50.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:50.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:50.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:50.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:50.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:50.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:50.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:50.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:50.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:50.501 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-04-23T19:04:50.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-04-23T19:04:50.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:50.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:50.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:50.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:50.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:50.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:50.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:50.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:50.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:51.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:51.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:51.108 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-04-23T19:04:51.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-04-23T19:04:51.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:51.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:51.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:04:51.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:51.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:51.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:51.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:51.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:51.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:51.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:51.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:51.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:52.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:52.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:52.101 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-23T19:04:52.286 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:52.333 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:52.385 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:52.434 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-04-23T19:04:52.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-04-23T19:04:52.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:52.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:52.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:52.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:52.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:52.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:52.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:52.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-04-23T19:04:52.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:53.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:53.145 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-04-23T19:04:53.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-04-23T19:04:53.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:53.293 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:53.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:53.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:53.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:53.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:53.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:53.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-04-23T19:04:53.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:53.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:53.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:53.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:53.852 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:53.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:53.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:54.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:54.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:54.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:54.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:54.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:54.335 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:54.344 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:54.432 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:54.482 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:54.526 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:54.549 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:54.600 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:54.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:54.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:54.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:54.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:54.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:54.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:54.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:55.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:55.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:55.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:55.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:55.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:55.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:55.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:55.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:55.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:55.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:55.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:55.365 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:55.397 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:55.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:55.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:55.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:55.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:55.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:55.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:55.797 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:55.880 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:55.901 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:55.910 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:55.919 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:55.927 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:55.936 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:55.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:55.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:56.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:56.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:56.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:56.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:04:56.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:04:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:56.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:56.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:56.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:56.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:56.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:56.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:56.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:56.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:56.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:56.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:56.825 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:56.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:04:57.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:57.052 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:04:57.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-04-23T19:04:57.194 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:04:57.307 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:04:57.338 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:04:57.347 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:04:57.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:04:57.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:57.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:57.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:57.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:04:57.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:57.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:57.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:57.877 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:04:57.946 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:04:58.018 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:04:58.075 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:04:58.087 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:04:58.156 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:04:58.221 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:04:58.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:58.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:58.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:58.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:58.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:58.629 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:04:58.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:04:58.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:04:58.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:58.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:58.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:04:58.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:59.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:59.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:04:59.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:04:59.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:04:59.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:04:59.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:04:59.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:04:59.429 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:04:59.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:04:59.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:04:59.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:04:59.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:04:59.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:04:59.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:04:59.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:04:59.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:04:59.947 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:00.047 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:00.076 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:00.124 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:00.200 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:00.251 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:00.326 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:00.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:00.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:00.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:00.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:00.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:00.549 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:00.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:00.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:00.740 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:00.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:00.799 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:00.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:00.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:01.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:01.049 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:01.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:01.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:01.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:01.235 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:01.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:01.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:01.394 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:01.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-04-23T19:05:01.429 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:01.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:01.566 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:01.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:01.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:01.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:01.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:01.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:02.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:02.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:02.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:02.203 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:02.267 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:02.310 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:02.377 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:02.475 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:02.525 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:02.545 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:02.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:02.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:02.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:02.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:02.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:02.990 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:03.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:03.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:03.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:03.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:03.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:03.302 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:03.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:03.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:03.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:03.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:03.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:03.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:03.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:03.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:03.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:03.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:03.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:04.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:04.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:04.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:04.260 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:04.269 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:04.278 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:04.287 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:04.296 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:04.305 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:04.358 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:04.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:04.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:04.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:04.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:04.749 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:04.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:04.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:04.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:05.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:05.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:05.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:05.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:05.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:05.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:05.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:05.322 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:05.375 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:05.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:05.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:05.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-04-23T19:05:05.525 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-04-23T19:05:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-04-23T19:05:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-23T19:05:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:05:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:05:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:05:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-04-23T19:05:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:07.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:07.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:07.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:07.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:07.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:07.923 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:07.965 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:08.020 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:08.087 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:08.153 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:08.234 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:08.306 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:08.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:08.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:08.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:08.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:08.464 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:08.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:08.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:08.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:08.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:08.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:08.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:08.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:09.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:09.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:09.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:09.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:09.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:09.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:09.397 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:09.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:09.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:09.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:09.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:09.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:09.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:09.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:09.927 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:09.973 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:10.116 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:10.171 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:10.187 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:10.255 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:10.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:10.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:10.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:10.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:10.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:10.587 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:10.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:10.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:10.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:10.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:10.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:10.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:10.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:10.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:11.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-04-23T19:05:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-04-23T19:05:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-04-23T19:05:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:05:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-23T19:05:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:12.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:12.908 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-04-23T19:05:12.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-04-23T19:05:13.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:13.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:13.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:13.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:05:13.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:13.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:13.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:13.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-23T19:05:13.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:13.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:13.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:13.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:13.728 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-04-23T19:05:13.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-04-23T19:05:13.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:13.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:13.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:13.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:05:14.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:14.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:14.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:14.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-23T19:05:14.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:14.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:14.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:14.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:14.678 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-04-23T19:05:14.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-04-23T19:05:14.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:14.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:14.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:14.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:15.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:15.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:15.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:15.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-23T19:05:15.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:15.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:15.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:15.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:15.460 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-04-23T19:05:15.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-04-23T19:05:15.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:15.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:15.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:15.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:15.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:15.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:15.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-23T19:05:16.069 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:16.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:16.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:16.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:16.205 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-04-23T19:05:16.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-04-23T19:05:16.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:16.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:16.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:16.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:16.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:16.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:16.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:16.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:16.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:17.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-04-23T19:05:17.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-04-23T19:05:17.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:17.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:17.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:17.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:17.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:17.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:17.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:17.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:17.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:17.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:17.842 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-04-23T19:05:17.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-04-23T19:05:17.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:17.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:17.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:18.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-23T19:05:18.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:18.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:18.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:18.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:18.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:18.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:18.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:18.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:18.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-04-23T19:05:18.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-04-23T19:05:18.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:18.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:18.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:18.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:18.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:18.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:18.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:18.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:18.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:18.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:18.998 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:05:19.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:05:19.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:05:19.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:19.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:19.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-04-23T19:05:19.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-04-23T19:05:19.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:19.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:19.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:19.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:19.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:19.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:19.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:19.568 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-04-23T19:05:19.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-04-23T19:05:19.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:05:19.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:19.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-04-23T19:05:19.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:20.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:20.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:20.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:20.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:20.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:20.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:20.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:20.319 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-04-23T19:05:20.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-04-23T19:05:20.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:20.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:20.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:20.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:20.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:20.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:21.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:21.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:21.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:21.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:21.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:21.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:21.303 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-04-23T19:05:21.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-04-23T19:05:21.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:21.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:21.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:21.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:21.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:21.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:21.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:21.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:21.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:21.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:22.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:22.013 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:22.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:22.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:22.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:22.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:22.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:22.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:22.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:22.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:22.270 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:22.336 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:22.440 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:22.496 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:22.563 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:22.643 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:22.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:22.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:22.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:22.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:22.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:22.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:22.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:22.972 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:23.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:23.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:23.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:23.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:23.336 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:23.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:23.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:23.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:23.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:23.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:23.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:23.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:23.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:23.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:23.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:23.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:24.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:24.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:24.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:24.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:24.243 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:24.360 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:24.413 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:24.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:24.524 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:24.541 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:24.549 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:24.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:24.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:24.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:24.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:24.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:24.858 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:24.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:24.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:25.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:25.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:25.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:25.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:25.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:25.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:25.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:25.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:25.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:25.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:25.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:25.532 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-04-23T19:05:25.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-04-23T19:05:25.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:25.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:25.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:25.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:25.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:25.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:25.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:25.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:25.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:25.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:26.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:26.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:26.192 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-04-23T19:05:26.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-04-23T19:05:26.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:26.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:26.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:26.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:26.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:26.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:26.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:26.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:26.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:26.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:26.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:27.004 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-04-23T19:05:27.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-04-23T19:05:27.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:27.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:27.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:27.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-23T19:05:27.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:27.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:27.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:27.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:27.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:27.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:27.628 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:27.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:27.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:27.820 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:28.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-04-23T19:05:28.754 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:28.825 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:28.837 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:28.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:28.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:28.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:28.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:28.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:28.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:28.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:29.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:29.242 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:29.293 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:29.418 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:29.456 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:29.537 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:29.642 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:29.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:29.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:29.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:29.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:30.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:30.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:30.279 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:30.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:30.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:30.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:30.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:30.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:30.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:30.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:30.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:30.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:30.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:30.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:30.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:31.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:31.130 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:31.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:31.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:31.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:31.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:31.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:31.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:31.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:31.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:31.412 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:31.542 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:31.572 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:31.606 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:31.666 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:31.674 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:31.683 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:31.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:31.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:31.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:31.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:31.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:31.972 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:32.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:32.087 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:32.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:32.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:32.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:32.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:32.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:32.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:32.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:32.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:32.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:32.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:32.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:32.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:32.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:32.654 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:32.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-23T19:05:32.733 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:32.782 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:32.918 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:32.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:32.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:33.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:33.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:33.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:33.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:33.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:33.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:33.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:33.544 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:33.638 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:33.698 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:33.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:33.715 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:33.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:33.732 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:33.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:33.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:33.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:34.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:34.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:34.170 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:34.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:34.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:34.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:34.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:34.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:34.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:34.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:34.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:34.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:34.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:34.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:34.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:34.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:34.709 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:34.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:34.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:34.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:35.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:35.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:35.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:35.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:35.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:35.185 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:35.226 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:35.368 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:35.427 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:35.480 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:35.553 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:35.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:35.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:35.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:35.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:35.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:35.865 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:36.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:36.061 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:36.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:36.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:36.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:36.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:36.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:36.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:36.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:36.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:36.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:36.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:36.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-04-23T19:05:36.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-04-23T19:05:36.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:05:36.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:36.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:36.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-04-23T19:05:36.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:36.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:36.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:36.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-23T19:05:36.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:37.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:37.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:37.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-04-23T19:05:37.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-04-23T19:05:37.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:37.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:37.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:37.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:37.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:37.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:37.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:37.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-04-23T19:05:37.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:37.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:37.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:37.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:38.034 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-04-23T19:05:38.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-04-23T19:05:38.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:05:38.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:38.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:38.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-04-23T19:05:38.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:38.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:38.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:38.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-23T19:05:38.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:38.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:38.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:38.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:38.923 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-04-23T19:05:38.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-04-23T19:05:38.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:38.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:39.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-04-23T19:05:39.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:39.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:39.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:39.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-23T19:05:39.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:39.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:39.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:39.508 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:39.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:39.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:39.609 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:39.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-04-23T19:05:39.679 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-04-23T19:05:39.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-04-23T19:05:39.761 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-04-23T19:05:39.847 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-04-23T19:05:39.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-04-23T19:05:39.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-04-23T19:05:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:39.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:40.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-04-23T19:05:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:40.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:40.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:40.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:40.296 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:40.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:40.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:40.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:40.475 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-04-23T19:05:40.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-04-23T19:05:40.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:40.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:40.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:40.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:41.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:41.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:41.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:41.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:41.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:41.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:41.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:41.441 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-04-23T19:05:41.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-04-23T19:05:41.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-04-23T19:05:41.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:41.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:41.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-04-23T19:05:41.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:41.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:41.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:41.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:41.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:41.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:42.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:42.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:42.145 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-04-23T19:05:42.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-04-23T19:05:42.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:42.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:42.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:42.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:42.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:42.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:42.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:42.532 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:42.583 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:42.654 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:42.702 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:42.736 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:42.866 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:42.940 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:43.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:43.057 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:43.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:43.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:43.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:43.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-04-23T19:05:43.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-04-23T19:05:43.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:43.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:43.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:43.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:43.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:43.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:43.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:43.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:43.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:43.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:43.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:43.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:43.986 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-04-23T19:05:43.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-04-23T19:05:44.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:44.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:44.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:44.112 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-04-23T19:05:44.227 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:44.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:44.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-04-23T19:05:44.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-04-23T19:05:44.372 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-04-23T19:05:44.453 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-04-23T19:05:44.489 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-04-23T19:05:44.605 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-04-23T19:05:44.614 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-04-23T19:05:44.664 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:44.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:44.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:44.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:44.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:44.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:44.970 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-04-23T19:05:44.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-04-23T19:05:44.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-04-23T19:05:44.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:45.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-04-23T19:05:45.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-04-23T19:05:45.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:45.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:45.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:45.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:45.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:45.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:45.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:45.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:45.676 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-04-23T19:05:45.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-04-23T19:05:45.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:45.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:45.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:45.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-04-23T19:05:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:45.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:45.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:46.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:46.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:46.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:46.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:46.317 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-04-23T19:05:46.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-04-23T19:05:46.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-04-23T19:05:46.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:46.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:46.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-04-23T19:05:46.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:46.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:46.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:46.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:46.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:46.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:46.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:46.931 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-04-23T19:05:47.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-04-23T19:05:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-04-23T19:05:47.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-04-23T19:05:47.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-04-23T19:05:47.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-23T19:05:47.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-04-23T19:05:47.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-04-23T19:05:47.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-04-23T19:05:47.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-04-23T19:05:47.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-04-23T19:05:47.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-04-23T19:05:47.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-04-23T19:05:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:47.808 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:47.920 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:47.995 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:48.068 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-04-23T19:05:48.219 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-04-23T19:05:48.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:48.324 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-04-23T19:05:48.404 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-04-23T19:05:48.485 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-23T19:05:48.522 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-23T19:05:48.565 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-23T19:05:48.656 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-23T19:05:49.530 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-23T19:05:49.587 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-23T19:05:49.596 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-23T19:05:49.604 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-23T19:05:49.613 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-23T19:05:49.676 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-23T19:05:49.698 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:49.707 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-04-23T19:05:49.773 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-23T19:05:49.781 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-23T19:05:49.790 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-23T19:05:49.799 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-23T19:05:49.808 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-23T19:05:49.816 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-23T19:05:49.825 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-23T19:05:49.895 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-23T19:05:49.925 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-23T19:05:50.032 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-23T19:05:50.110 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:50.125 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-04-23T19:05:50.133 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-23T19:05:50.140 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-23T19:05:50.214 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-23T19:05:50.260 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-23T19:05:50.296 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-23T19:05:50.401 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-23T19:05:50.435 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-23T19:05:50.469 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-23T19:05:50.479 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-23T19:05:50.553 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-23T19:05:50.597 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:50.645 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-04-23T19:05:50.689 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-23T19:05:50.752 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-23T19:05:50.818 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-23T19:05:50.855 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-23T19:05:50.917 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-23T19:05:50.975 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-23T19:05:51.049 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-23T19:05:51.114 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-23T19:05:51.143 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-23T19:05:51.230 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-23T19:05:51.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-23T19:05:51.386 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-04-23T19:05:51.394 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-04-23T19:05:51.432 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-23T19:05:51.460 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-04-23T19:05:51.531 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-04-23T19:05:51.615 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-04-23T19:05:51.700 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-04-23T19:05:51.712 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-04-23T19:05:51.758 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-04-23T19:05:51.796 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-04-23T19:05:51.843 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-04-23T19:05:51.894 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:51.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:52.010 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 88, 2024-04-23T19:05:52.053 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-04-23T19:05:52.100 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:52.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-04-23T19:05:52.210 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:52.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:52.410 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:52.429 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4053364918 2024-04-23T19:05:52.437 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:52.475 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:52.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:52.680 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:52.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-04-23T19:05:52.740 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:52.808 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:52.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:52.929 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:52.998 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2643760133 2024-04-23T19:05:53.007 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:53.067 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:53.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:53.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:53.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-04-23T19:05:53.324 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:53.333 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:53.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:53.498 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:53.566 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1079959461 2024-04-23T19:05:53.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:53.693 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:53.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:53.815 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:53.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-23T19:05:53.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:54.016 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:54.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:54.143 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:54.218 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3498087314 2024-04-23T19:05:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:54.343 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:54.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:54.461 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:54.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-04-23T19:05:54.567 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:54.639 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:54.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:54.756 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:54.792 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 182809329 2024-04-23T19:05:54.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:54.901 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:55.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:05:55.025 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:55.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-04-23T19:05:55.146 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-04-23T19:05:55.192 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-23T19:05:55.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-04-23T19:05:55.297 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-04-23T19:05:55.342 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3637122823 2024-04-23T19:05:55.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:55.470 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:55.479 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:05:55.488 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-23T19:05:55.532 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T19:05:55.624 DEBUG:tasks.check_counter:Getting stats from a 2024-04-23T19:05:55.660 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-23T19:05:55.812 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2118716360' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T19:05:55.841 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:24 smithi122 ceph-mon[32546]: pgmap v1041: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T19:05:55.898 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:26 smithi122 ceph-mon[32546]: pgmap v1042: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T19:05:55.952 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:28 smithi122 ceph-mon[32546]: pgmap v1043: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-23T19:05:56.012 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:30 smithi122 ceph-mon[32546]: pgmap v1044: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T19:05:56.113 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:32 smithi122 ceph-mon[32546]: pgmap v1045: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-23T19:05:56.165 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:34 smithi122 ceph-mon[32546]: pgmap v1046: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:36 smithi122 ceph-mon[32546]: pgmap v1047: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.243 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:38 smithi122 ceph-mon[32546]: pgmap v1048: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:56.333 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:40 smithi122 ceph-mon[32546]: pgmap v1049: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.376 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:42 smithi122 ceph-mon[32546]: pgmap v1050: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:56.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:44 smithi122 ceph-mon[32546]: pgmap v1051: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.505 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:46 smithi122 ceph-mon[32546]: pgmap v1052: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.514 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:48 smithi122 ceph-mon[32546]: pgmap v1053: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:56.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:50 smithi122 ceph-mon[32546]: pgmap v1054: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.698 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:52 smithi122 ceph-mon[32546]: pgmap v1055: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:56.759 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:54 smithi122 ceph-mon[32546]: pgmap v1056: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:56 smithi122 ceph-mon[32546]: pgmap v1057: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.818 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:58:58 smithi122 ceph-mon[32546]: pgmap v1058: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:56.895 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:00 smithi122 ceph-mon[32546]: pgmap v1059: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:56.957 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:02 smithi122 ceph-mon[32546]: pgmap v1060: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:05:57.040 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:04 smithi122 ceph-mon[32546]: pgmap v1061: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:05:57.154 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:06 smithi122 ceph-mon[32546]: pgmap v1062: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:05:57.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:08 smithi122 ceph-mon[32546]: pgmap v1063: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:05:57.259 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:09 smithi122 ceph-mon[32546]: osdmap e154: 12 total, 12 up, 12 in 2024-04-23T19:05:57.368 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:10 smithi122 ceph-mon[32546]: pgmap v1065: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:05:57.447 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:12 smithi122 ceph-mon[32546]: pgmap v1066: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:05:57.478 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:14 smithi122 ceph-mon[32546]: pgmap v1067: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:05:57.535 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:16 smithi122 ceph-mon[32546]: pgmap v1068: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:05:57.592 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:16 smithi122 ceph-mon[32546]: osdmap e155: 12 total, 12 up, 12 in 2024-04-23T19:05:57.637 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:17 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:05:57.711 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: pgmap v1070: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-04-23T19:05:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:57.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:57.763 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:57.880 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:05:57.982 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:05:58.034 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:18 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:58.131 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:19 smithi122 ceph-mon[32546]: pgmap v1071: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 220 B/s wr, 11 op/s 2024-04-23T19:05:58.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:19 smithi122 ceph-mon[32546]: pgmap v1072: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:05:58.246 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:21 smithi122 ceph-mon[32546]: pgmap v1073: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:05:58.304 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:23 smithi122 ceph-mon[32546]: pgmap v1074: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 140 B/s wr, 13 op/s 2024-04-23T19:05:58.373 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:25 smithi122 ceph-mon[32546]: pgmap v1075: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:05:58.412 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:27 smithi122 ceph-mon[32546]: pgmap v1076: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T19:05:58.421 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:29 smithi122 ceph-mon[32546]: pgmap v1077: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:05:58.435 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:31 smithi122 ceph-mon[32546]: pgmap v1078: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.445 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:33 smithi122 ceph-mon[32546]: pgmap v1079: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:58.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:34 smithi122 ceph-mon[32546]: pgmap v1080: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.498 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:37 smithi122 ceph-mon[32546]: pgmap v1081: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:58.543 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:39 smithi122 ceph-mon[32546]: pgmap v1082: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.584 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:41 smithi122 ceph-mon[32546]: pgmap v1083: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.648 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:43 smithi122 ceph-mon[32546]: pgmap v1084: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:58.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:45 smithi122 ceph-mon[32546]: pgmap v1085: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.789 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:47 smithi122 ceph-mon[32546]: pgmap v1086: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:58.799 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:49 smithi122 ceph-mon[32546]: pgmap v1087: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.869 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:51 smithi122 ceph-mon[32546]: pgmap v1088: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:58.935 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:53 smithi122 ceph-mon[32546]: pgmap v1089: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:59.038 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:55 smithi122 ceph-mon[32546]: pgmap v1090: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:59.112 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:57 smithi122 ceph-mon[32546]: pgmap v1091: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:05:59.120 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 18:59:59 smithi122 ceph-mon[32546]: pgmap v1092: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:59.127 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:00 smithi122 ceph-mon[32546]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-04-23T19:05:59.134 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:00 smithi122 ceph-mon[32546]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-04-23T19:05:59.143 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:00 smithi122 ceph-mon[32546]: daemon mds.d on smithi033 is in error state 2024-04-23T19:05:59.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:00 smithi122 ceph-mon[32546]: daemon mds.f on smithi122 is in error state 2024-04-23T19:05:59.175 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:01 smithi122 ceph-mon[32546]: pgmap v1093: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:05:59.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:03 smithi122 ceph-mon[32546]: pgmap v1094: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:05:59.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:05 smithi122 ceph-mon[32546]: pgmap v1095: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:05:59.270 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:07 smithi122 ceph-mon[32546]: pgmap v1096: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:05:59.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:09 smithi122 ceph-mon[32546]: pgmap v1097: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:05:59.377 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:09 smithi122 ceph-mon[32546]: osdmap e156: 12 total, 12 up, 12 in 2024-04-23T19:05:59.421 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:11 smithi122 ceph-mon[32546]: pgmap v1099: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:05:59.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:13 smithi122 ceph-mon[32546]: pgmap v1100: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:05:59.537 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:15 smithi122 ceph-mon[32546]: pgmap v1101: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:05:59.599 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:17 smithi122 ceph-mon[32546]: osdmap e157: 12 total, 12 up, 12 in 2024-04-23T19:05:59.656 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:17 smithi122 ceph-mon[32546]: pgmap v1103: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T19:05:59.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:19 smithi122 ceph-mon[32546]: pgmap v1104: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T19:05:59.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:19 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:05:59.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:19 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:05:59.779 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:19 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:05:59.787 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:19 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:05:59.796 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:20 smithi122 ceph-mon[32546]: pgmap v1105: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-04-23T19:05:59.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:20 smithi122 ceph-mon[32546]: pgmap v1106: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:05:59.919 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:22 smithi122 ceph-mon[32546]: pgmap v1107: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 150 B/s wr, 14 op/s 2024-04-23T19:05:59.992 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:24 smithi122 ceph-mon[32546]: pgmap v1108: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T19:06:00.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:26 smithi122 ceph-mon[32546]: pgmap v1109: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T19:06:00.093 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:28 smithi122 ceph-mon[32546]: pgmap v1110: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:06:00.156 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:30 smithi122 ceph-mon[32546]: pgmap v1111: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-23T19:06:00.186 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:32 smithi122 ceph-mon[32546]: pgmap v1112: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-23T19:06:00.251 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:34 smithi122 ceph-mon[32546]: pgmap v1113: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.329 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:36 smithi122 ceph-mon[32546]: pgmap v1114: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.388 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:38 smithi122 ceph-mon[32546]: pgmap v1115: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:00.427 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:40 smithi122 ceph-mon[32546]: pgmap v1116: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.436 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:42 smithi122 ceph-mon[32546]: pgmap v1117: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:00.444 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:44 smithi122 ceph-mon[32546]: pgmap v1118: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.471 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:46 smithi122 ceph-mon[32546]: pgmap v1119: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.504 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:48 smithi122 ceph-mon[32546]: pgmap v1120: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:00.567 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:50 smithi122 ceph-mon[32546]: pgmap v1121: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.652 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:52 smithi122 ceph-mon[32546]: pgmap v1122: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:00.731 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:54 smithi122 ceph-mon[32546]: pgmap v1123: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:56 smithi122 ceph-mon[32546]: pgmap v1124: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:00:58 smithi122 ceph-mon[32546]: pgmap v1125: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:00.759 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:00 smithi122 ceph-mon[32546]: pgmap v1126: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:00.768 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:02 smithi122 ceph-mon[32546]: pgmap v1127: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:00.842 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:04 smithi122 ceph-mon[32546]: pgmap v1128: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:00.906 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:06 smithi122 ceph-mon[32546]: pgmap v1129: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:00.960 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:08 smithi122 ceph-mon[32546]: pgmap v1130: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:01.041 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:09 smithi122 ceph-mon[32546]: osdmap e158: 12 total, 12 up, 12 in 2024-04-23T19:06:01.093 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:10 smithi122 ceph-mon[32546]: pgmap v1132: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:01.119 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:12 smithi122 ceph-mon[32546]: pgmap v1133: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:01.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:14 smithi122 ceph-mon[32546]: pgmap v1134: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:01.234 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:16 smithi122 ceph-mon[32546]: pgmap v1135: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:01.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:16 smithi122 ceph-mon[32546]: osdmap e159: 12 total, 12 up, 12 in 2024-04-23T19:06:01.376 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:18 smithi122 ceph-mon[32546]: pgmap v1137: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:01.383 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:19 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:01.391 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:20 smithi122 ceph-mon[32546]: pgmap v1138: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:01.449 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:20 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:01.474 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:20 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:01.558 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:20 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:01.604 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:21 smithi122 ceph-mon[32546]: pgmap v1139: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 116 B/s wr, 5 op/s 2024-04-23T19:06:01.638 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:21 smithi122 ceph-mon[32546]: pgmap v1140: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:06:01.697 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:23 smithi122 ceph-mon[32546]: pgmap v1141: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 150 B/s wr, 15 op/s 2024-04-23T19:06:01.703 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:25 smithi122 ceph-mon[32546]: pgmap v1142: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:06:01.711 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:27 smithi122 ceph-mon[32546]: pgmap v1143: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:01.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:29 smithi122 ceph-mon[32546]: pgmap v1144: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:01.871 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:31 smithi122 ceph-mon[32546]: pgmap v1145: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:01.959 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:33 smithi122 ceph-mon[32546]: pgmap v1146: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.146 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:35 smithi122 ceph-mon[32546]: pgmap v1147: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.255 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:37 smithi122 ceph-mon[32546]: pgmap v1148: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.283 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:39 smithi122 ceph-mon[32546]: pgmap v1149: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.292 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:41 smithi122 ceph-mon[32546]: pgmap v1150: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.301 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:43 smithi122 ceph-mon[32546]: pgmap v1151: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:45 smithi122 ceph-mon[32546]: pgmap v1152: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.377 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:47 smithi122 ceph-mon[32546]: pgmap v1153: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.437 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:49 smithi122 ceph-mon[32546]: pgmap v1154: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.493 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:51 smithi122 ceph-mon[32546]: pgmap v1155: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.583 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:53 smithi122 ceph-mon[32546]: pgmap v1156: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.613 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:55 smithi122 ceph-mon[32546]: pgmap v1157: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.685 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:57 smithi122 ceph-mon[32546]: pgmap v1158: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:02.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:01:59 smithi122 ceph-mon[32546]: pgmap v1159: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.784 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:01 smithi122 ceph-mon[32546]: pgmap v1160: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:02.941 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:03 smithi122 ceph-mon[32546]: pgmap v1161: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:02.975 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:05 smithi122 ceph-mon[32546]: pgmap v1162: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:02.984 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:07 smithi122 ceph-mon[32546]: pgmap v1163: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:02.993 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:09 smithi122 ceph-mon[32546]: pgmap v1164: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:03.002 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:09 smithi122 ceph-mon[32546]: osdmap e160: 12 total, 12 up, 12 in 2024-04-23T19:06:03.012 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:11 smithi122 ceph-mon[32546]: pgmap v1166: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:03.052 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:13 smithi122 ceph-mon[32546]: pgmap v1167: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:03.092 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:15 smithi122 ceph-mon[32546]: pgmap v1168: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:03.135 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:17 smithi122 ceph-mon[32546]: pgmap v1169: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:03.226 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:17 smithi122 ceph-mon[32546]: osdmap e161: 12 total, 12 up, 12 in 2024-04-23T19:06:03.271 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:19 smithi122 ceph-mon[32546]: pgmap v1171: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-04-23T19:06:03.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:20 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:03.317 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:21 smithi122 ceph-mon[32546]: pgmap v1172: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:03.326 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:21 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:03.415 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:21 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:03.487 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:21 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:03.524 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:22 smithi122 ceph-mon[32546]: pgmap v1173: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:03.586 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:22 smithi122 ceph-mon[32546]: pgmap v1174: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:03.655 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:24 smithi122 ceph-mon[32546]: pgmap v1175: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:03.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:26 smithi122 ceph-mon[32546]: pgmap v1176: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:03.755 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:28 smithi122 ceph-mon[32546]: pgmap v1177: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:03.800 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:30 smithi122 ceph-mon[32546]: pgmap v1178: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:03.855 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:32 smithi122 ceph-mon[32546]: pgmap v1179: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:03.919 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:34 smithi122 ceph-mon[32546]: pgmap v1180: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:03.951 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:36 smithi122 ceph-mon[32546]: pgmap v1181: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.005 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:38 smithi122 ceph-mon[32546]: pgmap v1182: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:04.039 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:40 smithi122 ceph-mon[32546]: pgmap v1183: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.048 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:42 smithi122 ceph-mon[32546]: pgmap v1184: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:04.058 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:44 smithi122 ceph-mon[32546]: pgmap v1185: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.067 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:46 smithi122 ceph-mon[32546]: pgmap v1186: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.077 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:48 smithi122 ceph-mon[32546]: pgmap v1187: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:04.199 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:50 smithi122 ceph-mon[32546]: pgmap v1188: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.227 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:52 smithi122 ceph-mon[32546]: pgmap v1189: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:04.269 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:54 smithi122 ceph-mon[32546]: pgmap v1190: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.344 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:56 smithi122 ceph-mon[32546]: pgmap v1191: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.386 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:02:58 smithi122 ceph-mon[32546]: pgmap v1192: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:04.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:00 smithi122 ceph-mon[32546]: pgmap v1193: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:04.477 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:02 smithi122 ceph-mon[32546]: pgmap v1194: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:04.486 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:04 smithi122 ceph-mon[32546]: pgmap v1195: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:04.495 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:06 smithi122 ceph-mon[32546]: pgmap v1196: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:04.549 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:08 smithi122 ceph-mon[32546]: pgmap v1197: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:04.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:09 smithi122 ceph-mon[32546]: osdmap e162: 12 total, 12 up, 12 in 2024-04-23T19:06:04.617 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:10 smithi122 ceph-mon[32546]: pgmap v1199: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:04.687 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:12 smithi122 ceph-mon[32546]: pgmap v1200: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:04.714 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:14 smithi122 ceph-mon[32546]: pgmap v1201: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:04.777 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:16 smithi122 ceph-mon[32546]: pgmap v1202: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:04.821 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:16 smithi122 ceph-mon[32546]: osdmap e163: 12 total, 12 up, 12 in 2024-04-23T19:06:04.871 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:18 smithi122 ceph-mon[32546]: pgmap v1204: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-04-23T19:06:04.932 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:20 smithi122 ceph-mon[32546]: pgmap v1205: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:04.978 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:21 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:05.018 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:22 smithi122 ceph-mon[32546]: pgmap v1206: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:05.102 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:22 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:05.145 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:22 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:05.196 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:22 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:05.243 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:23 smithi122 ceph-mon[32546]: pgmap v1207: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:05.255 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:23 smithi122 ceph-mon[32546]: pgmap v1208: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:05.263 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:25 smithi122 ceph-mon[32546]: pgmap v1209: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T19:06:05.271 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:27 smithi122 ceph-mon[32546]: pgmap v1210: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-23T19:06:05.295 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:29 smithi122 ceph-mon[32546]: pgmap v1211: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:05.398 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:31 smithi122 ceph-mon[32546]: pgmap v1212: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:05.472 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:33 smithi122 ceph-mon[32546]: pgmap v1213: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:05.532 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:35 smithi122 ceph-mon[32546]: pgmap v1214: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.575 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:37 smithi122 ceph-mon[32546]: pgmap v1215: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:05.582 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:39 smithi122 ceph-mon[32546]: pgmap v1216: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.590 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:41 smithi122 ceph-mon[32546]: pgmap v1217: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.665 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:43 smithi122 ceph-mon[32546]: pgmap v1218: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:05.765 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:45 smithi122 ceph-mon[32546]: pgmap v1219: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.819 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:47 smithi122 ceph-mon[32546]: pgmap v1220: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:05.885 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:49 smithi122 ceph-mon[32546]: pgmap v1221: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:51 smithi122 ceph-mon[32546]: pgmap v1222: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:05.936 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:53 smithi122 ceph-mon[32546]: pgmap v1223: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:05.988 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:55 smithi122 ceph-mon[32546]: pgmap v1224: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:06.063 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:57 smithi122 ceph-mon[32546]: pgmap v1225: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:06.123 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:03:59 smithi122 ceph-mon[32546]: pgmap v1226: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:06.158 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:01 smithi122 ceph-mon[32546]: pgmap v1227: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:06.177 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:03 smithi122 ceph-mon[32546]: pgmap v1228: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T19:06:06.185 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:05 smithi122 ceph-mon[32546]: pgmap v1229: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T19:06:06.304 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:07 smithi122 ceph-mon[32546]: pgmap v1230: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:06.342 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:09 smithi122 ceph-mon[32546]: pgmap v1231: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T19:06:06.392 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:09 smithi122 ceph-mon[32546]: osdmap e164: 12 total, 12 up, 12 in 2024-04-23T19:06:06.440 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:11 smithi122 ceph-mon[32546]: pgmap v1233: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:06.475 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:13 smithi122 ceph-mon[32546]: pgmap v1234: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:06.551 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:15 smithi122 ceph-mon[32546]: pgmap v1235: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:06.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:17 smithi122 ceph-mon[32546]: pgmap v1236: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:06.652 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:17 smithi122 ceph-mon[32546]: osdmap e165: 12 total, 12 up, 12 in 2024-04-23T19:06:06.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:19 smithi122 ceph-mon[32546]: pgmap v1238: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-04-23T19:06:06.811 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:21 smithi122 ceph-mon[32546]: pgmap v1239: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:06.857 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:22 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:06.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: pgmap v1240: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:06.999 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:07.081 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:07.132 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: pgmap v1241: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 115 B/s wr, 11 op/s 2024-04-23T19:06:07.189 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: pgmap v1242: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:07.257 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:07.286 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:25 smithi122 ceph-mon[32546]: pgmap v1243: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:06:07.338 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:27 smithi122 ceph-mon[32546]: pgmap v1244: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:07.407 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:29 smithi122 ceph-mon[32546]: pgmap v1245: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:07.457 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:31 smithi122 ceph-mon[32546]: pgmap v1246: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:07.540 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:33 smithi122 ceph-mon[32546]: pgmap v1247: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:07.596 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:35 smithi122 ceph-mon[32546]: pgmap v1248: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:07.614 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:37 smithi122 ceph-mon[32546]: pgmap v1249: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:07.700 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:39 smithi122 ceph-mon[32546]: pgmap v1250: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:07.780 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:41 smithi122 ceph-mon[32546]: pgmap v1251: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:07.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:43 smithi122 ceph-mon[32546]: pgmap v1252: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:07.900 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:45 smithi122 ceph-mon[32546]: pgmap v1253: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:07.964 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:47 smithi122 ceph-mon[32546]: pgmap v1254: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:08.030 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:49 smithi122 ceph-mon[32546]: pgmap v1255: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:51 smithi122 ceph-mon[32546]: pgmap v1256: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.132 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:53 smithi122 ceph-mon[32546]: pgmap v1257: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:08.170 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:55 smithi122 ceph-mon[32546]: pgmap v1258: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.226 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:57 smithi122 ceph-mon[32546]: pgmap v1259: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:08.275 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:04:59 smithi122 ceph-mon[32546]: pgmap v1260: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.284 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:01 smithi122 ceph-mon[32546]: pgmap v1261: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.293 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:03 smithi122 ceph-mon[32546]: pgmap v1262: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:08.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:05 smithi122 ceph-mon[32546]: pgmap v1263: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:08.310 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:07 smithi122 ceph-mon[32546]: pgmap v1264: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:08.405 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:09 smithi122 ceph-mon[32546]: pgmap v1265: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:08.448 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:11 smithi122 ceph-mon[32546]: pgmap v1266: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:08.491 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:13 smithi122 ceph-mon[32546]: pgmap v1267: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:08.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:15 smithi122 ceph-mon[32546]: pgmap v1268: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:08.625 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:17 smithi122 ceph-mon[32546]: pgmap v1269: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:08.712 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:19 smithi122 ceph-mon[32546]: pgmap v1270: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:08.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:21 smithi122 ceph-mon[32546]: pgmap v1271: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:08.863 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:22 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:08.940 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:23 smithi122 ceph-mon[32546]: pgmap v1272: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T19:06:08.962 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:08.971 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:08.993 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:09.049 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:24 smithi122 ceph-mon[32546]: pgmap v1273: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T19:06:09.125 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:24 smithi122 ceph-mon[32546]: pgmap v1274: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:09.231 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:25 smithi122 ceph-mon[32546]: pgmap v1275: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:09.289 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:28 smithi122 ceph-mon[32546]: pgmap v1276: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:09.304 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:30 smithi122 ceph-mon[32546]: pgmap v1277: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:09.362 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:32 smithi122 ceph-mon[32546]: pgmap v1278: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:09.441 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:34 smithi122 ceph-mon[32546]: pgmap v1279: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:09.498 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:36 smithi122 ceph-mon[32546]: pgmap v1280: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:09.633 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:38 smithi122 ceph-mon[32546]: pgmap v1281: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:09.715 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:40 smithi122 ceph-mon[32546]: pgmap v1282: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:09.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:42 smithi122 ceph-mon[32546]: pgmap v1283: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:09.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:44 smithi122 ceph-mon[32546]: pgmap v1284: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:09.752 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:46 smithi122 ceph-mon[32546]: pgmap v1285: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:09.762 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:48 smithi122 ceph-mon[32546]: pgmap v1286: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:09.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:50 smithi122 ceph-mon[32546]: pgmap v1287: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:09.797 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:52 smithi122 ceph-mon[32546]: pgmap v1288: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:09.887 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:54 smithi122 ceph-mon[32546]: pgmap v1289: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.052 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2118716360' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T19:06:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:24 smithi106 ceph-mon[32716]: pgmap v1041: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T19:06:10.123 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:26 smithi106 ceph-mon[32716]: pgmap v1042: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T19:06:10.212 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:28 smithi106 ceph-mon[32716]: pgmap v1043: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-23T19:06:10.274 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:30 smithi106 ceph-mon[32716]: pgmap v1044: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T19:06:10.436 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:32 smithi106 ceph-mon[32716]: pgmap v1045: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-23T19:06:10.468 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:34 smithi106 ceph-mon[32716]: pgmap v1046: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.511 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:36 smithi106 ceph-mon[32716]: pgmap v1047: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:38 smithi106 ceph-mon[32716]: pgmap v1048: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:10.687 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:40 smithi106 ceph-mon[32716]: pgmap v1049: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.765 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:42 smithi106 ceph-mon[32716]: pgmap v1050: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:10.787 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:44 smithi106 ceph-mon[32716]: pgmap v1051: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:46 smithi106 ceph-mon[32716]: pgmap v1052: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.871 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:48 smithi106 ceph-mon[32716]: pgmap v1053: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:10.906 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:50 smithi106 ceph-mon[32716]: pgmap v1054: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:10.963 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:52 smithi106 ceph-mon[32716]: pgmap v1055: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:11.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:54 smithi106 ceph-mon[32716]: pgmap v1056: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:11.092 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:56 smithi106 ceph-mon[32716]: pgmap v1057: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:11.102 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:58:58 smithi106 ceph-mon[32716]: pgmap v1058: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:11.111 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:00 smithi106 ceph-mon[32716]: pgmap v1059: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:11.119 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:02 smithi106 ceph-mon[32716]: pgmap v1060: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:11.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:04 smithi106 ceph-mon[32716]: pgmap v1061: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:11.198 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:06 smithi106 ceph-mon[32716]: pgmap v1062: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:11.310 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:08 smithi106 ceph-mon[32716]: pgmap v1063: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:11.353 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:09 smithi106 ceph-mon[32716]: osdmap e154: 12 total, 12 up, 12 in 2024-04-23T19:06:11.430 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:10 smithi106 ceph-mon[32716]: pgmap v1065: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:11.440 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:12 smithi106 ceph-mon[32716]: pgmap v1066: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:11.449 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:14 smithi106 ceph-mon[32716]: pgmap v1067: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:11.457 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:16 smithi106 ceph-mon[32716]: pgmap v1068: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:11.468 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:16 smithi106 ceph-mon[32716]: osdmap e155: 12 total, 12 up, 12 in 2024-04-23T19:06:11.494 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:17 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:11.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: pgmap v1070: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-04-23T19:06:11.592 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:11.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:11.703 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:11.739 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:11.805 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:11.814 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:11.824 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:18 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:11.886 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:19 smithi106 ceph-mon[32716]: pgmap v1071: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 220 B/s wr, 11 op/s 2024-04-23T19:06:11.963 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:19 smithi106 ceph-mon[32716]: pgmap v1072: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:06:12.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:21 smithi106 ceph-mon[32716]: pgmap v1073: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:06:12.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:23 smithi106 ceph-mon[32716]: pgmap v1074: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 140 B/s wr, 13 op/s 2024-04-23T19:06:12.237 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:25 smithi106 ceph-mon[32716]: pgmap v1075: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:06:12.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:27 smithi106 ceph-mon[32716]: pgmap v1076: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T19:06:12.321 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:29 smithi106 ceph-mon[32716]: pgmap v1077: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:12.396 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:31 smithi106 ceph-mon[32716]: pgmap v1078: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.431 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:33 smithi106 ceph-mon[32716]: pgmap v1079: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:12.482 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:34 smithi106 ceph-mon[32716]: pgmap v1080: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.565 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:37 smithi106 ceph-mon[32716]: pgmap v1081: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:12.574 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:39 smithi106 ceph-mon[32716]: pgmap v1082: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.630 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:41 smithi106 ceph-mon[32716]: pgmap v1083: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.639 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:43 smithi106 ceph-mon[32716]: pgmap v1084: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:12.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:45 smithi106 ceph-mon[32716]: pgmap v1085: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.680 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:47 smithi106 ceph-mon[32716]: pgmap v1086: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:12.749 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:49 smithi106 ceph-mon[32716]: pgmap v1087: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.820 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:51 smithi106 ceph-mon[32716]: pgmap v1088: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:12.908 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:53 smithi106 ceph-mon[32716]: pgmap v1089: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:12.976 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:55 smithi106 ceph-mon[32716]: pgmap v1090: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:13.051 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:57 smithi106 ceph-mon[32716]: pgmap v1091: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:13.082 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 18:59:59 smithi106 ceph-mon[32716]: pgmap v1092: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:13.132 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:00 smithi106 ceph-mon[32716]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-04-23T19:06:13.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:00 smithi106 ceph-mon[32716]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-04-23T19:06:13.246 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:00 smithi106 ceph-mon[32716]: daemon mds.d on smithi033 is in error state 2024-04-23T19:06:13.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:00 smithi106 ceph-mon[32716]: daemon mds.f on smithi122 is in error state 2024-04-23T19:06:13.384 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:01 smithi106 ceph-mon[32716]: pgmap v1093: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:13.413 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:03 smithi106 ceph-mon[32716]: pgmap v1094: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:13.422 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:05 smithi106 ceph-mon[32716]: pgmap v1095: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:13.470 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:07 smithi106 ceph-mon[32716]: pgmap v1096: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:13.577 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:09 smithi106 ceph-mon[32716]: pgmap v1097: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:13.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:09 smithi106 ceph-mon[32716]: osdmap e156: 12 total, 12 up, 12 in 2024-04-23T19:06:13.645 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:11 smithi106 ceph-mon[32716]: pgmap v1099: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:13.704 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:13 smithi106 ceph-mon[32716]: pgmap v1100: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:13.780 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:15 smithi106 ceph-mon[32716]: pgmap v1101: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:13.800 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:17 smithi106 ceph-mon[32716]: osdmap e157: 12 total, 12 up, 12 in 2024-04-23T19:06:13.841 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:17 smithi106 ceph-mon[32716]: pgmap v1103: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T19:06:13.934 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:19 smithi106 ceph-mon[32716]: pgmap v1104: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T19:06:13.979 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:19 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:14.031 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:19 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:14.099 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:19 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:14.163 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:19 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:14.208 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:20 smithi106 ceph-mon[32716]: pgmap v1105: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-04-23T19:06:14.257 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:20 smithi106 ceph-mon[32716]: pgmap v1106: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:06:14.335 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:22 smithi106 ceph-mon[32716]: pgmap v1107: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 150 B/s wr, 14 op/s 2024-04-23T19:06:14.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:24 smithi106 ceph-mon[32716]: pgmap v1108: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T19:06:14.448 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:26 smithi106 ceph-mon[32716]: pgmap v1109: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T19:06:14.457 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:28 smithi106 ceph-mon[32716]: pgmap v1110: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:06:14.467 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:30 smithi106 ceph-mon[32716]: pgmap v1111: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-23T19:06:14.478 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:32 smithi106 ceph-mon[32716]: pgmap v1112: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-23T19:06:14.490 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:34 smithi106 ceph-mon[32716]: pgmap v1113: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:14.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:36 smithi106 ceph-mon[32716]: pgmap v1114: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:14.702 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:38 smithi106 ceph-mon[32716]: pgmap v1115: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:14.734 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:40 smithi106 ceph-mon[32716]: pgmap v1116: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:14.841 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:42 smithi106 ceph-mon[32716]: pgmap v1117: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:14.900 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:44 smithi106 ceph-mon[32716]: pgmap v1118: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:14.946 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:46 smithi106 ceph-mon[32716]: pgmap v1119: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:15.026 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:48 smithi106 ceph-mon[32716]: pgmap v1120: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:15.080 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:50 smithi106 ceph-mon[32716]: pgmap v1121: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:15.179 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:52 smithi106 ceph-mon[32716]: pgmap v1122: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:15.187 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:54 smithi106 ceph-mon[32716]: pgmap v1123: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:15.197 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:56 smithi106 ceph-mon[32716]: pgmap v1124: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:15.208 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:00:58 smithi106 ceph-mon[32716]: pgmap v1125: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:15.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:00 smithi106 ceph-mon[32716]: pgmap v1126: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:15.382 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:02 smithi106 ceph-mon[32716]: pgmap v1127: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:15.455 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:04 smithi106 ceph-mon[32716]: pgmap v1128: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:15.499 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:06 smithi106 ceph-mon[32716]: pgmap v1129: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:15.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:08 smithi106 ceph-mon[32716]: pgmap v1130: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:15.619 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:09 smithi106 ceph-mon[32716]: osdmap e158: 12 total, 12 up, 12 in 2024-04-23T19:06:15.685 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:10 smithi106 ceph-mon[32716]: pgmap v1132: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:15.730 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:12 smithi106 ceph-mon[32716]: pgmap v1133: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:15.851 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:14 smithi106 ceph-mon[32716]: pgmap v1134: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:15.915 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:16 smithi106 ceph-mon[32716]: pgmap v1135: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:15.933 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:16 smithi106 ceph-mon[32716]: osdmap e159: 12 total, 12 up, 12 in 2024-04-23T19:06:15.978 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:18 smithi106 ceph-mon[32716]: pgmap v1137: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:16.068 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:19 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:16.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:20 smithi106 ceph-mon[32716]: pgmap v1138: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:16.178 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:20 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:16.217 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:20 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:16.227 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:20 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:16.317 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:21 smithi106 ceph-mon[32716]: pgmap v1139: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 116 B/s wr, 5 op/s 2024-04-23T19:06:16.386 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:21 smithi106 ceph-mon[32716]: pgmap v1140: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:06:16.443 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:23 smithi106 ceph-mon[32716]: pgmap v1141: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 150 B/s wr, 15 op/s 2024-04-23T19:06:16.488 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:25 smithi106 ceph-mon[32716]: pgmap v1142: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:06:16.497 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:27 smithi106 ceph-mon[32716]: pgmap v1143: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:16.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:29 smithi106 ceph-mon[32716]: pgmap v1144: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:16.578 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:31 smithi106 ceph-mon[32716]: pgmap v1145: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:16.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:33 smithi106 ceph-mon[32716]: pgmap v1146: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:16.754 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:35 smithi106 ceph-mon[32716]: pgmap v1147: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:16.823 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:37 smithi106 ceph-mon[32716]: pgmap v1148: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:16.848 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:39 smithi106 ceph-mon[32716]: pgmap v1149: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:16.875 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:41 smithi106 ceph-mon[32716]: pgmap v1150: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:16.939 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:43 smithi106 ceph-mon[32716]: pgmap v1151: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:16.983 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:45 smithi106 ceph-mon[32716]: pgmap v1152: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:47 smithi106 ceph-mon[32716]: pgmap v1153: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:17.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:49 smithi106 ceph-mon[32716]: pgmap v1154: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.125 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:51 smithi106 ceph-mon[32716]: pgmap v1155: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.199 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:53 smithi106 ceph-mon[32716]: pgmap v1156: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:17.269 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:55 smithi106 ceph-mon[32716]: pgmap v1157: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.410 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:57 smithi106 ceph-mon[32716]: pgmap v1158: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:17.419 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:01:59 smithi106 ceph-mon[32716]: pgmap v1159: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.428 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:01 smithi106 ceph-mon[32716]: pgmap v1160: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:17.486 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:03 smithi106 ceph-mon[32716]: pgmap v1161: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:17.552 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:05 smithi106 ceph-mon[32716]: pgmap v1162: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:17.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:07 smithi106 ceph-mon[32716]: pgmap v1163: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:17.701 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:09 smithi106 ceph-mon[32716]: pgmap v1164: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:17.768 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:09 smithi106 ceph-mon[32716]: osdmap e160: 12 total, 12 up, 12 in 2024-04-23T19:06:17.815 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:11 smithi106 ceph-mon[32716]: pgmap v1166: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:17.831 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:13 smithi106 ceph-mon[32716]: pgmap v1167: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:17.874 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:15 smithi106 ceph-mon[32716]: pgmap v1168: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:17.950 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:17 smithi106 ceph-mon[32716]: pgmap v1169: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:18.034 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:17 smithi106 ceph-mon[32716]: osdmap e161: 12 total, 12 up, 12 in 2024-04-23T19:06:18.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:19 smithi106 ceph-mon[32716]: pgmap v1171: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-04-23T19:06:18.115 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:20 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:18.209 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:21 smithi106 ceph-mon[32716]: pgmap v1172: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:18.246 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:21 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:18.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:21 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:18.336 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:21 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:18.400 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:22 smithi106 ceph-mon[32716]: pgmap v1173: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:18.464 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:22 smithi106 ceph-mon[32716]: pgmap v1174: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:18.474 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:24 smithi106 ceph-mon[32716]: pgmap v1175: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:18.512 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:26 smithi106 ceph-mon[32716]: pgmap v1176: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:18.629 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:28 smithi106 ceph-mon[32716]: pgmap v1177: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:18.745 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:30 smithi106 ceph-mon[32716]: pgmap v1178: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:18.786 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:32 smithi106 ceph-mon[32716]: pgmap v1179: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:18.841 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:34 smithi106 ceph-mon[32716]: pgmap v1180: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:18.866 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:36 smithi106 ceph-mon[32716]: pgmap v1181: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:18.955 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:38 smithi106 ceph-mon[32716]: pgmap v1182: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:19.027 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:40 smithi106 ceph-mon[32716]: pgmap v1183: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.074 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:42 smithi106 ceph-mon[32716]: pgmap v1184: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:19.111 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:44 smithi106 ceph-mon[32716]: pgmap v1185: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.200 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:46 smithi106 ceph-mon[32716]: pgmap v1186: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:48 smithi106 ceph-mon[32716]: pgmap v1187: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:19.277 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:50 smithi106 ceph-mon[32716]: pgmap v1188: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.308 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:52 smithi106 ceph-mon[32716]: pgmap v1189: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:19.364 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:54 smithi106 ceph-mon[32716]: pgmap v1190: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.492 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:56 smithi106 ceph-mon[32716]: pgmap v1191: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.500 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:02:58 smithi106 ceph-mon[32716]: pgmap v1192: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:19.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:00 smithi106 ceph-mon[32716]: pgmap v1193: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:19.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:02 smithi106 ceph-mon[32716]: pgmap v1194: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:19.634 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:04 smithi106 ceph-mon[32716]: pgmap v1195: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:19.669 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:06 smithi106 ceph-mon[32716]: pgmap v1196: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:19.729 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:08 smithi106 ceph-mon[32716]: pgmap v1197: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:19.817 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:09 smithi106 ceph-mon[32716]: osdmap e162: 12 total, 12 up, 12 in 2024-04-23T19:06:19.841 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:10 smithi106 ceph-mon[32716]: pgmap v1199: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:19.850 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:12 smithi106 ceph-mon[32716]: pgmap v1200: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:19.859 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:14 smithi106 ceph-mon[32716]: pgmap v1201: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:19.884 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:16 smithi106 ceph-mon[32716]: pgmap v1202: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:19.942 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:16 smithi106 ceph-mon[32716]: osdmap e163: 12 total, 12 up, 12 in 2024-04-23T19:06:20.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:18 smithi106 ceph-mon[32716]: pgmap v1204: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-04-23T19:06:20.084 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:20 smithi106 ceph-mon[32716]: pgmap v1205: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:20.142 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:21 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:20.209 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:22 smithi106 ceph-mon[32716]: pgmap v1206: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:20.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:22 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:20.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:22 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:20.353 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:22 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:20.479 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:23 smithi106 ceph-mon[32716]: pgmap v1207: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:20.546 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:23 smithi106 ceph-mon[32716]: pgmap v1208: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:20.594 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:25 smithi106 ceph-mon[32716]: pgmap v1209: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T19:06:20.641 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:27 smithi106 ceph-mon[32716]: pgmap v1210: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-23T19:06:20.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:29 smithi106 ceph-mon[32716]: pgmap v1211: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:20.701 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:31 smithi106 ceph-mon[32716]: pgmap v1212: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:20.733 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:33 smithi106 ceph-mon[32716]: pgmap v1213: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:20.791 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:35 smithi106 ceph-mon[32716]: pgmap v1214: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:20.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:37 smithi106 ceph-mon[32716]: pgmap v1215: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:20.888 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:39 smithi106 ceph-mon[32716]: pgmap v1216: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:20.947 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:41 smithi106 ceph-mon[32716]: pgmap v1217: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:20.974 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:43 smithi106 ceph-mon[32716]: pgmap v1218: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:21.022 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:45 smithi106 ceph-mon[32716]: pgmap v1219: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:47 smithi106 ceph-mon[32716]: pgmap v1220: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:21.111 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:49 smithi106 ceph-mon[32716]: pgmap v1221: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:51 smithi106 ceph-mon[32716]: pgmap v1222: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.202 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:53 smithi106 ceph-mon[32716]: pgmap v1223: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:21.301 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:55 smithi106 ceph-mon[32716]: pgmap v1224: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.319 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:57 smithi106 ceph-mon[32716]: pgmap v1225: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:21.355 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:03:59 smithi106 ceph-mon[32716]: pgmap v1226: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.401 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:01 smithi106 ceph-mon[32716]: pgmap v1227: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:21.452 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:03 smithi106 ceph-mon[32716]: pgmap v1228: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T19:06:21.525 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:05 smithi106 ceph-mon[32716]: pgmap v1229: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T19:06:21.574 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:07 smithi106 ceph-mon[32716]: pgmap v1230: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:21.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:09 smithi106 ceph-mon[32716]: pgmap v1231: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T19:06:21.595 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:09 smithi106 ceph-mon[32716]: osdmap e164: 12 total, 12 up, 12 in 2024-04-23T19:06:21.604 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:11 smithi106 ceph-mon[32716]: pgmap v1233: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:21.667 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:13 smithi106 ceph-mon[32716]: pgmap v1234: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:21.708 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:15 smithi106 ceph-mon[32716]: pgmap v1235: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:21.758 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:17 smithi106 ceph-mon[32716]: pgmap v1236: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:21.848 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:17 smithi106 ceph-mon[32716]: osdmap e165: 12 total, 12 up, 12 in 2024-04-23T19:06:21.923 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:19 smithi106 ceph-mon[32716]: pgmap v1238: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-04-23T19:06:21.995 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:21 smithi106 ceph-mon[32716]: pgmap v1239: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:22.034 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:22 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:22.139 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: pgmap v1240: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:22.221 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:22.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:22.344 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: pgmap v1241: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 115 B/s wr, 11 op/s 2024-04-23T19:06:22.361 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: pgmap v1242: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:22.370 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:22.415 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:25 smithi106 ceph-mon[32716]: pgmap v1243: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:06:22.475 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:27 smithi106 ceph-mon[32716]: pgmap v1244: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:22.551 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:29 smithi106 ceph-mon[32716]: pgmap v1245: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:22.612 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:31 smithi106 ceph-mon[32716]: pgmap v1246: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:22.689 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:33 smithi106 ceph-mon[32716]: pgmap v1247: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:22.720 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:35 smithi106 ceph-mon[32716]: pgmap v1248: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:22.784 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:37 smithi106 ceph-mon[32716]: pgmap v1249: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:22.836 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:39 smithi106 ceph-mon[32716]: pgmap v1250: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:22.902 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:41 smithi106 ceph-mon[32716]: pgmap v1251: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:22.982 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:43 smithi106 ceph-mon[32716]: pgmap v1252: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:23.033 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:45 smithi106 ceph-mon[32716]: pgmap v1253: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:47 smithi106 ceph-mon[32716]: pgmap v1254: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:23.207 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:49 smithi106 ceph-mon[32716]: pgmap v1255: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.281 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:51 smithi106 ceph-mon[32716]: pgmap v1256: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:53 smithi106 ceph-mon[32716]: pgmap v1257: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:23.343 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:55 smithi106 ceph-mon[32716]: pgmap v1258: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.351 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:57 smithi106 ceph-mon[32716]: pgmap v1259: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:23.360 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:04:59 smithi106 ceph-mon[32716]: pgmap v1260: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.369 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:01 smithi106 ceph-mon[32716]: pgmap v1261: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.390 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:03 smithi106 ceph-mon[32716]: pgmap v1262: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:23.450 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:05 smithi106 ceph-mon[32716]: pgmap v1263: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:23.547 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:07 smithi106 ceph-mon[32716]: pgmap v1264: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:23.615 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:09 smithi106 ceph-mon[32716]: pgmap v1265: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:23.656 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:11 smithi106 ceph-mon[32716]: pgmap v1266: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:23.700 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:13 smithi106 ceph-mon[32716]: pgmap v1267: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:23.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:15 smithi106 ceph-mon[32716]: pgmap v1268: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:23.798 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:17 smithi106 ceph-mon[32716]: pgmap v1269: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:23.869 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:19 smithi106 ceph-mon[32716]: pgmap v1270: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:24.001 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:21 smithi106 ceph-mon[32716]: pgmap v1271: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:24.048 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:22 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:24.099 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:23 smithi106 ceph-mon[32716]: pgmap v1272: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T19:06:24.153 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:24.232 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:24.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:24.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:24 smithi106 ceph-mon[32716]: pgmap v1273: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T19:06:24.404 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:24 smithi106 ceph-mon[32716]: pgmap v1274: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:24.412 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:25 smithi106 ceph-mon[32716]: pgmap v1275: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:24.421 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:28 smithi106 ceph-mon[32716]: pgmap v1276: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:24.467 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:30 smithi106 ceph-mon[32716]: pgmap v1277: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:24.496 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:32 smithi106 ceph-mon[32716]: pgmap v1278: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:24.567 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:34 smithi106 ceph-mon[32716]: pgmap v1279: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:24.617 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:36 smithi106 ceph-mon[32716]: pgmap v1280: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:24.753 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:38 smithi106 ceph-mon[32716]: pgmap v1281: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:24.815 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:40 smithi106 ceph-mon[32716]: pgmap v1282: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:24.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:42 smithi106 ceph-mon[32716]: pgmap v1283: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:24.951 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:44 smithi106 ceph-mon[32716]: pgmap v1284: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.012 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:46 smithi106 ceph-mon[32716]: pgmap v1285: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:48 smithi106 ceph-mon[32716]: pgmap v1286: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:25.190 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:50 smithi106 ceph-mon[32716]: pgmap v1287: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.270 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:52 smithi106 ceph-mon[32716]: pgmap v1288: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:25.337 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:54 smithi106 ceph-mon[32716]: pgmap v1289: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.388 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:56 smithi106 ceph-mon[32716]: pgmap v1290: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.442 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:05:58 smithi106 ceph-mon[32716]: pgmap v1291: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:25.474 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:00 smithi106 ceph-mon[32716]: pgmap v1292: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:25.916 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:02 smithi106 ceph-mon[32716]: pgmap v1293: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:26.036 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:04 smithi106 ceph-mon[32716]: pgmap v1294: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:26.101 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:06 smithi106 ceph-mon[32716]: pgmap v1295: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:26.157 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:08 smithi106 ceph-mon[32716]: pgmap v1296: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:26.166 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:09 smithi106 ceph-mon[32716]: osdmap e166: 12 total, 12 up, 12 in 2024-04-23T19:06:26.323 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:10 smithi106 ceph-mon[32716]: pgmap v1298: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:26.380 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:12 smithi106 ceph-mon[32716]: pgmap v1299: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:26.442 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:14 smithi106 ceph-mon[32716]: pgmap v1300: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:26.557 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:16 smithi106 ceph-mon[32716]: pgmap v1301: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:26.616 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:16 smithi106 ceph-mon[32716]: osdmap e167: 12 total, 12 up, 12 in 2024-04-23T19:06:26.664 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:18 smithi106 ceph-mon[32716]: pgmap v1303: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:26.691 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:20 smithi106 ceph-mon[32716]: pgmap v1304: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:26.751 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:22 smithi106 ceph-mon[32716]: pgmap v1305: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:26.870 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:23 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:26.900 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:24 smithi106 ceph-mon[32716]: pgmap v1306: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:26.909 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:24 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:26.918 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:24 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:26.926 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:24 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:27.003 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:56 smithi122 ceph-mon[32546]: pgmap v1290: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:27.068 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:05:58 smithi122 ceph-mon[32546]: pgmap v1291: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:27.121 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:00 smithi122 ceph-mon[32546]: pgmap v1292: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:27.210 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:02 smithi122 ceph-mon[32546]: pgmap v1293: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:27.318 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:04 smithi122 ceph-mon[32546]: pgmap v1294: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:27.372 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:06 smithi122 ceph-mon[32546]: pgmap v1295: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:27.434 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:08 smithi122 ceph-mon[32546]: pgmap v1296: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:27.546 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:09 smithi122 ceph-mon[32546]: osdmap e166: 12 total, 12 up, 12 in 2024-04-23T19:06:27.598 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:10 smithi122 ceph-mon[32546]: pgmap v1298: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:27.656 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:12 smithi122 ceph-mon[32546]: pgmap v1299: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:27.705 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:14 smithi122 ceph-mon[32546]: pgmap v1300: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:27.808 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:16 smithi122 ceph-mon[32546]: pgmap v1301: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:27.858 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:16 smithi122 ceph-mon[32546]: osdmap e167: 12 total, 12 up, 12 in 2024-04-23T19:06:27.930 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:18 smithi122 ceph-mon[32546]: pgmap v1303: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:27.969 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:20 smithi122 ceph-mon[32546]: pgmap v1304: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:27.979 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:22 smithi122 ceph-mon[32546]: pgmap v1305: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:28.023 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:23 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:28.102 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:24 smithi122 ceph-mon[32546]: pgmap v1306: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:28.192 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:24 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:28.272 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:24 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:28.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:24 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:28.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2118716360' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-23T19:06:28.371 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:24 smithi033 ceph-mon[29274]: pgmap v1041: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-23T19:06:28.408 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:26 smithi033 ceph-mon[29274]: pgmap v1042: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-23T19:06:28.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:28 smithi033 ceph-mon[29274]: pgmap v1043: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-23T19:06:28.537 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:30 smithi033 ceph-mon[29274]: pgmap v1044: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-23T19:06:28.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:32 smithi033 ceph-mon[29274]: pgmap v1045: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-23T19:06:28.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:34 smithi033 ceph-mon[29274]: pgmap v1046: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:28.714 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:36 smithi033 ceph-mon[29274]: pgmap v1047: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:28.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:38 smithi033 ceph-mon[29274]: pgmap v1048: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:28.792 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:40 smithi033 ceph-mon[29274]: pgmap v1049: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:28.850 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:42 smithi033 ceph-mon[29274]: pgmap v1050: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:28.883 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:44 smithi033 ceph-mon[29274]: pgmap v1051: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:28.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:46 smithi033 ceph-mon[29274]: pgmap v1052: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:29.020 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:48 smithi033 ceph-mon[29274]: pgmap v1053: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:29.053 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:50 smithi033 ceph-mon[29274]: pgmap v1054: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:29.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:52 smithi033 ceph-mon[29274]: pgmap v1055: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:29.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:54 smithi033 ceph-mon[29274]: pgmap v1056: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:29.208 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:56 smithi033 ceph-mon[29274]: pgmap v1057: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:29.251 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:58:58 smithi033 ceph-mon[29274]: pgmap v1058: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:29.298 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:00 smithi033 ceph-mon[29274]: pgmap v1059: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:29.400 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:02 smithi033 ceph-mon[29274]: pgmap v1060: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:29.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:04 smithi033 ceph-mon[29274]: pgmap v1061: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:29.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:06 smithi033 ceph-mon[29274]: pgmap v1062: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:29.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:08 smithi033 ceph-mon[29274]: pgmap v1063: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:29.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:09 smithi033 ceph-mon[29274]: osdmap e154: 12 total, 12 up, 12 in 2024-04-23T19:06:29.629 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:10 smithi033 ceph-mon[29274]: pgmap v1065: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:29.690 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:12 smithi033 ceph-mon[29274]: pgmap v1066: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:29.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:14 smithi033 ceph-mon[29274]: pgmap v1067: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:29.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:16 smithi033 ceph-mon[29274]: pgmap v1068: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:29.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:16 smithi033 ceph-mon[29274]: osdmap e155: 12 total, 12 up, 12 in 2024-04-23T19:06:29.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:17 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:29.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: pgmap v1070: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 237 B/s wr, 12 op/s 2024-04-23T19:06:29.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:29.977 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:30.099 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:30.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:30.225 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:30.335 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:30.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:18 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:30.434 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:19 smithi033 ceph-mon[29274]: pgmap v1071: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 220 B/s wr, 11 op/s 2024-04-23T19:06:30.475 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:19 smithi033 ceph-mon[29274]: pgmap v1072: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:06:30.551 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:21 smithi033 ceph-mon[29274]: pgmap v1073: 97 pgs: 97 active+clean; 15 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 140 B/s wr, 7 op/s 2024-04-23T19:06:30.599 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:23 smithi033 ceph-mon[29274]: pgmap v1074: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 140 B/s wr, 13 op/s 2024-04-23T19:06:30.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:25 smithi033 ceph-mon[29274]: pgmap v1075: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:06:30.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:27 smithi033 ceph-mon[29274]: pgmap v1076: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-23T19:06:30.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:29 smithi033 ceph-mon[29274]: pgmap v1077: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:30.722 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:31 smithi033 ceph-mon[29274]: pgmap v1078: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:30.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:33 smithi033 ceph-mon[29274]: pgmap v1079: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:30.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:34 smithi033 ceph-mon[29274]: pgmap v1080: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:30.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:37 smithi033 ceph-mon[29274]: pgmap v1081: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:30.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:39 smithi033 ceph-mon[29274]: pgmap v1082: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.053 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:41 smithi033 ceph-mon[29274]: pgmap v1083: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.129 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:43 smithi033 ceph-mon[29274]: pgmap v1084: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:31.188 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:45 smithi033 ceph-mon[29274]: pgmap v1085: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:47 smithi033 ceph-mon[29274]: pgmap v1086: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:31.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:49 smithi033 ceph-mon[29274]: pgmap v1087: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.410 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:51 smithi033 ceph-mon[29274]: pgmap v1088: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:53 smithi033 ceph-mon[29274]: pgmap v1089: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:31.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:55 smithi033 ceph-mon[29274]: pgmap v1090: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.436 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:57 smithi033 ceph-mon[29274]: pgmap v1091: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:31.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 18:59:59 smithi033 ceph-mon[29274]: pgmap v1092: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:00 smithi033 ceph-mon[29274]: Health detail: HEALTH_WARN 2 failed cephadm daemon(s) 2024-04-23T19:06:31.539 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:00 smithi033 ceph-mon[29274]: [WRN] CEPHADM_FAILED_DAEMON: 2 failed cephadm daemon(s) 2024-04-23T19:06:31.613 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:00 smithi033 ceph-mon[29274]: daemon mds.d on smithi033 is in error state 2024-04-23T19:06:31.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:00 smithi033 ceph-mon[29274]: daemon mds.f on smithi122 is in error state 2024-04-23T19:06:31.740 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:01 smithi033 ceph-mon[29274]: pgmap v1093: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:31.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:03 smithi033 ceph-mon[29274]: pgmap v1094: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:31.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:05 smithi033 ceph-mon[29274]: pgmap v1095: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:31.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:07 smithi033 ceph-mon[29274]: pgmap v1096: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:31.867 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:09 smithi033 ceph-mon[29274]: pgmap v1097: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:31.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:09 smithi033 ceph-mon[29274]: osdmap e156: 12 total, 12 up, 12 in 2024-04-23T19:06:32.000 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:11 smithi033 ceph-mon[29274]: pgmap v1099: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:32.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:13 smithi033 ceph-mon[29274]: pgmap v1100: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:32.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:15 smithi033 ceph-mon[29274]: pgmap v1101: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:32.157 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:17 smithi033 ceph-mon[29274]: osdmap e157: 12 total, 12 up, 12 in 2024-04-23T19:06:32.226 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:17 smithi033 ceph-mon[29274]: pgmap v1103: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-23T19:06:32.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:19 smithi033 ceph-mon[29274]: pgmap v1104: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 206 B/s wr, 10 op/s 2024-04-23T19:06:32.334 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:19 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:32.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:19 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:32.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:19 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:32.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:19 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:32.616 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:20 smithi033 ceph-mon[29274]: pgmap v1105: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 232 B/s wr, 11 op/s 2024-04-23T19:06:32.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:20 smithi033 ceph-mon[29274]: pgmap v1106: 97 pgs: 97 active+clean; 15 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:06:32.740 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:22 smithi033 ceph-mon[29274]: pgmap v1107: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 150 B/s wr, 14 op/s 2024-04-23T19:06:32.806 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:24 smithi033 ceph-mon[29274]: pgmap v1108: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-23T19:06:32.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:26 smithi033 ceph-mon[29274]: pgmap v1109: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-23T19:06:32.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:28 smithi033 ceph-mon[29274]: pgmap v1110: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:06:32.993 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:30 smithi033 ceph-mon[29274]: pgmap v1111: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-04-23T19:06:33.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:32 smithi033 ceph-mon[29274]: pgmap v1112: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s 2024-04-23T19:06:33.125 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:34 smithi033 ceph-mon[29274]: pgmap v1113: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:36 smithi033 ceph-mon[29274]: pgmap v1114: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.236 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:38 smithi033 ceph-mon[29274]: pgmap v1115: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:33.274 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:40 smithi033 ceph-mon[29274]: pgmap v1116: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.317 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:42 smithi033 ceph-mon[29274]: pgmap v1117: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:33.384 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:44 smithi033 ceph-mon[29274]: pgmap v1118: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:46 smithi033 ceph-mon[29274]: pgmap v1119: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.500 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:48 smithi033 ceph-mon[29274]: pgmap v1120: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:33.509 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:50 smithi033 ceph-mon[29274]: pgmap v1121: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:52 smithi033 ceph-mon[29274]: pgmap v1122: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:33.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:54 smithi033 ceph-mon[29274]: pgmap v1123: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:56 smithi033 ceph-mon[29274]: pgmap v1124: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.736 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:00:58 smithi033 ceph-mon[29274]: pgmap v1125: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:33.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:00 smithi033 ceph-mon[29274]: pgmap v1126: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:33.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:02 smithi033 ceph-mon[29274]: pgmap v1127: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:33.864 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:04 smithi033 ceph-mon[29274]: pgmap v1128: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:34.013 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:06 smithi033 ceph-mon[29274]: pgmap v1129: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:34.100 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:08 smithi033 ceph-mon[29274]: pgmap v1130: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:34.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:09 smithi033 ceph-mon[29274]: osdmap e158: 12 total, 12 up, 12 in 2024-04-23T19:06:34.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:10 smithi033 ceph-mon[29274]: pgmap v1132: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:34.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:12 smithi033 ceph-mon[29274]: pgmap v1133: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:34.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:14 smithi033 ceph-mon[29274]: pgmap v1134: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:34.396 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:16 smithi033 ceph-mon[29274]: pgmap v1135: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:34.500 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:16 smithi033 ceph-mon[29274]: osdmap e159: 12 total, 12 up, 12 in 2024-04-23T19:06:34.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:18 smithi033 ceph-mon[29274]: pgmap v1137: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:34.598 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:19 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:34.661 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:20 smithi033 ceph-mon[29274]: pgmap v1138: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:34.708 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:20 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:34.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:20 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:34.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:20 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:34.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:21 smithi033 ceph-mon[29274]: pgmap v1139: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 116 B/s wr, 5 op/s 2024-04-23T19:06:34.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:21 smithi033 ceph-mon[29274]: pgmap v1140: 97 pgs: 97 active+clean; 15 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 150 B/s wr, 7 op/s 2024-04-23T19:06:35.012 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:23 smithi033 ceph-mon[29274]: pgmap v1141: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 150 B/s wr, 15 op/s 2024-04-23T19:06:35.116 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:25 smithi033 ceph-mon[29274]: pgmap v1142: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:06:35.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:27 smithi033 ceph-mon[29274]: pgmap v1143: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:35.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:29 smithi033 ceph-mon[29274]: pgmap v1144: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:35.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:31 smithi033 ceph-mon[29274]: pgmap v1145: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:35.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:33 smithi033 ceph-mon[29274]: pgmap v1146: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:35.377 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:35 smithi033 ceph-mon[29274]: pgmap v1147: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:37 smithi033 ceph-mon[29274]: pgmap v1148: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:35.501 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:39 smithi033 ceph-mon[29274]: pgmap v1149: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.509 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:41 smithi033 ceph-mon[29274]: pgmap v1150: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.516 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:43 smithi033 ceph-mon[29274]: pgmap v1151: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:35.605 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:45 smithi033 ceph-mon[29274]: pgmap v1152: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.688 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:47 smithi033 ceph-mon[29274]: pgmap v1153: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:35.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:49 smithi033 ceph-mon[29274]: pgmap v1154: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.873 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:51 smithi033 ceph-mon[29274]: pgmap v1155: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:35.956 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:53 smithi033 ceph-mon[29274]: pgmap v1156: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:35.985 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:55 smithi033 ceph-mon[29274]: pgmap v1157: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:36.055 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:57 smithi033 ceph-mon[29274]: pgmap v1158: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:01:59 smithi033 ceph-mon[29274]: pgmap v1159: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:36.226 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:01 smithi033 ceph-mon[29274]: pgmap v1160: 97 pgs: 97 active+clean; 15 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:36.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:03 smithi033 ceph-mon[29274]: pgmap v1161: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:36.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:05 smithi033 ceph-mon[29274]: pgmap v1162: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:36.393 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:07 smithi033 ceph-mon[29274]: pgmap v1163: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:36.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:09 smithi033 ceph-mon[29274]: pgmap v1164: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:36.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:09 smithi033 ceph-mon[29274]: osdmap e160: 12 total, 12 up, 12 in 2024-04-23T19:06:36.597 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:11 smithi033 ceph-mon[29274]: pgmap v1166: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:36.604 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:13 smithi033 ceph-mon[29274]: pgmap v1167: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:36.613 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:15 smithi033 ceph-mon[29274]: pgmap v1168: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-23T19:06:36.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:17 smithi033 ceph-mon[29274]: pgmap v1169: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:36.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:17 smithi033 ceph-mon[29274]: osdmap e161: 12 total, 12 up, 12 in 2024-04-23T19:06:36.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:19 smithi033 ceph-mon[29274]: pgmap v1171: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 214 B/s wr, 10 op/s 2024-04-23T19:06:36.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:20 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:37.022 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:21 smithi033 ceph-mon[29274]: pgmap v1172: 97 pgs: 97 active+clean; 15 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:37.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:21 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:37.192 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:21 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:37.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:21 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:37.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:22 smithi033 ceph-mon[29274]: pgmap v1173: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:37.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:22 smithi033 ceph-mon[29274]: pgmap v1174: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:37.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:24 smithi033 ceph-mon[29274]: pgmap v1175: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:26 smithi033 ceph-mon[29274]: pgmap v1176: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:37.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:28 smithi033 ceph-mon[29274]: pgmap v1177: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:37.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:30 smithi033 ceph-mon[29274]: pgmap v1178: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:37.588 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:32 smithi033 ceph-mon[29274]: pgmap v1179: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:37.595 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:34 smithi033 ceph-mon[29274]: pgmap v1180: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:37.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:36 smithi033 ceph-mon[29274]: pgmap v1181: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:37.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:38 smithi033 ceph-mon[29274]: pgmap v1182: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:37.735 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:40 smithi033 ceph-mon[29274]: pgmap v1183: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:37.783 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:42 smithi033 ceph-mon[29274]: pgmap v1184: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:37.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:44 smithi033 ceph-mon[29274]: pgmap v1185: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:37.958 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:46 smithi033 ceph-mon[29274]: pgmap v1186: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:38.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:48 smithi033 ceph-mon[29274]: pgmap v1187: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:38.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:50 smithi033 ceph-mon[29274]: pgmap v1188: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:38.141 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:52 smithi033 ceph-mon[29274]: pgmap v1189: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:38.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:54 smithi033 ceph-mon[29274]: pgmap v1190: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:38.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:56 smithi033 ceph-mon[29274]: pgmap v1191: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:38.241 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:02:58 smithi033 ceph-mon[29274]: pgmap v1192: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:38.277 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:00 smithi033 ceph-mon[29274]: pgmap v1193: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:38.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:02 smithi033 ceph-mon[29274]: pgmap v1194: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:38.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:04 smithi033 ceph-mon[29274]: pgmap v1195: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:38.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:06 smithi033 ceph-mon[29274]: pgmap v1196: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:06:38.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:08 smithi033 ceph-mon[29274]: pgmap v1197: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:38.586 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:09 smithi033 ceph-mon[29274]: osdmap e162: 12 total, 12 up, 12 in 2024-04-23T19:06:38.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:10 smithi033 ceph-mon[29274]: pgmap v1199: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:38.701 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:12 smithi033 ceph-mon[29274]: pgmap v1200: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:38.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:14 smithi033 ceph-mon[29274]: pgmap v1201: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:38.846 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:16 smithi033 ceph-mon[29274]: pgmap v1202: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:38.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:16 smithi033 ceph-mon[29274]: osdmap e163: 12 total, 12 up, 12 in 2024-04-23T19:06:38.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:18 smithi033 ceph-mon[29274]: pgmap v1204: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 240 B/s wr, 12 op/s 2024-04-23T19:06:39.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:20 smithi033 ceph-mon[29274]: pgmap v1205: 97 pgs: 97 active+clean; 15 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:39.094 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:21 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:39.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:22 smithi033 ceph-mon[29274]: pgmap v1206: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:39.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:22 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:39.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:22 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:39.403 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:22 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:39.473 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:23 smithi033 ceph-mon[29274]: pgmap v1207: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:39.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:23 smithi033 ceph-mon[29274]: pgmap v1208: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 151 B/s wr, 15 op/s 2024-04-23T19:06:39.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:25 smithi033 ceph-mon[29274]: pgmap v1209: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-23T19:06:39.615 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:27 smithi033 ceph-mon[29274]: pgmap v1210: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-23T19:06:39.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:29 smithi033 ceph-mon[29274]: pgmap v1211: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:39.706 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:31 smithi033 ceph-mon[29274]: pgmap v1212: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-23T19:06:39.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:33 smithi033 ceph-mon[29274]: pgmap v1213: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:39.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:35 smithi033 ceph-mon[29274]: pgmap v1214: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:39.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:37 smithi033 ceph-mon[29274]: pgmap v1215: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:39.792 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:39 smithi033 ceph-mon[29274]: pgmap v1216: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:39.802 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:41 smithi033 ceph-mon[29274]: pgmap v1217: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:39.811 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:43 smithi033 ceph-mon[29274]: pgmap v1218: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:39.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:45 smithi033 ceph-mon[29274]: pgmap v1219: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:39.967 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:47 smithi033 ceph-mon[29274]: pgmap v1220: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:40.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:49 smithi033 ceph-mon[29274]: pgmap v1221: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:40.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:51 smithi033 ceph-mon[29274]: pgmap v1222: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:40.131 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:53 smithi033 ceph-mon[29274]: pgmap v1223: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:40.183 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:55 smithi033 ceph-mon[29274]: pgmap v1224: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:40.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:57 smithi033 ceph-mon[29274]: pgmap v1225: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:40.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:03:59 smithi033 ceph-mon[29274]: pgmap v1226: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:40.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:01 smithi033 ceph-mon[29274]: pgmap v1227: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:40.488 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:03 smithi033 ceph-mon[29274]: pgmap v1228: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-23T19:06:40.594 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:05 smithi033 ceph-mon[29274]: pgmap v1229: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-23T19:06:40.653 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:07 smithi033 ceph-mon[29274]: pgmap v1230: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:40.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:09 smithi033 ceph-mon[29274]: pgmap v1231: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 26 op/s 2024-04-23T19:06:40.776 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:09 smithi033 ceph-mon[29274]: osdmap e164: 12 total, 12 up, 12 in 2024-04-23T19:06:40.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:11 smithi033 ceph-mon[29274]: pgmap v1233: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:40.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:13 smithi033 ceph-mon[29274]: pgmap v1234: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:40.971 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:15 smithi033 ceph-mon[29274]: pgmap v1235: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-23T19:06:41.076 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:17 smithi033 ceph-mon[29274]: pgmap v1236: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:41.148 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:17 smithi033 ceph-mon[29274]: osdmap e165: 12 total, 12 up, 12 in 2024-04-23T19:06:41.212 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:19 smithi033 ceph-mon[29274]: pgmap v1238: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 223 B/s wr, 11 op/s 2024-04-23T19:06:41.234 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:21 smithi033 ceph-mon[29274]: pgmap v1239: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:41.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:22 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:41.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: pgmap v1240: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:41.395 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:41.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:41.501 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: pgmap v1241: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 115 B/s wr, 11 op/s 2024-04-23T19:06:41.508 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: pgmap v1242: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-23T19:06:41.573 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:41.632 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:25 smithi033 ceph-mon[29274]: pgmap v1243: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-23T19:06:41.692 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:27 smithi033 ceph-mon[29274]: pgmap v1244: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:41.733 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:29 smithi033 ceph-mon[29274]: pgmap v1245: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:41.807 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:31 smithi033 ceph-mon[29274]: pgmap v1246: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:41.849 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:33 smithi033 ceph-mon[29274]: pgmap v1247: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:41.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:35 smithi033 ceph-mon[29274]: pgmap v1248: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:41.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:37 smithi033 ceph-mon[29274]: pgmap v1249: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:41.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:39 smithi033 ceph-mon[29274]: pgmap v1250: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.056 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:41 smithi033 ceph-mon[29274]: pgmap v1251: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:43 smithi033 ceph-mon[29274]: pgmap v1252: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:42.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:45 smithi033 ceph-mon[29274]: pgmap v1253: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.239 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:47 smithi033 ceph-mon[29274]: pgmap v1254: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:42.268 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:49 smithi033 ceph-mon[29274]: pgmap v1255: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.324 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:51 smithi033 ceph-mon[29274]: pgmap v1256: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:53 smithi033 ceph-mon[29274]: pgmap v1257: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:42.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:55 smithi033 ceph-mon[29274]: pgmap v1258: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.503 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:57 smithi033 ceph-mon[29274]: pgmap v1259: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:42.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:04:59 smithi033 ceph-mon[29274]: pgmap v1260: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:01 smithi033 ceph-mon[29274]: pgmap v1261: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:03 smithi033 ceph-mon[29274]: pgmap v1262: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:42.649 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:05 smithi033 ceph-mon[29274]: pgmap v1263: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:42.659 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:07 smithi033 ceph-mon[29274]: pgmap v1264: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:42.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:09 smithi033 ceph-mon[29274]: pgmap v1265: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:42.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:11 smithi033 ceph-mon[29274]: pgmap v1266: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:42.715 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:13 smithi033 ceph-mon[29274]: pgmap v1267: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:42.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:15 smithi033 ceph-mon[29274]: pgmap v1268: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-23T19:06:42.734 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:17 smithi033 ceph-mon[29274]: pgmap v1269: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-23T19:06:42.759 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:19 smithi033 ceph-mon[29274]: pgmap v1270: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:42.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:21 smithi033 ceph-mon[29274]: pgmap v1271: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-23T19:06:42.885 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:22 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:42.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:23 smithi033 ceph-mon[29274]: pgmap v1272: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-23T19:06:43.024 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:43.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:43.187 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:43.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:24 smithi033 ceph-mon[29274]: pgmap v1273: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-23T19:06:43.287 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:24 smithi033 ceph-mon[29274]: pgmap v1274: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:43.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:25 smithi033 ceph-mon[29274]: pgmap v1275: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:06:43.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:28 smithi033 ceph-mon[29274]: pgmap v1276: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:43.357 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:30 smithi033 ceph-mon[29274]: pgmap v1277: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:43.365 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:32 smithi033 ceph-mon[29274]: pgmap v1278: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:06:43.461 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:34 smithi033 ceph-mon[29274]: pgmap v1279: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:06:43.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:36 smithi033 ceph-mon[29274]: pgmap v1280: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:43.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:38 smithi033 ceph-mon[29274]: pgmap v1281: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:43.612 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:40 smithi033 ceph-mon[29274]: pgmap v1282: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:43.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:42 smithi033 ceph-mon[29274]: pgmap v1283: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:43.747 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:44 smithi033 ceph-mon[29274]: pgmap v1284: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:43.793 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:46 smithi033 ceph-mon[29274]: pgmap v1285: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:43.823 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:48 smithi033 ceph-mon[29274]: pgmap v1286: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:43.883 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:50 smithi033 ceph-mon[29274]: pgmap v1287: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:43.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:52 smithi033 ceph-mon[29274]: pgmap v1288: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:44.011 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:54 smithi033 ceph-mon[29274]: pgmap v1289: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:44.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:56 smithi033 ceph-mon[29274]: pgmap v1290: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:44.026 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:05:58 smithi033 ceph-mon[29274]: pgmap v1291: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:44.033 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:00 smithi033 ceph-mon[29274]: pgmap v1292: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:44.110 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:02 smithi033 ceph-mon[29274]: pgmap v1293: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:06:44.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:04 smithi033 ceph-mon[29274]: pgmap v1294: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:44.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:06 smithi033 ceph-mon[29274]: pgmap v1295: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:06:44.354 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:08 smithi033 ceph-mon[29274]: pgmap v1296: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:06:44.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:09 smithi033 ceph-mon[29274]: osdmap e166: 12 total, 12 up, 12 in 2024-04-23T19:06:44.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:10 smithi033 ceph-mon[29274]: pgmap v1298: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:44.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:12 smithi033 ceph-mon[29274]: pgmap v1299: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:44.694 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:14 smithi033 ceph-mon[29274]: pgmap v1300: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:44.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:16 smithi033 ceph-mon[29274]: pgmap v1301: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:06:44.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:16 smithi033 ceph-mon[29274]: osdmap e167: 12 total, 12 up, 12 in 2024-04-23T19:06:44.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:18 smithi033 ceph-mon[29274]: pgmap v1303: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 230 B/s wr, 11 op/s 2024-04-23T19:06:44.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:20 smithi033 ceph-mon[29274]: pgmap v1304: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:06:44.961 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:22 smithi033 ceph-mon[29274]: pgmap v1305: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:45.036 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:23 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:06:45.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:24 smithi033 ceph-mon[29274]: pgmap v1306: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-23T19:06:45.132 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:24 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:06:45.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:24 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:06:45.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:24 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:06:45.300 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T19:06:45.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:25 smithi033 ceph-mon[29274]: pgmap v1307: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:06:45.401 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:25 smithi033 ceph-mon[29274]: pgmap v1308: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:06:45.445 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-23T19:06:45.494 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-04-23T19:06:45.579 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 3298, 2024-04-23T19:06:45.668 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 3264, 2024-04-23T19:06:45.783 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 2784672, 2024-04-23T19:06:45.830 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1227206, 2024-04-23T19:06:45.870 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-04-23T19:06:45.891 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-04-23T19:06:45.941 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.451798993, 2024-04-23T19:06:46.017 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.157595348, 2024-04-23T19:06:46.073 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.143188270, 2024-04-23T19:06:46.123 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.119100108, 2024-04-23T19:06:46.164 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T19:06:46.217 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3263, 2024-04-23T19:06:46.269 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.180967927, 2024-04-23T19:06:46.304 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000055460 2024-04-23T19:06:46.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:46.395 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-23T19:06:46.404 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:46.412 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:46.421 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:46.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:46.523 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1941648, 2024-04-23T19:06:46.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 429344 2024-04-23T19:06:46.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:46.684 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-04-23T19:06:46.736 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2403, 2024-04-23T19:06:46.764 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2909, 2024-04-23T19:06:46.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 593588, 2024-04-23T19:06:46.778 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1926028, 2024-04-23T19:06:46.842 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 10, 2024-04-23T19:06:46.888 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 5, 2024-04-23T19:06:46.958 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.322982260, 2024-04-23T19:06:47.068 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.107180139, 2024-04-23T19:06:47.138 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.072585627, 2024-04-23T19:06:47.147 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.113337683, 2024-04-23T19:06:47.160 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T19:06:47.230 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2907, 2024-04-23T19:06:47.359 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.115024474, 2024-04-23T19:06:47.389 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000039568 2024-04-23T19:06:47.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:47.463 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-23T19:06:47.510 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:47.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:47.616 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:47.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:47.696 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-23T19:06:47.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 1164032 2024-04-23T19:06:47.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:47.791 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-04-23T19:06:47.878 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2515, 2024-04-23T19:06:47.934 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2517, 2024-04-23T19:06:47.957 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 660402, 2024-04-23T19:06:48.020 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 799771, 2024-04-23T19:06:48.060 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 6, 2024-04-23T19:06:48.129 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-04-23T19:06:48.170 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.276907733, 2024-04-23T19:06:48.245 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.091516778, 2024-04-23T19:06:48.298 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.073450613, 2024-04-23T19:06:48.341 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.084198544, 2024-04-23T19:06:48.407 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-04-23T19:06:48.417 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2515, 2024-04-23T19:06:48.425 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.103755590, 2024-04-23T19:06:48.434 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000041254 2024-04-23T19:06:48.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:48.544 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-04-23T19:06:48.585 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:48.771 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:48.777 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:48.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:48.866 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-23T19:06:48.987 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-04-23T19:06:49.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:49.065 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-04-23T19:06:49.074 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-04-23T19:06:49.100 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-04-23T19:06:49.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:49.210 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-04-23T19:06:49.306 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-23T19:06:49.353 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-23T19:06:49.386 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 11583, 2024-04-23T19:06:49.446 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.413984883, 2024-04-23T19:06:49.454 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000035740 2024-04-23T19:06:49.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:06:49.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:49.699 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-04-23T19:06:49.754 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-04-23T19:06:49.821 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-04-23T19:06:49.848 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:49.915 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:50.031 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:50.086 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:06:50.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:50.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-23T19:06:50.165 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-04-23T19:06:50.315 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-04-23T19:06:50.377 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-04-23T19:06:50.409 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:50.484 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:50.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:50.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:50.719 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-04-23T19:06:50.778 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-04-23T19:06:50.890 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-04-23T19:06:50.938 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-04-23T19:06:50.961 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-04-23T19:06:51.041 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-04-23T19:06:51.085 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-04-23T19:06:51.141 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 0, 2024-04-23T19:06:51.281 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 0, 2024-04-23T19:06:51.315 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 0, 2024-04-23T19:06:51.380 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-04-23T19:06:51.441 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 0, 2024-04-23T19:06:51.476 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-04-23T19:06:51.546 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-04-23T19:06:51.617 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-04-23T19:06:51.644 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-04-23T19:06:51.682 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-04-23T19:06:51.859 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-04-23T19:06:51.920 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-04-23T19:06:51.986 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-04-23T19:06:52.030 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-04-23T19:06:52.062 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-04-23T19:06:52.132 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-04-23T19:06:52.157 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-04-23T19:06:52.199 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-04-23T19:06:52.246 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-04-23T19:06:52.278 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-04-23T19:06:52.356 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-04-23T19:06:52.383 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-04-23T19:06:52.440 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-04-23T19:06:52.489 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-04-23T19:06:52.555 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-04-23T19:06:52.563 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-04-23T19:06:52.570 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-04-23T19:06:52.610 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-04-23T19:06:52.702 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-04-23T19:06:52.748 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-04-23T19:06:52.804 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-04-23T19:06:52.875 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-04-23T19:06:53.004 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-04-23T19:06:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-04-23T19:06:53.169 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-04-23T19:06:53.264 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-04-23T19:06:53.287 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-04-23T19:06:53.337 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-04-23T19:06:53.454 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-04-23T19:06:53.512 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-04-23T19:06:53.585 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-23T19:06:53.630 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-23T19:06:53.675 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-04-23T19:06:53.727 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-04-23T19:06:53.840 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-04-23T19:06:53.890 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-04-23T19:06:53.919 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:53.965 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-04-23T19:06:54.012 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-04-23T19:06:54.114 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-04-23T19:06:54.161 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-04-23T19:06:54.201 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-04-23T19:06:54.229 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-04-23T19:06:54.245 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-04-23T19:06:54.378 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-04-23T19:06:54.433 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-04-23T19:06:54.481 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-04-23T19:06:54.546 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-04-23T19:06:54.621 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-04-23T19:06:54.663 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-04-23T19:06:54.733 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-04-23T19:06:54.821 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-04-23T19:06:54.890 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-04-23T19:06:54.947 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-04-23T19:06:55.002 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-04-23T19:06:55.074 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-04-23T19:06:55.208 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-04-23T19:06:55.244 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-04-23T19:06:55.252 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-04-23T19:06:55.259 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-04-23T19:06:55.367 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-04-23T19:06:55.428 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-04-23T19:06:55.466 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0 2024-04-23T19:06:55.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:55.572 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-04-23T19:06:55.634 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-04-23T19:06:55.652 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-04-23T19:06:55.660 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-04-23T19:06:55.732 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-04-23T19:06:55.741 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 5, 2024-04-23T19:06:55.750 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-04-23T19:06:55.805 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-04-23T19:06:55.869 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-04-23T19:06:55.936 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-04-23T19:06:56.039 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-04-23T19:06:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 2, 2024-04-23T19:06:56.154 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 2, 2024-04-23T19:06:56.221 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-04-23T19:06:56.283 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-04-23T19:06:56.342 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 4194304, 2024-04-23T19:06:56.374 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 4195815, 2024-04-23T19:06:56.430 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 4195674, 2024-04-23T19:06:56.523 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-04-23T19:06:56.607 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:56.764 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:56.840 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:56.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:56.994 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 5 2024-04-23T19:06:57.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:57.097 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-04-23T19:06:57.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 3, 2024-04-23T19:06:57.271 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 3, 2024-04-23T19:06:57.335 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 0, 2024-04-23T19:06:57.384 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 1, 2024-04-23T19:06:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 1, 2024-04-23T19:06:57.549 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 0, 2024-04-23T19:06:57.614 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 0, 2024-04-23T19:06:57.623 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 0, 2024-04-23T19:06:57.651 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 0, 2024-04-23T19:06:57.720 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-04-23T19:06:57.787 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-04-23T19:06:57.888 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-04-23T19:06:58.003 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 50200, 2024-04-23T19:06:58.088 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 215416 2024-04-23T19:06:58.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:58.227 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-04-23T19:06:58.308 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-04-23T19:06:58.382 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-04-23T19:06:58.463 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-04-23T19:06:58.499 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-04-23T19:06:58.543 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-04-23T19:06:58.636 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-04-23T19:06:58.672 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:58.681 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:58.689 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:58.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:58.757 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-04-23T19:06:58.766 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:58.849 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:58.964 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:59.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:59.092 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-04-23T19:06:59.191 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:59.227 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:59.267 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:59.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:59.434 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-04-23T19:06:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:59.564 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:59.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:59.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:06:59.750 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-04-23T19:06:59.798 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:06:59.856 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:06:59.924 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:06:59.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:00.110 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-04-23T19:07:00.195 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:00.235 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:00.322 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:00.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:00.410 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-04-23T19:07:00.455 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:00.496 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:00.554 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:00.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:00.768 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-04-23T19:07:00.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:00.866 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:00.898 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:01.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:01.091 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-04-23T19:07:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:01.296 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:01.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:01.395 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-04-23T19:07:01.472 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:01.526 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:01.626 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:01.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:01.765 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-04-23T19:07:01.873 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:01.884 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:01.893 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:01.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:01.966 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-04-23T19:07:02.056 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:02.109 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:02.167 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:02.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:02.234 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-04-23T19:07:02.243 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:02.335 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:02.394 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:02.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:02.518 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-04-23T19:07:02.543 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:02.551 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:02.666 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:02.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:02.826 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-04-23T19:07:02.850 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:02.885 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:02.967 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:03.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:03.146 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-04-23T19:07:03.269 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:03.348 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:03.380 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:03.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:03.508 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-04-23T19:07:03.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:03.658 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:03.731 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:03.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:03.857 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-04-23T19:07:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:03.970 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:04.008 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:04.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-04-23T19:07:04.159 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:04.178 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:04.311 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:04.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:04.489 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-04-23T19:07:04.548 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:04.564 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:04.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:04.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:04.680 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-04-23T19:07:04.759 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:04.805 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:04.851 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:04.870 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-04-23T19:07:04.879 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:04.940 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:04.989 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:05.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:05.160 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-04-23T19:07:05.271 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:05.451 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:05.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:05.550 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-04-23T19:07:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-04-23T19:07:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-04-23T19:07:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-04-23T19:07:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-04-23T19:07:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-04-23T19:07:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-04-23T19:07:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-04-23T19:07:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-04-23T19:07:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-04-23T19:07:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-04-23T19:07:07.651 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-04-23T19:07:07.660 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-04-23T19:07:07.670 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 2, 2024-04-23T19:07:07.726 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-04-23T19:07:07.772 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-04-23T19:07:07.846 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-04-23T19:07:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 4687, 2024-04-23T19:07:07.984 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-04-23T19:07:08.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:08.106 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-04-23T19:07:08.151 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-04-23T19:07:08.193 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-04-23T19:07:08.273 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-04-23T19:07:08.387 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-04-23T19:07:08.400 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-04-23T19:07:08.450 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-04-23T19:07:08.575 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-23T19:07:08.634 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-04-23T19:07:08.679 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-23T19:07:08.725 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-04-23T19:07:08.789 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-04-23T19:07:08.887 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-04-23T19:07:08.954 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-23T19:07:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-04-23T19:07:09.111 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-04-23T19:07:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-04-23T19:07:09.215 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-04-23T19:07:09.260 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-04-23T19:07:09.313 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-23T19:07:09.388 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-04-23T19:07:09.424 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-23T19:07:09.454 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-04-23T19:07:09.552 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-04-23T19:07:09.645 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-04-23T19:07:09.672 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-04-23T19:07:09.680 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-04-23T19:07:09.688 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-23T19:07:09.761 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-04-23T19:07:09.814 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-04-23T19:07:09.877 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-04-23T19:07:09.936 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-04-23T19:07:09.981 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-04-23T19:07:10.046 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-04-23T19:07:10.126 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-04-23T19:07:10.152 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-04-23T19:07:10.197 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-04-23T19:07:10.268 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 190194, 2024-04-23T19:07:10.327 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 54, 2024-04-23T19:07:10.351 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-04-23T19:07:10.433 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-04-23T19:07:10.462 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-04-23T19:07:10.499 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-04-23T19:07:10.538 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-04-23T19:07:10.582 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-04-23T19:07:10.642 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-04-23T19:07:10.664 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-04-23T19:07:10.673 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 8128, 2024-04-23T19:07:10.758 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 169, 2024-04-23T19:07:10.857 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-04-23T19:07:10.923 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-04-23T19:07:10.974 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-04-23T19:07:11.009 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-04-23T19:07:11.048 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 8512, 2024-04-23T19:07:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 13, 2024-04-23T19:07:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-04-23T19:07:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-04-23T19:07:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-04-23T19:07:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-04-23T19:07:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-04-23T19:07:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-04-23T19:07:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-04-23T19:07:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 7420, 2024-04-23T19:07:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-04-23T19:07:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-04-23T19:07:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 7420, 2024-04-23T19:07:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-04-23T19:07:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 7420, 2024-04-23T19:07:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 4.861921719, 2024-04-23T19:07:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000655245 2024-04-23T19:07:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-04-23T19:07:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-04-23T19:07:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 7420, 2024-04-23T19:07:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 7421 2024-04-23T19:07:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 7420, 2024-04-23T19:07:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 7420, 2024-04-23T19:07:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-04-23T19:07:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-04-23T19:07:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-04-23T19:07:12.787 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 4902, 2024-04-23T19:07:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-04-23T19:07:12.892 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 2516, 2024-04-23T19:07:12.937 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-04-23T19:07:12.973 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-04-23T19:07:13.005 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-04-23T19:07:13.056 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-04-23T19:07:13.069 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-04-23T19:07:13.077 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-04-23T19:07:13.148 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-04-23T19:07:13.201 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-04-23T19:07:13.227 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-04-23T19:07:13.271 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-04-23T19:07:13.340 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-04-23T19:07:13.402 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-04-23T19:07:13.433 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-04-23T19:07:13.478 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-04-23T19:07:13.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-04-23T19:07:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-04-23T19:07:13.633 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-04-23T19:07:13.664 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-04-23T19:07:13.720 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-04-23T19:07:13.749 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-04-23T19:07:13.793 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-04-23T19:07:13.809 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-04-23T19:07:13.907 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-04-23T19:07:13.980 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-04-23T19:07:14.004 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-04-23T19:07:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-04-23T19:07:14.069 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-04-23T19:07:14.098 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-04-23T19:07:14.163 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-04-23T19:07:14.236 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-04-23T19:07:14.320 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-04-23T19:07:14.429 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-04-23T19:07:14.483 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-04-23T19:07:14.539 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-04-23T19:07:14.586 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-04-23T19:07:14.639 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-04-23T19:07:14.704 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-04-23T19:07:14.772 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-04-23T19:07:14.781 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 167, 2024-04-23T19:07:14.790 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-04-23T19:07:14.896 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 82, 2024-04-23T19:07:14.943 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 5, 2024-04-23T19:07:14.992 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 9, 2024-04-23T19:07:15.101 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 4, 2024-04-23T19:07:15.134 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-04-23T19:07:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-04-23T19:07:15.271 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-04-23T19:07:15.312 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-04-23T19:07:15.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:15.494 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-04-23T19:07:15.576 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-04-23T19:07:15.647 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-04-23T19:07:15.678 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-04-23T19:07:15.743 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-04-23T19:07:15.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:15.777 INFO:teuthology.orchestra.run.smithi033.stdout: "osd-slow-ops": { 2024-04-23T19:07:15.787 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-04-23T19:07:15.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:15.902 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-04-23T19:07:15.971 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-04-23T19:07:16.018 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-04-23T19:07:16.062 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-04-23T19:07:16.125 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-04-23T19:07:16.196 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 0, 2024-04-23T19:07:16.229 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-04-23T19:07:16.324 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-04-23T19:07:16.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:16.454 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-23T19:07:16.504 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-23T19:07:16.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-23T19:07:16.662 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-23T19:07:16.732 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 8212, 2024-04-23T19:07:16.755 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 3256395, 2024-04-23T19:07:16.766 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-23T19:07:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 8212, 2024-04-23T19:07:16.942 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-23T19:07:17.090 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-23T19:07:17.133 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 8212, 2024-04-23T19:07:17.174 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 3256395, 2024-04-23T19:07:17.248 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-23T19:07:17.282 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:17.382 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:17.475 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:17.540 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:17.685 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-04-23T19:07:17.737 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-23T19:07:17.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-04-23T19:07:17.818 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-23T19:07:17.827 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-23T19:07:17.837 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-23T19:07:17.887 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-23T19:07:17.930 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-23T19:07:17.977 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 7420, 2024-04-23T19:07:18.051 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 1516, 2024-04-23T19:07:18.116 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 7420, 2024-04-23T19:07:18.174 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1516, 2024-04-23T19:07:18.181 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-23T19:07:18.190 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:18.199 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:18.209 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:18.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:18.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:18.412 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-04-23T19:07:18.468 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-23T19:07:18.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-04-23T19:07:18.647 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-23T19:07:18.713 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-23T19:07:18.770 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-23T19:07:18.816 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-23T19:07:18.862 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-23T19:07:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 7420, 2024-04-23T19:07:18.881 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 7420, 2024-04-23T19:07:18.950 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 7420, 2024-04-23T19:07:19.036 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 7420, 2024-04-23T19:07:19.085 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-23T19:07:19.209 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:19.244 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:19.345 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:19.394 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:19.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:19.519 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-04-23T19:07:19.559 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-23T19:07:19.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-23T19:07:19.628 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-23T19:07:19.671 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-23T19:07:19.740 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-23T19:07:19.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-23T19:07:19.874 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-23T19:07:19.882 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-23T19:07:19.924 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-23T19:07:20.015 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-04-23T19:07:20.088 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-04-23T19:07:20.213 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-23T19:07:20.251 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:20.324 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:20.374 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:20.463 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:20.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-23T19:07:20.601 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x563525ac06e0": { 2024-04-23T19:07:20.630 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-04-23T19:07:20.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-04-23T19:07:20.756 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-04-23T19:07:20.829 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-04-23T19:07:20.869 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-04-23T19:07:20.935 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-04-23T19:07:20.958 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-04-23T19:07:21.014 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-04-23T19:07:21.025 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-04-23T19:07:21.074 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-04-23T19:07:21.153 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-04-23T19:07:21.269 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-04-23T19:07:21.350 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-04-23T19:07:21.460 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-04-23T19:07:21.499 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-04-23T19:07:21.568 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:21.654 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-23T19:07:21.699 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-23T19:07:21.718 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:27 smithi033 ceph-mon[29274]: pgmap v1309: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:21.770 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-23T19:07:21.828 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-23T19:07:21.873 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-23T19:07:21.953 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-23T19:07:22.034 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-23T19:07:22.116 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-23T19:07:22.208 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-23T19:07:22.259 DEBUG:tasks.check_counter:Getting stats from d 2024-04-23T19:07:22.363 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-23T19:07:22.417 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:29 smithi033 ceph-mon[29274]: pgmap v1310: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:22.481 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:31 smithi033 ceph-mon[29274]: pgmap v1311: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:22.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:33 smithi033 ceph-mon[29274]: pgmap v1312: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:22.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:35 smithi033 ceph-mon[29274]: pgmap v1313: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:22.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:37 smithi033 ceph-mon[29274]: pgmap v1314: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:22.768 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:39 smithi033 ceph-mon[29274]: pgmap v1315: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:22.777 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:41 smithi033 ceph-mon[29274]: pgmap v1316: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:22.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:43 smithi033 ceph-mon[29274]: pgmap v1317: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:22.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:45 smithi033 ceph-mon[29274]: pgmap v1318: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:22.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:47 smithi033 ceph-mon[29274]: pgmap v1319: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:23.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:49 smithi033 ceph-mon[29274]: pgmap v1320: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.076 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:51 smithi033 ceph-mon[29274]: pgmap v1321: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.085 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:53 smithi033 ceph-mon[29274]: pgmap v1322: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:23.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:55 smithi033 ceph-mon[29274]: pgmap v1323: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:57 smithi033 ceph-mon[29274]: pgmap v1324: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:23.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:06:59 smithi033 ceph-mon[29274]: pgmap v1325: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.334 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:01 smithi033 ceph-mon[29274]: pgmap v1326: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.389 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:03 smithi033 ceph-mon[29274]: pgmap v1327: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:23.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:05 smithi033 ceph-mon[29274]: pgmap v1328: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:23.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:07 smithi033 ceph-mon[29274]: pgmap v1329: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:07:23.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:09 smithi033 ceph-mon[29274]: pgmap v1330: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:07:23.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:09 smithi033 ceph-mon[29274]: osdmap e168: 12 total, 12 up, 12 in 2024-04-23T19:07:23.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:11 smithi033 ceph-mon[29274]: pgmap v1332: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:23.784 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:13 smithi033 ceph-mon[29274]: pgmap v1333: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:23.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:15 smithi033 ceph-mon[29274]: pgmap v1334: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:23.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:17 smithi033 ceph-mon[29274]: pgmap v1335: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:24.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:17 smithi033 ceph-mon[29274]: osdmap e169: 12 total, 12 up, 12 in 2024-04-23T19:07:24.109 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:19 smithi033 ceph-mon[29274]: pgmap v1337: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-23T19:07:24.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:21 smithi033 ceph-mon[29274]: pgmap v1338: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:24.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:23 smithi033 ceph-mon[29274]: pgmap v1339: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:24.347 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:25 smithi122 ceph-mon[32546]: pgmap v1307: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:07:24.372 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:25 smithi122 ceph-mon[32546]: pgmap v1308: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:07:24.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:27 smithi122 ceph-mon[32546]: pgmap v1309: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:24.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:29 smithi122 ceph-mon[32546]: pgmap v1310: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:24.533 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:31 smithi122 ceph-mon[32546]: pgmap v1311: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:24.636 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:33 smithi122 ceph-mon[32546]: pgmap v1312: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:24.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:35 smithi122 ceph-mon[32546]: pgmap v1313: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:24.751 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:37 smithi122 ceph-mon[32546]: pgmap v1314: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:24.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:39 smithi122 ceph-mon[32546]: pgmap v1315: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:24.901 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:41 smithi122 ceph-mon[32546]: pgmap v1316: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:24.972 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:43 smithi122 ceph-mon[32546]: pgmap v1317: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:25.025 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:45 smithi122 ceph-mon[32546]: pgmap v1318: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.033 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:47 smithi122 ceph-mon[32546]: pgmap v1319: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:25.041 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:49 smithi122 ceph-mon[32546]: pgmap v1320: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.048 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:51 smithi122 ceph-mon[32546]: pgmap v1321: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:53 smithi122 ceph-mon[32546]: pgmap v1322: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:25.063 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:55 smithi122 ceph-mon[32546]: pgmap v1323: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.106 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:57 smithi122 ceph-mon[32546]: pgmap v1324: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:25.178 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:06:59 smithi122 ceph-mon[32546]: pgmap v1325: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:01 smithi122 ceph-mon[32546]: pgmap v1326: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.322 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:03 smithi122 ceph-mon[32546]: pgmap v1327: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:25.388 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:05 smithi122 ceph-mon[32546]: pgmap v1328: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:25.454 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:07 smithi122 ceph-mon[32546]: pgmap v1329: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:07:25.504 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:09 smithi122 ceph-mon[32546]: pgmap v1330: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:07:25.567 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:09 smithi122 ceph-mon[32546]: osdmap e168: 12 total, 12 up, 12 in 2024-04-23T19:07:25.631 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:11 smithi122 ceph-mon[32546]: pgmap v1332: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:25.708 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:13 smithi122 ceph-mon[32546]: pgmap v1333: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:25.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:15 smithi122 ceph-mon[32546]: pgmap v1334: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:25.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:17 smithi122 ceph-mon[32546]: pgmap v1335: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:25.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:17 smithi122 ceph-mon[32546]: osdmap e169: 12 total, 12 up, 12 in 2024-04-23T19:07:25.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:19 smithi122 ceph-mon[32546]: pgmap v1337: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-23T19:07:25.754 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:21 smithi122 ceph-mon[32546]: pgmap v1338: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:25.762 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:23 smithi122 ceph-mon[32546]: pgmap v1339: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:25.770 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:25 smithi106 ceph-mon[32716]: pgmap v1307: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 116 B/s wr, 11 op/s 2024-04-23T19:07:25.837 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:25 smithi106 ceph-mon[32716]: pgmap v1308: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-23T19:07:25.897 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:27 smithi106 ceph-mon[32716]: pgmap v1309: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:25.952 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:29 smithi106 ceph-mon[32716]: pgmap v1310: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:26.107 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:31 smithi106 ceph-mon[32716]: pgmap v1311: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:26.175 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:33 smithi106 ceph-mon[32716]: pgmap v1312: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-23T19:07:26.267 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:35 smithi106 ceph-mon[32716]: pgmap v1313: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:37 smithi106 ceph-mon[32716]: pgmap v1314: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:26.379 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:39 smithi106 ceph-mon[32716]: pgmap v1315: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.407 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:41 smithi106 ceph-mon[32716]: pgmap v1316: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.471 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:43 smithi106 ceph-mon[32716]: pgmap v1317: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:26.516 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:45 smithi106 ceph-mon[32716]: pgmap v1318: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.605 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:47 smithi106 ceph-mon[32716]: pgmap v1319: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:26.664 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:49 smithi106 ceph-mon[32716]: pgmap v1320: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.673 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:51 smithi106 ceph-mon[32716]: pgmap v1321: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.682 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:53 smithi106 ceph-mon[32716]: pgmap v1322: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:26.691 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:55 smithi106 ceph-mon[32716]: pgmap v1323: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.700 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:57 smithi106 ceph-mon[32716]: pgmap v1324: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:26.759 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:06:59 smithi106 ceph-mon[32716]: pgmap v1325: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.850 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:01 smithi106 ceph-mon[32716]: pgmap v1326: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:26.936 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:03 smithi106 ceph-mon[32716]: pgmap v1327: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:27.055 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:05 smithi106 ceph-mon[32716]: pgmap v1328: 97 pgs: 97 active+clean; 15 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:27.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:07 smithi106 ceph-mon[32716]: pgmap v1329: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 44 KiB/s wr, 29 op/s 2024-04-23T19:07:27.216 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:09 smithi106 ceph-mon[32716]: pgmap v1330: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-04-23T19:07:27.278 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:09 smithi106 ceph-mon[32716]: osdmap e168: 12 total, 12 up, 12 in 2024-04-23T19:07:27.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:11 smithi106 ceph-mon[32716]: pgmap v1332: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:27.383 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:13 smithi106 ceph-mon[32716]: pgmap v1333: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:27.477 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:15 smithi106 ceph-mon[32716]: pgmap v1334: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:07:27.576 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:17 smithi106 ceph-mon[32716]: pgmap v1335: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:27.622 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:17 smithi106 ceph-mon[32716]: osdmap e169: 12 total, 12 up, 12 in 2024-04-23T19:07:27.664 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:19 smithi106 ceph-mon[32716]: pgmap v1337: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 213 B/s wr, 10 op/s 2024-04-23T19:07:27.693 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:21 smithi106 ceph-mon[32716]: pgmap v1338: 97 pgs: 97 active+clean; 15 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-23T19:07:27.703 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:23 smithi106 ceph-mon[32716]: pgmap v1339: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:27.716 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:24 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:07:27.725 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:24 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:07:27.801 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:25 smithi106 ceph-mon[32716]: pgmap v1340: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:27.877 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:27.933 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:28.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: pgmap v1341: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T19:07:28.127 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:28.219 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:28.277 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.362 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.497 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T19:07:28.506 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T19:07:28.515 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.524 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.533 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.576 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:26 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:28.662 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:24 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:07:28.772 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:25 smithi122 ceph-mon[32546]: pgmap v1340: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:28.801 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:28.810 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:28.922 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: pgmap v1341: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T19:07:29.017 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:29.093 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:29.143 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.221 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.250 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.283 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.331 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T19:07:29.406 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T19:07:29.465 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.537 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.546 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.555 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:26 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/mon.a/config 2024-04-23T19:07:29.618 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:25 smithi033 ceph-mon[29274]: pgmap v1340: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 102 B/s wr, 9 op/s 2024-04-23T19:07:29.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-23T19:07:29.749 INFO:teuthology.orchestra.run.smithi033.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-04-23T19:07:29.867 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/github.com_ceph_ceph_57e0caf76fdb7ab8b1358588f292d08519163844/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi033 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 766ecab6-019d-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump' 2024-04-23T19:07:29.903 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-23T19:07:29.998 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T19:07:30.027 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T19:07:30.060 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T19:07:30.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:30.189 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:30.204 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: pgmap v1341: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-23T19:07:30.213 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:30.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:30.262 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.368 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-23T19:07:30.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi106 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-23T19:07:30.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.841 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:26 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:30.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:27 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T19:07:30.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:27 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T19:07:30.987 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: pgmap v1342: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:31.082 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:31.137 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:31.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:31.197 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:31.205 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:31.211 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:31.219 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:07:31.226 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:07:31.311 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:31.464 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:30 smithi033 ceph-mon[29274]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T19:07:31.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:30 smithi033 ceph-mon[29274]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T19:07:31.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:30 smithi033 ceph-mon[29274]: pgmap v1343: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-23T19:07:31.698 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:30 smithi033 ceph-mon[29274]: pgmap v1344: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:07:31.787 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:27 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T19:07:31.834 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:27 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T19:07:31.882 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: pgmap v1342: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:31.911 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:31.986 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:32.003 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.010 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.017 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.025 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.032 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:07:32.085 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:07:32.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:32.189 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:30 smithi106 ceph-mon[32716]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T19:07:32.260 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:30 smithi106 ceph-mon[32716]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T19:07:32.330 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:30 smithi106 ceph-mon[32716]: pgmap v1343: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-23T19:07:32.432 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:30 smithi106 ceph-mon[32716]: pgmap v1344: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:07:32.473 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:27 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-23T19:07:32.517 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:27 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi122 to 184727244: error parsing value: Value '184727244' is below minimum 939524096 2024-04-23T19:07:32.578 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: pgmap v1342: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:32.627 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:32.684 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:32.695 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.704 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.713 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-23T19:07:32.731 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:07:32.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:07:32.827 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:07:32.886 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:30 smithi122 ceph-mon[32546]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-04-23T19:07:32.945 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:30 smithi122 ceph-mon[32546]: Unable to set osd_memory_target on smithi033 to 185897062: error parsing value: Value '185897062' is below minimum 939524096 2024-04-23T19:07:33.007 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:30 smithi122 ceph-mon[32546]: pgmap v1343: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-23T19:07:33.037 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:30 smithi122 ceph-mon[32546]: pgmap v1344: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 11 op/s 2024-04-23T19:07:33.047 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:32 smithi106 ceph-mon[32716]: pgmap v1345: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:33.056 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:32 smithi122 ceph-mon[32546]: pgmap v1345: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:33.128 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T19:07:33.149 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T19:07:33.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:32 smithi033 ceph-mon[29274]: pgmap v1345: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:33.256 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:07:33.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 -- 172.21.15.33:0/1995392043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 msgr2=0x7f77ec10e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:33.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 --2- 172.21.15.33:0/1995392043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec10e9e0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009ca0 tx=0x7f77e402a950 comp rx=0 tx=0).stop 2024-04-23T19:07:33.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 -- 172.21.15.33:0/1995392043 shutdown_connections 2024-04-23T19:07:33.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 --2- 172.21.15.33:0/1995392043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec10e9e0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:33.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 --2- 172.21.15.33:0/1995392043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103b50 0x7f77ec10c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:33.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 --2- 172.21.15.33:0/1995392043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103230 0x7f77ec103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 -- 172.21.15.33:0/1995392043 >> 172.21.15.33:0/1995392043 conn(0x7f77ec077fb0 msgr2=0x7f77ec0783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 -- 172.21.15.33:0/1995392043 shutdown_connections 2024-04-23T19:07:33.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 -- 172.21.15.33:0/1995392043 wait complete. 2024-04-23T19:07:33.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.464+0000 7f77f166b640 1 Processor -- start 2024-04-23T19:07:33.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 -- start start 2024-04-23T19:07:34.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103230 0x7f77ec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:34.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103b50 0x7f77ec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:34.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:34.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec1121d0 con 0x7f77ec10c570 2024-04-23T19:07:34.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec112050 con 0x7f77ec103b50 2024-04-23T19:07:34.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77f166b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77ec112350 con 0x7f77ec103230 2024-04-23T19:07:34.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:34.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41216/0 (socket says 172.21.15.33:41216) 2024-04-23T19:07:34.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 -- 172.21.15.33:0/2244571289 learned_addr learned my addr 172.21.15.33:0/2244571289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:34.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77ea7fc640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103b50 0x7f77ec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eaffd640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103230 0x7f77ec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 -- 172.21.15.33:0/2244571289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103230 msgr2=0x7f77ec198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103230 0x7f77ec198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 -- 172.21.15.33:0/2244571289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103b50 msgr2=0x7f77ec198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:34.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103b50 0x7f77ec198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:34.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.465+0000 7f77eb7fe640 1 -- 172.21.15.33:0/2244571289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ec19d6f0 con 0x7f77ec10c570 2024-04-23T19:07:34.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77eb7fe640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f77e4004af0 tx=0x7f77e4031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4034040 con 0x7f77ec10c570 2024-04-23T19:07:35.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ec19d980 con 0x7f77ec10c570 2024-04-23T19:07:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e40308f0 con 0x7f77ec10c570 2024-04-23T19:07:35.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e4004600 con 0x7f77ec10c570 2024-04-23T19:07:35.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.466+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ec1a5270 con 0x7f77ec10c570 2024-04-23T19:07:35.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.467+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ec0691c0 con 0x7f77ec10c570 2024-04-23T19:07:35.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.468+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77e4037020 con 0x7f77ec10c570 2024-04-23T19:07:35.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.468+0000 7f77e3fff640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 0x7f77d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:35.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.469+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f77e40f15e0 con 0x7f77ec10c570 2024-04-23T19:07:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.469+0000 7f77eaffd640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 0x7f77d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:35.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.469+0000 7f77eaffd640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 0x7f77d80a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005d00 tx=0x7f77dc00b410 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:35.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.472+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f77e4080ff0 con 0x7f77ec10c570 2024-04-23T19:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.662+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77ec107720 con 0x7f77ec10c570 2024-04-23T19:07:35.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.663+0000 7f77e3fff640 1 -- 172.21.15.33:0/2244571289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+23195 (secure 0 0 0) 0x7f77e40bdd20 con 0x7f77ec10c570 2024-04-23T19:07:35.834 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:34 smithi122 ceph-mon[32546]: pgmap v1346: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:35.884 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:34 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2244571289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:07:35.933 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:34 smithi106 ceph-mon[32716]: pgmap v1346: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:36.016 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:34 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2244571289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:07:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T19:07:36.182 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T19:07:16.212951+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":76,"snap_epoch":168,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":64,"length":1},{"begin":76,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:07:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 msgr2=0x7f77d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:36.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 0x7f77d80a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f77dc005d00 tx=0x7f77dc00b410 comp rx=0 tx=0).stop 2024-04-23T19:07:36.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 msgr2=0x7f77ec19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:36.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f77e4004af0 tx=0x7f77e4031f50 comp rx=0 tx=0).stop 2024-04-23T19:07:36.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 shutdown_connections 2024-04-23T19:07:36.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.665+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f77d80a3d20 0x7f77d80a61e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:36.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77ec10c570 0x7f77ec19cfc0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:36.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ec103b50 0x7f77ec198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 --2- 172.21.15.33:0/2244571289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77ec103230 0x7f77ec198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:36.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 >> 172.21.15.33:0/2244571289 conn(0x7f77ec077fb0 msgr2=0x7f77ec0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 shutdown_connections 2024-04-23T19:07:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:33.666+0000 7f77f166b640 1 -- 172.21.15.33:0/2244571289 wait complete. 2024-04-23T19:07:36.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-23T18:19:48.156180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-23T18:24:24.905524+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-23T18:24:25.596587+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '168', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 76, 'snap_epoch': 168, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-23T19:07:36.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-23T19:07:36.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:34 smithi033 ceph-mon[29274]: pgmap v1346: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:36.831 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:34 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2244571289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:36 smithi033 ceph-mon[29274]: pgmap v1347: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:36.986 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:36 smithi122 ceph-mon[32546]: pgmap v1347: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:37.040 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:36 smithi106 ceph-mon[32716]: pgmap v1347: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-23T19:07:37.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- 172.21.15.33:0/2402188936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 msgr2=0x7f446410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:37.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 --2- 172.21.15.33:0/2402188936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f446410ee40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f445c009ca0 tx=0x7f445c02a8c0 comp rx=0 tx=0).stop 2024-04-23T19:07:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- 172.21.15.33:0/2402188936 shutdown_connections 2024-04-23T19:07:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 --2- 172.21.15.33:0/2402188936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f446410ee40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:37.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 --2- 172.21.15.33:0/2402188936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406b1f0 0x7f4464106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:37.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 --2- 172.21.15.33:0/2402188936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446406a840 0x7f446406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:37.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- 172.21.15.33:0/2402188936 >> 172.21.15.33:0/2402188936 conn(0x7f4464077eb0 msgr2=0x7f44640782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:37.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- 172.21.15.33:0/2402188936 shutdown_connections 2024-04-23T19:07:37.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- 172.21.15.33:0/2402188936 wait complete. 2024-04-23T19:07:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 Processor -- start 2024-04-23T19:07:37.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.101+0000 7f446a39b640 1 -- start start 2024-04-23T19:07:37.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 0x7f4464198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446419cfd0 0x7f446419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:37.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44641121b0 con 0x7f4464107270 2024-04-23T19:07:37.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4464112030 con 0x7f446406a840 2024-04-23T19:07:37.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f446a39b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4464112330 con 0x7f446419cfd0 2024-04-23T19:07:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47482/0 (socket says 172.21.15.33:47482) 2024-04-23T19:07:37.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 -- 172.21.15.33:0/2100657233 learned_addr learned my addr 172.21.15.33:0/2100657233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:37.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f4463fff640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 0x7f4464198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:38.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f4468938640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446419cfd0 0x7f446419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:38.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 -- 172.21.15.33:0/2100657233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446419cfd0 msgr2=0x7f446419d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:38.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446419cfd0 0x7f446419d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:38.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 -- 172.21.15.33:0/2100657233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 msgr2=0x7f4464198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:38.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 0x7f4464198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:38.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 -- 172.21.15.33:0/2100657233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446419dac0 con 0x7f4464107270 2024-04-23T19:07:38.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f4463fff640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 0x7f4464198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:07:38.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44637fe640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9c0 tx=0x7f444c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.102+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c00cd80 con 0x7f4464107270 2024-04-23T19:07:38.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.103+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f444c0054d0 con 0x7f4464107270 2024-04-23T19:07:38.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.103+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f446419ddb0 con 0x7f4464107270 2024-04-23T19:07:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.103+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444c010430 con 0x7f4464107270 2024-04-23T19:07:38.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.103+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a55f0 con 0x7f4464107270 2024-04-23T19:07:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.104+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44640691c0 con 0x7f4464107270 2024-04-23T19:07:39.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.105+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f444c004fb0 con 0x7f4464107270 2024-04-23T19:07:39.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.106+0000 7f44617fa640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 0x7f44500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:39.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.106+0000 7f4463fff640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 0x7f44500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.106+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f444c0d0e30 con 0x7f4464107270 2024-04-23T19:07:39.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.106+0000 7f4463fff640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 0x7f44500a6260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4454009c20 tx=0x7f4454005f90 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.109+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f444c053ff0 con 0x7f4464107270 2024-04-23T19:07:39.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.298+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f446406b1f0 con 0x7f4464107270 2024-04-23T19:07:39.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.298+0000 7f44617fa640 1 -- 172.21.15.33:0/2100657233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) ==== 93+0+10 (secure 0 0 0) 0x7f444c09d460 con 0x7f4464107270 2024-04-23T19:07:39.436 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-23T19:07:39.459 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:38 smithi122 ceph-mon[32546]: pgmap v1348: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:39.540 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:38 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2100657233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T19:07:39.667 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:38 smithi106 ceph-mon[32716]: pgmap v1348: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:39.750 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:38 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2100657233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T19:07:39.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 msgr2=0x7f44500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 0x7f44500a6260 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4454009c20 tx=0x7f4454005f90 comp rx=0 tx=0).stop 2024-04-23T19:07:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 msgr2=0x7f4464198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:40.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f444c00e9c0 tx=0x7f444c00ee80 comp rx=0 tx=0).stop 2024-04-23T19:07:40.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 shutdown_connections 2024-04-23T19:07:40.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f44500a3da0 0x7f44500a6260 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f446419cfd0 0x7f446419d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4464107270 0x7f4464198c50 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 --2- 172.21.15.33:0/2100657233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f446406a840 0x7f4464198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 >> 172.21.15.33:0/2100657233 conn(0x7f4464077eb0 msgr2=0x7f44640fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:40.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 shutdown_connections 2024-04-23T19:07:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:37.301+0000 7f446a39b640 1 -- 172.21.15.33:0/2100657233 wait complete. 2024-04-23T19:07:40.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-23T19:07:40.380 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:38 smithi033 ceph-mon[29274]: pgmap v1348: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-23T19:07:40.389 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:38 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2100657233' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-23T19:07:40.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 -- 172.21.15.33:0/1944354990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec810c510 msgr2=0x7fbec810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/1944354990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec810c510 0x7fbec810e9b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009ca0 tx=0x7fbec002a9e0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 -- 172.21.15.33:0/1944354990 shutdown_connections 2024-04-23T19:07:40.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/1944354990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec810c510 0x7fbec810e9b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/1944354990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 0x7fbec810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/1944354990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec81031d0 0x7fbec81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:40.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 -- 172.21.15.33:0/1944354990 >> 172.21.15.33:0/1944354990 conn(0x7fbec80771d0 msgr2=0x7fbec80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:40.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 -- 172.21.15.33:0/1944354990 shutdown_connections 2024-04-23T19:07:41.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.592+0000 7fbecc8dc640 1 -- 172.21.15.33:0/1944354990 wait complete. 2024-04-23T19:07:41.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 Processor -- start 2024-04-23T19:07:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 -- start start 2024-04-23T19:07:41.081 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:40 smithi122 ceph-mon[32546]: pgmap v1349: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:41.129 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:40 smithi106 ceph-mon[32716]: pgmap v1349: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:41.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 0x7fbec8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 0x7fbec819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8112180 con 0x7fbec81031d0 2024-04-23T19:07:41.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8112000 con 0x7fbec8103af0 2024-04-23T19:07:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.593+0000 7fbecc8dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbec8112300 con 0x7fbec810c510 2024-04-23T19:07:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:41.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47504/0 (socket says 172.21.15.33:47504) 2024-04-23T19:07:41.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 -- 172.21.15.33:0/240785723 learned_addr learned my addr 172.21.15.33:0/240785723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:41.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbebffff640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 0x7fbec8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:41.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec6d9d640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 0x7fbec819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 -- 172.21.15.33:0/240785723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 msgr2=0x7fbec819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 0x7fbec819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 -- 172.21.15.33:0/240785723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 msgr2=0x7fbec8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 0x7fbec8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 -- 172.21.15.33:0/240785723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec819d640 con 0x7fbec81031d0 2024-04-23T19:07:42.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec6d9d640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 0x7fbec819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:07:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbec659c640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800d950 tx=0x7fbeb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb8017070 con 0x7fbec81031d0 2024-04-23T19:07:42.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeb8009c30 con 0x7fbec81031d0 2024-04-23T19:07:42.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec819d930 con 0x7fbec81031d0 2024-04-23T19:07:42.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800f620 con 0x7fbec81031d0 2024-04-23T19:07:42.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.594+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec81a51c0 con 0x7fbec81031d0 2024-04-23T19:07:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.596+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec80691c0 con 0x7fbec81031d0 2024-04-23T19:07:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.597+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbeb8010460 con 0x7fbec81031d0 2024-04-23T19:07:42.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.597+0000 7fbebf7fe640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 0x7fbeb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:42.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.597+0000 7fbebffff640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 0x7fbeb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:42.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.597+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7fbeb80d0470 con 0x7fbec81031d0 2024-04-23T19:07:42.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.598+0000 7fbebffff640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 0x7fbeb40a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbec810e3b0 tx=0x7fbeb000b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.602+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeb809cb00 con 0x7fbec81031d0 2024-04-23T19:07:43.050 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:40 smithi033 ceph-mon[29274]: pgmap v1349: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-23T19:07:43.085 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:41 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/240785723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T19:07:43.125 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:42 smithi122 ceph-mon[32546]: pgmap v1350: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:43.133 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:41 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/240785723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T19:07:43.142 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:42 smithi106 ceph-mon[32716]: pgmap v1350: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:43.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.789+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbec802d5c0 con 0x7fbec81031d0 2024-04-23T19:07:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.790+0000 7fbebf7fe640 1 -- 172.21.15.33:0/240785723 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) ==== 104+0+11 (secure 0 0 0) 0x7fbeb8009de0 con 0x7fbec81031d0 2024-04-23T19:07:43.340 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-23T19:07:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 msgr2=0x7fbeb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:43.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 0x7fbeb40a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbec810e3b0 tx=0x7fbeb000b390 comp rx=0 tx=0).stop 2024-04-23T19:07:43.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 msgr2=0x7fbec81986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fbeb800d950 tx=0x7fbeb800de10 comp rx=0 tx=0).stop 2024-04-23T19:07:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 shutdown_connections 2024-04-23T19:07:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbeb40a3d20 0x7fbeb40a61e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbec810c510 0x7fbec819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbec8103af0 0x7fbec8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 --2- 172.21.15.33:0/240785723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbec81031d0 0x7fbec81986a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.791+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 >> 172.21.15.33:0/240785723 conn(0x7fbec80771d0 msgr2=0x7fbec80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.792+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 shutdown_connections 2024-04-23T19:07:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:40.792+0000 7fbecc8dc640 1 -- 172.21.15.33:0/240785723 wait complete. 2024-04-23T19:07:43.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-23T19:07:44.017 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:41 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/240785723' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-23T19:07:44.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:42 smithi033 ceph-mon[29274]: pgmap v1350: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 -- 172.21.15.33:0/1554009063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60102b40 msgr2=0x7fef60106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:44.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 --2- 172.21.15.33:0/1554009063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60102b40 0x7fef60106b80 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fef54009900 tx=0x7fef5402a410 comp rx=0 tx=0).stop 2024-04-23T19:07:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 -- 172.21.15.33:0/1554009063 shutdown_connections 2024-04-23T19:07:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 --2- 172.21.15.33:0/1554009063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60107260 0x7fef6010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:44.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 --2- 172.21.15.33:0/1554009063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60102b40 0x7fef60106b80 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:44.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 --2- 172.21.15.33:0/1554009063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60102190 0x7fef60102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 -- 172.21.15.33:0/1554009063 >> 172.21.15.33:0/1554009063 conn(0x7fef600fd990 msgr2=0x7fef600ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 -- 172.21.15.33:0/1554009063 shutdown_connections 2024-04-23T19:07:44.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.299+0000 7fef65ecf640 1 -- 172.21.15.33:0/1554009063 wait complete. 2024-04-23T19:07:44.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 Processor -- start 2024-04-23T19:07:44.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 -- start start 2024-04-23T19:07:44.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:44 smithi122 ceph-mon[32546]: pgmap v1351: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:44.731 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:44 smithi106 ceph-mon[32716]: pgmap v1351: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:44.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60102190 0x7fef60074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 0x7fef60074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:44.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef60112180 con 0x7fef60074cd0 2024-04-23T19:07:44.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef60112000 con 0x7fef60102190 2024-04-23T19:07:44.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef65ecf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fef60112300 con 0x7fef60107260 2024-04-23T19:07:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:45.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47518/0 (socket says 172.21.15.33:47518) 2024-04-23T19:07:45.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef5ffff640 1 -- 172.21.15.33:0/172467121 learned_addr learned my addr 172.21.15.33:0/172467121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:45.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef5f7fe640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60102190 0x7fef60074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:45.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.300+0000 7fef5effd640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 0x7fef60074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:45.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 -- 172.21.15.33:0/172467121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 msgr2=0x7fef60074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:45.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 0x7fef60074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:45.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 -- 172.21.15.33:0/172467121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60102190 msgr2=0x7fef60074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:45.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60102190 0x7fef60074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:45.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 -- 172.21.15.33:0/172467121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef601a7030 con 0x7fef60074cd0 2024-04-23T19:07:45.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5effd640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 0x7fef60074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-23T19:07:45.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5ffff640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fef5800eaa0 tx=0x7fef5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:45.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5800cdb0 con 0x7fef60074cd0 2024-04-23T19:07:45.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef58005020 con 0x7fef60074cd0 2024-04-23T19:07:45.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef601a7320 con 0x7fef60074cd0 2024-04-23T19:07:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef580116d0 con 0x7fef60074cd0 2024-04-23T19:07:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.301+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef601a7820 con 0x7fef60074cd0 2024-04-23T19:07:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.302+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef600691c0 con 0x7fef60074cd0 2024-04-23T19:07:45.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.303+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef580051d0 con 0x7fef60074cd0 2024-04-23T19:07:46.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.304+0000 7fef5cff9640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 0x7fef480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:46.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.304+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7fef580d0a10 con 0x7fef60074cd0 2024-04-23T19:07:46.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.304+0000 7fef5f7fe640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 0x7fef480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:46.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.304+0000 7fef5f7fe640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 0x7fef480a6210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef500058c0 tx=0x7fef5000a4c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.309+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef58016030 con 0x7fef60074cd0 2024-04-23T19:07:46.474 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:44 smithi033 ceph-mon[29274]: pgmap v1351: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:46.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.496+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef60102b40 con 0x7fef60074cd0 2024-04-23T19:07:46.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.497+0000 7fef5cff9640 1 -- 172.21.15.33:0/172467121 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) ==== 100+0+11 (secure 0 0 0) 0x7fef5809d0a0 con 0x7fef60074cd0 2024-04-23T19:07:46.555 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-23T19:07:46.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 msgr2=0x7fef480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:46.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 0x7fef480a6210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef500058c0 tx=0x7fef5000a4c0 comp rx=0 tx=0).stop 2024-04-23T19:07:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 msgr2=0x7fef601a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fef5800eaa0 tx=0x7fef5800ef60 comp rx=0 tx=0).stop 2024-04-23T19:07:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 shutdown_connections 2024-04-23T19:07:46.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fef480a3d50 0x7fef480a6210 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef60074cd0 0x7fef601a6930 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:47.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef60107260 0x7fef60074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:47.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 --2- 172.21.15.33:0/172467121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef60102190 0x7fef60074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:47.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 >> 172.21.15.33:0/172467121 conn(0x7fef600fd990 msgr2=0x7fef600fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:47.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 shutdown_connections 2024-04-23T19:07:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:44.499+0000 7fef65ecf640 1 -- 172.21.15.33:0/172467121 wait complete. 2024-04-23T19:07:47.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-23T19:07:47.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:45 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/172467121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T19:07:47.453 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:46 smithi033 ceph-mon[29274]: pgmap v1352: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:47.525 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:45 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/172467121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T19:07:47.642 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:46 smithi122 ceph-mon[32546]: pgmap v1352: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:47.651 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:45 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/172467121' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-23T19:07:47.660 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:46 smithi106 ceph-mon[32716]: pgmap v1352: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 -- 172.21.15.33:0/3955177891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0069d00 msgr2=0x7f99e0106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 --2- 172.21.15.33:0/3955177891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0069d00 0x7f99e0106b60 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009900 tx=0x7f99d002a410 comp rx=0 tx=0).stop 2024-04-23T19:07:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 -- 172.21.15.33:0/3955177891 shutdown_connections 2024-04-23T19:07:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 --2- 172.21.15.33:0/3955177891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e0107240 0x7f99e010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:47.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 --2- 172.21.15.33:0/3955177891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0069d00 0x7f99e0106b60 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 --2- 172.21.15.33:0/3955177891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 0x7f99e0069730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 -- 172.21.15.33:0/3955177891 >> 172.21.15.33:0/3955177891 conn(0x7f99e0077cb0 msgr2=0x7f99e00780c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:48.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 -- 172.21.15.33:0/3955177891 shutdown_connections 2024-04-23T19:07:48.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 -- 172.21.15.33:0/3955177891 wait complete. 2024-04-23T19:07:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.640+0000 7f99e7423640 1 Processor -- start 2024-04-23T19:07:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 -- start start 2024-04-23T19:07:48.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 0x7f99e0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:48.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 0x7f99e019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:48.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99e01121e0 con 0x7f99e0107240 2024-04-23T19:07:48.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112060 con 0x7f99e019d020 2024-04-23T19:07:48.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e7423640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99e0112360 con 0x7f99e0069350 2024-04-23T19:07:48.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52976/0 (socket says 172.21.15.33:52976) 2024-04-23T19:07:48.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 -- 172.21.15.33:0/2912134870 learned_addr learned my addr 172.21.15.33:0/2912134870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:48.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e59c0640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 0x7f99e019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e51bf640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 0x7f99e0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 -- 172.21.15.33:0/2912134870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 msgr2=0x7f99e0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:49.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 0x7f99e0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:49.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 -- 172.21.15.33:0/2912134870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 msgr2=0x7f99e019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:49.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 0x7f99e019d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:49.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 -- 172.21.15.33:0/2912134870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e019db10 con 0x7f99e0107240 2024-04-23T19:07:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e59c0640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 0x7f99e019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:07:49.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.641+0000 7f99e49be640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006010 tx=0x7f99d0004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:49.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.642+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d003f030 con 0x7f99e0107240 2024-04-23T19:07:49.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.642+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e019dda0 con 0x7f99e0107240 2024-04-23T19:07:49.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.642+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99d0003aa0 con 0x7f99e0107240 2024-04-23T19:07:49.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.642+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d0035070 con 0x7f99e0107240 2024-04-23T19:07:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.642+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e01a5570 con 0x7f99e0107240 2024-04-23T19:07:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.643+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e0068d70 con 0x7f99e0107240 2024-04-23T19:07:49.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.644+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f99d0042030 con 0x7f99e0107240 2024-04-23T19:07:49.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.644+0000 7f99d67fc640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.645+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f99d00f10e0 con 0x7f99e0107240 2024-04-23T19:07:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.645+0000 7f99e51bf640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:49.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.645+0000 7f99e51bf640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009890 tx=0x7f99dc008040 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:50.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.648+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99d00bd770 con 0x7f99e0107240 2024-04-23T19:07:50.177 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:48 smithi122 ceph-mon[32546]: pgmap v1353: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:50.213 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:48 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2912134870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T19:07:50.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:48 smithi106 ceph-mon[32716]: pgmap v1353: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:50.286 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:48 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2912134870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T19:07:50.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.849+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f99e0199ab0 con 0x7f99e0107240 2024-04-23T19:07:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.849+0000 7f99d67fc640 1 -- 172.21.15.33:0/2912134870 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v169) ==== 81+0+944 (secure 0 0 0) 0x7f99e0199ab0 con 0x7f99e0107240 2024-04-23T19:07:50.556 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/851272344 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:50.643 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4092751446 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:50.745 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2434561620 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:50.775 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2158874591 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:50.809 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4245296611 2024-04-24T18:17:24.067933+0000 2024-04-23T19:07:50.875 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4110897883 2024-04-24T18:17:24.067933+0000 2024-04-23T19:07:50.934 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2953329503 2024-04-24T18:17:24.067933+0000 2024-04-23T19:07:50.979 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2158874591 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:50.987 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1526532469 2024-04-24T18:16:57.302656+0000 2024-04-23T19:07:50.997 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2988933181 2024-04-24T18:26:29.462803+0000 2024-04-23T19:07:51.091 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4245296611 2024-04-24T18:17:24.067933+0000 2024-04-23T19:07:51.164 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4072285085 2024-04-24T18:16:57.302656+0000 2024-04-23T19:07:51.207 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/438669199 2024-04-24T18:16:57.302656+0000 2024-04-23T19:07:51.253 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1526532469 2024-04-24T18:16:57.302656+0000 2024-04-23T19:07:51.277 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3363990252 2024-04-24T18:16:57.302656+0000 2024-04-23T19:07:51.285 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1895522836 2024-04-24T18:17:24.067933+0000 2024-04-23T19:07:51.294 INFO:teuthology.orchestra.run.smithi033.stderr:listed 16 entries 2024-04-23T19:07:51.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 msgr2=0x7f99cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009890 tx=0x7f99dc008040 comp rx=0 tx=0).stop 2024-04-23T19:07:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 msgr2=0x7f99e0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:51.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006010 tx=0x7f99d0004060 comp rx=0 tx=0).stop 2024-04-23T19:07:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 shutdown_connections 2024-04-23T19:07:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f99cc0a3d20 0x7f99cc0a61e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:51.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f99e019d020 0x7f99e019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:51.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99e0107240 0x7f99e0198ca0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:51.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 --2- 172.21.15.33:0/2912134870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e0069350 0x7f99e0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:51.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 >> 172.21.15.33:0/2912134870 conn(0x7f99e0077cb0 msgr2=0x7f99e00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:51.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 shutdown_connections 2024-04-23T19:07:52.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:47.852+0000 7f99e7423640 1 -- 172.21.15.33:0/2912134870 wait complete. 2024-04-23T19:07:52.141 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-23T19:07:52.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:52.230 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-23T19:07:52.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:48 smithi033 ceph-mon[29274]: pgmap v1353: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:52.336 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:48 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2912134870' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-23T19:07:52.397 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:50 smithi033 ceph-mon[29274]: pgmap v1354: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:52.504 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:50 smithi122 ceph-mon[32546]: pgmap v1354: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:52.566 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:50 smithi106 ceph-mon[32716]: pgmap v1354: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:52.601 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T19:07:52.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:52.697 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-23T19:07:52.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:52 smithi122 ceph-mon[32546]: pgmap v1355: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:52.827 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:52 smithi106 ceph-mon[32716]: pgmap v1355: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:52 smithi033 ceph-mon[29274]: pgmap v1355: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-23T19:07:53.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:53.106 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-23T19:07:53.165 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-23T19:07:53.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:53.254 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-23T19:07:53.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:53.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-23T19:07:53.479 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-23T19:07:53.510 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-23T19:07:53.565 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-23T19:07:53.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7153 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/40f2c761cc105a358eacd597617db920e5f096bf5e6475c0a2b018f36eda497c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/40f2c761cc105a358eacd597617db920e5f096bf5e6475c0a2b018f36eda497c/diff,workdir=/var/lib/containers/storage/overlay/40f2c761cc105a358eacd597617db920e5f096bf5e6475c0a2b018f36eda497c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/09435050a3003b78d041b9d028013c8e2ae8d94eccb648e9963c9e4f0e05eeac/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/09435050a3003b78d041b9d028013c8e2ae8d94eccb648e9963c9e4f0e05eeac/diff,workdir=/var/lib/containers/storage/overlay/09435050a3003b78d041b9d028013c8e2ae8d94eccb648e9963c9e4f0e05eeac/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/699a2d3b68766d213848a5911e1be35bf8f1bdb992f79171542cf78bbd846a9e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/699a2d3b68766d213848a5911e1be35bf8f1bdb992f79171542cf78bbd846a9e/diff,workdir=/var/lib/containers/storage/overlay/699a2d3b68766d213848a5911e1be35bf8f1bdb992f79171542cf78bbd846a9e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/51fd88f0083a09094653f7a55e2edd52258adc0d3df0ea8fe8a7319521b9744b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/51fd88f0083a09094653f7a55e2edd52258adc0d3df0ea8fe8a7319521b9744b/diff,workdir=/var/lib/containers/storage/overlay/51fd88f0083a09094653f7a55e2edd52258adc0d3df0ea8fe8a7319521b9744b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/89a3126444ccbc415581735a487e3d10e74b504f83c848a2b6feeb1b03d2629f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/89a3126444ccbc415581735a487e3d10e74b504f83c848a2b6feeb1b03d2629f/diff,workdir=/var/lib/containers/storage/overlay/89a3126444ccbc415581735a487e3d10e74b504f83c848a2b6feeb1b03d2629f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/54e1e8482d47825b1d96dacda501b122566a2dc668ee21906979b11f97fbf9f8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/54e1e8482d47825b1d96dacda501b122566a2dc668ee21906979b11f97fbf9f8/diff,workdir=/var/lib/containers/storage/overlay/54e1e8482d47825b1d96dacda501b122566a2dc668ee21906979b11f97fbf9f8/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/48e309a819fdfa36ec6457a815f58dfd8cf37978d298d037ba47d07931bea22a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/48e309a819fdfa36ec6457a815f58dfd8cf37978d298d037ba47d07931bea22a/diff,workdir=/var/lib/containers/storage/overlay/48e309a819fdfa36ec6457a815f58dfd8cf37978d298d037ba47d07931bea22a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/69a8ed8282f2e0c23310a0907e61a857a86a1eaf8991505231629c06505140dd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/69a8ed8282f2e0c23310a0907e61a857a86a1eaf8991505231629c06505140dd/diff,workdir=/var/lib/containers/storage/overlay/69a8ed8282f2e0c23310a0907e61a857a86a1eaf8991505231629c06505140dd/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 overlay /var/lib/containers/storage/overlay/067da376041b1883d073c8a14d23ee3802b2a8f552bf4e6789119200f1aa68c9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/YC3WSLPB5T66LMN7LSHAXQUVNQ:/var/lib/containers/storage/overlay/l/N6TDRHKX6ZJRUZ5KC3FBE4HVDO,upperdir=/var/lib/containers/storage/overlay/067da376041b1883d073c8a14d23ee3802b2a8f552bf4e6789119200f1aa68c9/diff,workdir=/var/lib/containers/storage/overlay/067da376041b1883d073c8a14d23ee3802b2a8f552bf4e6789119200f1aa68c9/work,metacopy=on,volatile 0 0 2024-04-23T19:07:53.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:53.856 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-23T19:07:53.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-23T19:07:54.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-23T19:07:54.086 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-04-23T19:07:54.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:07:54.105 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-23T19:07:54.226 INFO:tasks.cephadm:Teardown begin 2024-04-23T19:07:54.265 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-23T19:07:54.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T19:07:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.504+0000 7f3de6fcd640 1 -- 172.21.15.33:0/2796898905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de010c570 msgr2=0x7f3de010e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.504+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/2796898905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de010c570 0x7f3de010e9e0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f3dd8009ca0 tx=0x7f3dd802a950 comp rx=0 tx=0).stop 2024-04-23T19:07:54.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.504+0000 7f3de6fcd640 1 -- 172.21.15.33:0/2796898905 shutdown_connections 2024-04-23T19:07:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/2796898905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de010c570 0x7f3de010e9e0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:54.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/2796898905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103b50 0x7f3de010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/2796898905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de0103230 0x7f3de0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:54.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- 172.21.15.33:0/2796898905 >> 172.21.15.33:0/2796898905 conn(0x7f3de0077fb0 msgr2=0x7f3de00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:54.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- 172.21.15.33:0/2796898905 shutdown_connections 2024-04-23T19:07:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- 172.21.15.33:0/2796898905 wait complete. 2024-04-23T19:07:54.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 Processor -- start 2024-04-23T19:07:54.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- start start 2024-04-23T19:07:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103230 0x7f3de0100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:54.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de010c570 0x7f3de0101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:55.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3de01121d0 con 0x7f3de0103b50 2024-04-23T19:07:55.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0112050 con 0x7f3de0103230 2024-04-23T19:07:55.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.505+0000 7f3de6fcd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3de0112350 con 0x7f3de010c570 2024-04-23T19:07:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:55.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52992/0 (socket says 172.21.15.33:52992) 2024-04-23T19:07:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 -- 172.21.15.33:0/4134723787 learned_addr learned my addr 172.21.15.33:0/4134723787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:55.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3de556a640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de010c570 0x7f3de0101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:55.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3de4d69640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103230 0x7f3de0100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:55.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 -- 172.21.15.33:0/4134723787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de010c570 msgr2=0x7f3de0101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:55.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de010c570 0x7f3de0101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:55.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 -- 172.21.15.33:0/4134723787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103230 msgr2=0x7f3de0100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:55.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103230 0x7f3de0100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:55.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 -- 172.21.15.33:0/4134723787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de01a6b10 con 0x7f3de0103b50 2024-04-23T19:07:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3ddffff640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c920 tx=0x7f3dc800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:55.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8007c70 con 0x7f3de0103b50 2024-04-23T19:07:55.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc8007e10 con 0x7f3de0103b50 2024-04-23T19:07:55.845 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:54 smithi122 ceph-mon[32546]: pgmap v1356: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:55.905 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:55 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/4134723787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:07:55.985 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:54 smithi106 ceph-mon[32716]: pgmap v1356: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:56.122 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:55 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/4134723787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:07:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de01a6e00 con 0x7f3de0103b50 2024-04-23T19:07:56.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.506+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8018820 con 0x7f3de0103b50 2024-04-23T19:07:56.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.507+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de01a7270 con 0x7f3de0103b50 2024-04-23T19:07:56.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.508+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de00691c0 con 0x7f3de0103b50 2024-04-23T19:07:56.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.509+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3dc800f450 con 0x7f3de0103b50 2024-04-23T19:07:56.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.509+0000 7f3dddffb640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 0x7f3dcc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:56.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.510+0000 7f3de4d69640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 0x7f3dcc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:56.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.510+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f3dc80d0760 con 0x7f3de0103b50 2024-04-23T19:07:56.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.510+0000 7f3de4d69640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 0x7f3dcc0a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005fd0 tx=0x7f3dd000b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:56.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.514+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3dc8014030 con 0x7f3de0103b50 2024-04-23T19:07:56.681 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:54 smithi033 ceph-mon[29274]: pgmap v1356: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:56.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.742+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3de0102360 con 0x7f3de0103b50 2024-04-23T19:07:56.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.743+0000 7f3dddffb640 1 -- 172.21.15.33:0/4134723787 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+10274 (secure 0 0 0) 0x7f3dc809cdf0 con 0x7f3de0103b50 2024-04-23T19:07:56.724 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T19:07:56.732 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:58:16.088238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_18756":{"gid":18756,"name":"j","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/217544075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":217544075},{"type":"v1","addr":"172.21.15.33:6835","nonce":217544075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24466,24482,24551,24490]},"id":1}]} 2024-04-23T19:07:56.871 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:56 smithi122 ceph-mon[32546]: pgmap v1357: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:56.939 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:56 smithi106 ceph-mon[32716]: pgmap v1357: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:57.032 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 50 2024-04-23T19:07:57.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 msgr2=0x7f3dcc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:57.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 0x7f3dcc0a6210 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005fd0 tx=0x7f3dd000b390 comp rx=0 tx=0).stop 2024-04-23T19:07:57.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 msgr2=0x7f3de01013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c920 tx=0x7f3dc800cde0 comp rx=0 tx=0).stop 2024-04-23T19:07:57.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 shutdown_connections 2024-04-23T19:07:57.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f3dcc0a3d50 0x7f3dcc0a6210 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3de010c570 0x7f3de0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:57.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3de0103b50 0x7f3de01013c0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:57.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 --2- 172.21.15.33:0/4134723787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3de0103230 0x7f3de0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 >> 172.21.15.33:0/4134723787 conn(0x7f3de0077fb0 msgr2=0x7f3de00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:57.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 shutdown_connections 2024-04-23T19:07:57.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:54.745+0000 7f3de6fcd640 1 -- 172.21.15.33:0/4134723787 wait complete. 2024-04-23T19:07:57.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-23T19:07:57.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:55 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/4134723787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:07:57.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:56 smithi033 ceph-mon[29274]: pgmap v1357: 97 pgs: 97 active+clean; 15 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-23T19:07:58.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 -- 172.21.15.33:0/158803999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024102bf0 msgr2=0x7f8024106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:58.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 --2- 172.21.15.33:0/158803999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024102bf0 0x7f8024106ba0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f800c009900 tx=0x7f800c02a410 comp rx=0 tx=0).stop 2024-04-23T19:07:58.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 -- 172.21.15.33:0/158803999 shutdown_connections 2024-04-23T19:07:58.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 --2- 172.21.15.33:0/158803999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024107280 0x7f802410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:58.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 --2- 172.21.15.33:0/158803999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024102bf0 0x7f8024106ba0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:58.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 --2- 172.21.15.33:0/158803999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024102240 0x7f8024102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 -- 172.21.15.33:0/158803999 >> 172.21.15.33:0/158803999 conn(0x7f80240790e0 msgr2=0x7f80240794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:07:58.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 -- 172.21.15.33:0/158803999 shutdown_connections 2024-04-23T19:07:58.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.066+0000 7f8028c3a640 1 -- 172.21.15.33:0/158803999 wait complete. 2024-04-23T19:07:58.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 Processor -- start 2024-04-23T19:07:58.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 -- start start 2024-04-23T19:07:58.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 0x7f8024074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:58.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:58.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024074cd0 0x7f80241a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:07:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80241121a0 con 0x7f8024107280 2024-04-23T19:07:58.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8024112020 con 0x7f8024074cd0 2024-04-23T19:07:58.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f8028c3a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8024112320 con 0x7f8024102240 2024-04-23T19:07:58.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f801bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:58.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f801bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49126/0 (socket says 172.21.15.33:49126) 2024-04-23T19:07:58.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.067+0000 7f801bfff640 1 -- 172.21.15.33:0/2211493340 learned_addr learned my addr 172.21.15.33:0/2211493340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:07:59.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f8022d9d640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024074cd0 0x7f80241a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:59.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f802259c640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 0x7f8024074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:07:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 -- 172.21.15.33:0/2211493340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 msgr2=0x7f8024074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:59.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 0x7f8024074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 -- 172.21.15.33:0/2211493340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024074cd0 msgr2=0x7f80241a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:07:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024074cd0 0x7f80241a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:07:59.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 -- 172.21.15.33:0/2211493340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80241a7030 con 0x7f8024107280 2024-04-23T19:07:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f802259c640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 0x7f8024074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:07:59.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801bfff640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f800c02a930 tx=0x7f800c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:07:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c03f030 con 0x7f8024107280 2024-04-23T19:07:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800c003aa0 con 0x7f8024107280 2024-04-23T19:07:59.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80241a72c0 con 0x7f8024107280 2024-04-23T19:07:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c035070 con 0x7f8024107280 2024-04-23T19:07:59.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.068+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80241a77b0 con 0x7f8024107280 2024-04-23T19:07:59.912 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:58 smithi122 ceph-mon[32546]: pgmap v1358: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:00.138 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:07:58 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2211493340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:08:00.197 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:58 smithi106 ceph-mon[32716]: pgmap v1358: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:00.272 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:07:58 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2211493340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:08:00.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.069+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80240691c0 con 0x7f8024107280 2024-04-23T19:08:00.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.071+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f800c042030 con 0x7f8024107280 2024-04-23T19:08:00.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.071+0000 7f801b7fe640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 0x7f80100a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:00.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.072+0000 7f802259c640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 0x7f80100a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.072+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f800c0f0e90 con 0x7f8024107280 2024-04-23T19:08:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.072+0000 7f802259c640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 0x7f80100a6210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f80140058c0 tx=0x7f8014005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:00.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.076+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f800c0bd520 con 0x7f8024107280 2024-04-23T19:08:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.303+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802410a4a0 con 0x7f8024107280 2024-04-23T19:08:00.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.304+0000 7f801b7fe640 1 -- 172.21.15.33:0/2211493340 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+10274 (secure 0 0 0) 0x7f800c0bce20 con 0x7f8024107280 2024-04-23T19:08:00.713 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T19:08:00.722 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-23T18:24:26.661481+0000","modified":"2024-04-23T18:58:16.088238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24551,"mds_1":24515,"mds_2":24490,"mds_3":24482,"mds_4":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14628":{"gid":14628,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/1373616976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":1373616976},{"type":"v1","addr":"172.21.15.33:6839","nonce":1373616976}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14640":{"gid":14640,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/363309287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":363309287},{"type":"v1","addr":"172.21.15.33:6841","nonce":363309287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_18756":{"gid":18756,"name":"j","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6835/217544075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":217544075},{"type":"v1","addr":"172.21.15.33:6835","nonce":217544075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6833/2194426420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2194426420},{"type":"v1","addr":"172.21.15.122:6833","nonce":2194426420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24446":{"gid":24446,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6833/637332963","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":637332963},{"type":"v1","addr":"172.21.15.106:6833","nonce":637332963}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6837/3846576090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":3846576090},{"type":"v1","addr":"172.21.15.122:6837","nonce":3846576090}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":9,"addr":"172.21.15.106:6835/3607877814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3607877814},{"type":"v1","addr":"172.21.15.106:6835","nonce":3607877814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6839/538103378","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":538103378},{"type":"v1","addr":"172.21.15.122:6839","nonce":538103378}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24515":{"gid":24515,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.106:6837/319312146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":319312146},{"type":"v1","addr":"172.21.15.106:6837","nonce":319312146}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24551":{"gid":24551,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/830372376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":830372376},{"type":"v1","addr":"172.21.15.106:6839","nonce":830372376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24551,"qdb_cluster":[24515,24466,24482,24551,24490]},"id":1}]} 2024-04-23T19:08:00.812 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 50 2024-04-23T19:08:00.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 msgr2=0x7f80100a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:00.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 0x7f80100a6210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f80140058c0 tx=0x7f8014005850 comp rx=0 tx=0).stop 2024-04-23T19:08:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 msgr2=0x7f8024074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f800c02a930 tx=0x7f800c004270 comp rx=0 tx=0).stop 2024-04-23T19:08:01.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 shutdown_connections 2024-04-23T19:08:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f80100a3d50 0x7f80100a6210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:01.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8024074cd0 0x7f80241a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:01.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8024107280 0x7f8024074790 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 --2- 172.21.15.33:0/2211493340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8024102240 0x7f8024074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:01.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 >> 172.21.15.33:0/2211493340 conn(0x7f80240790e0 msgr2=0x7f8024100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:01.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 shutdown_connections 2024-04-23T19:08:01.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:07:58.306+0000 7f8028c3a640 1 -- 172.21.15.33:0/2211493340 wait complete. 2024-04-23T19:08:01.711 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:00 smithi122 ceph-mon[32546]: pgmap v1359: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:01.741 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:00 smithi106 ceph-mon[32716]: pgmap v1359: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:01.801 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-23T19:08:01.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:58 smithi033 ceph-mon[29274]: pgmap v1358: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:01.994 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:07:58 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2211493340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-23T19:08:02.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:00 smithi033 ceph-mon[29274]: pgmap v1359: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:02.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 -- 172.21.15.33:0/1428076069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474102bf0 msgr2=0x7f8474106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:02.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 --2- 172.21.15.33:0/1428076069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474102bf0 0x7f8474106ba0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8464009900 tx=0x7f846402a410 comp rx=0 tx=0).stop 2024-04-23T19:08:02.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 -- 172.21.15.33:0/1428076069 shutdown_connections 2024-04-23T19:08:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 --2- 172.21.15.33:0/1428076069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474107280 0x7f847410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:02.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 --2- 172.21.15.33:0/1428076069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474102bf0 0x7f8474106ba0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 --2- 172.21.15.33:0/1428076069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8474102240 0x7f8474102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 -- 172.21.15.33:0/1428076069 >> 172.21.15.33:0/1428076069 conn(0x7f84740790e0 msgr2=0x7f84740794f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:02.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 -- 172.21.15.33:0/1428076069 shutdown_connections 2024-04-23T19:08:02.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.187+0000 7f847c175640 1 -- 172.21.15.33:0/1428076069 wait complete. 2024-04-23T19:08:02.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 Processor -- start 2024-04-23T19:08:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 -- start start 2024-04-23T19:08:02.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474102240 0x7f8474198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:03.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:03.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f847419d020 0x7f847419d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:03.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84741121a0 con 0x7f8474107280 2024-04-23T19:08:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8474112020 con 0x7f8474102240 2024-04-23T19:08:03.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f847c175640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8474112320 con 0x7f847419d020 2024-04-23T19:08:03.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f8479710640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:03.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.188+0000 7f8479710640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49132/0 (socket says 172.21.15.33:49132) 2024-04-23T19:08:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 -- 172.21.15.33:0/1411513081 learned_addr learned my addr 172.21.15.33:0/1411513081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:08:03.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f847a712640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f847419d020 0x7f847419d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:03.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479f11640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474102240 0x7f8474198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 -- 172.21.15.33:0/1411513081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f847419d020 msgr2=0x7f847419d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:03.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f847419d020 0x7f847419d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:03.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 -- 172.21.15.33:0/1411513081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474102240 msgr2=0x7f8474198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474102240 0x7f8474198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:03.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 -- 172.21.15.33:0/1411513081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847419db10 con 0x7f8474107280 2024-04-23T19:08:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f8479710640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f846402a930 tx=0x7f8464004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:03.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846403f030 con 0x7f8474107280 2024-04-23T19:08:04.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464003aa0 con 0x7f8474107280 2024-04-23T19:08:04.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847419dda0 con 0x7f8474107280 2024-04-23T19:08:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464035070 con 0x7f8474107280 2024-04-23T19:08:04.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.189+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84741a55c0 con 0x7f8474107280 2024-04-23T19:08:04.406 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:02 smithi122 ceph-mon[32546]: pgmap v1360: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:04.465 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:03 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1411513081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:08:04.570 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:02 smithi106 ceph-mon[32716]: pgmap v1360: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:04.691 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:03 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1411513081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:08:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.190+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8474199ab0 con 0x7f8474107280 2024-04-23T19:08:04.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.191+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8464042030 con 0x7f8474107280 2024-04-23T19:08:04.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.192+0000 7f846affd640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 0x7f84600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:04.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.192+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f84640f1220 con 0x7f8474107280 2024-04-23T19:08:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.192+0000 7f8479f11640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 0x7f84600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:04.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.193+0000 7f8479f11640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 0x7f84600a6260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8470005fd0 tx=0x7f8470005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:04.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.196+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84640bd8b0 con 0x7f8474107280 2024-04-23T19:08:05.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.384+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8474102bf0 con 0x7f8474107280 2024-04-23T19:08:05.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.385+0000 7f846affd640 1 -- 172.21.15.33:0/1411513081 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) ==== 74+0+23195 (secure 0 0 0) 0x7f84640bd1b0 con 0x7f8474107280 2024-04-23T19:08:05.276 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-23T19:08:05.310 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":169,"fsid":"766ecab6-019d-11ef-bc93-c7b262605968","created":"2024-04-23T18:16:27.249730+0000","modified":"2024-04-23T19:07:16.212951+0000","last_up_change":"2024-04-23T18:23:36.505476+0000","last_in_change":"2024-04-23T18:23:16.099159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-23T18:19:48.156180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-23T18:24:24.905524+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-23T18:24:25.596587+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":76,"snap_epoch":168,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a02f5d4-0a5f-4cf7-a5d1-1887e04a575c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6803","nonce":3729126407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6805","nonce":3729126407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6809","nonce":3729126407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3729126407},{"type":"v1","addr":"172.21.15.33:6807","nonce":3729126407}]},"public_addr":"172.21.15.33:6803/3729126407","cluster_addr":"172.21.15.33:6805/3729126407","heartbeat_back_addr":"172.21.15.33:6809/3729126407","heartbeat_front_addr":"172.21.15.33:6807/3729126407","state":["exists","up"]},{"osd":1,"uuid":"c3bf75bc-fda4-44d5-be29-dbb282a8a56a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6801","nonce":1202694989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6803","nonce":1202694989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6807","nonce":1202694989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1202694989},{"type":"v1","addr":"172.21.15.106:6805","nonce":1202694989}]},"public_addr":"172.21.15.106:6801/1202694989","cluster_addr":"172.21.15.106:6803/1202694989","heartbeat_back_addr":"172.21.15.106:6807/1202694989","heartbeat_front_addr":"172.21.15.106:6805/1202694989","state":["exists","up"]},{"osd":2,"uuid":"77497994-8bdd-45c2-8521-089e1016f545","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6801","nonce":2314180839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6803","nonce":2314180839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6807","nonce":2314180839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":2314180839},{"type":"v1","addr":"172.21.15.122:6805","nonce":2314180839}]},"public_addr":"172.21.15.122:6801/2314180839","cluster_addr":"172.21.15.122:6803/2314180839","heartbeat_back_addr":"172.21.15.122:6807/2314180839","heartbeat_front_addr":"172.21.15.122:6805/2314180839","state":["exists","up"]},{"osd":3,"uuid":"70af3b74-4df3-4c39-8562-3b0052a1319f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6811","nonce":1512542347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6813","nonce":1512542347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6817","nonce":1512542347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1512542347},{"type":"v1","addr":"172.21.15.33:6815","nonce":1512542347}]},"public_addr":"172.21.15.33:6811/1512542347","cluster_addr":"172.21.15.33:6813/1512542347","heartbeat_back_addr":"172.21.15.33:6817/1512542347","heartbeat_front_addr":"172.21.15.33:6815/1512542347","state":["exists","up"]},{"osd":4,"uuid":"b8f450e4-1ae9-4833-90de-3668d806c829","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6809","nonce":2880981819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6811","nonce":2880981819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6815","nonce":2880981819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2880981819},{"type":"v1","addr":"172.21.15.106:6813","nonce":2880981819}]},"public_addr":"172.21.15.106:6809/2880981819","cluster_addr":"172.21.15.106:6811/2880981819","heartbeat_back_addr":"172.21.15.106:6815/2880981819","heartbeat_front_addr":"172.21.15.106:6813/2880981819","state":["exists","up"]},{"osd":5,"uuid":"a3bd5fd8-f114-476a-be25-de2b536d9601","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6809","nonce":2410305978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6811","nonce":2410305978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6815","nonce":2410305978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2410305978},{"type":"v1","addr":"172.21.15.122:6813","nonce":2410305978}]},"public_addr":"172.21.15.122:6809/2410305978","cluster_addr":"172.21.15.122:6811/2410305978","heartbeat_back_addr":"172.21.15.122:6815/2410305978","heartbeat_front_addr":"172.21.15.122:6813/2410305978","state":["exists","up"]},{"osd":6,"uuid":"c84631e0-7950-46a1-9e3a-24f56cab53f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6819","nonce":3512939411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6821","nonce":3512939411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6825","nonce":3512939411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3512939411},{"type":"v1","addr":"172.21.15.33:6823","nonce":3512939411}]},"public_addr":"172.21.15.33:6819/3512939411","cluster_addr":"172.21.15.33:6821/3512939411","heartbeat_back_addr":"172.21.15.33:6825/3512939411","heartbeat_front_addr":"172.21.15.33:6823/3512939411","state":["exists","up"]},{"osd":7,"uuid":"8a454eca-15b1-4305-bf15-24a47be985d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6817","nonce":4041578359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6819","nonce":4041578359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6823","nonce":4041578359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":4041578359},{"type":"v1","addr":"172.21.15.106:6821","nonce":4041578359}]},"public_addr":"172.21.15.106:6817/4041578359","cluster_addr":"172.21.15.106:6819/4041578359","heartbeat_back_addr":"172.21.15.106:6823/4041578359","heartbeat_front_addr":"172.21.15.106:6821/4041578359","state":["exists","up"]},{"osd":8,"uuid":"574d7ce1-62ac-4720-9f2a-d40cdf230ffc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6817","nonce":497939734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6819","nonce":497939734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6823","nonce":497939734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":497939734},{"type":"v1","addr":"172.21.15.122:6821","nonce":497939734}]},"public_addr":"172.21.15.122:6817/497939734","cluster_addr":"172.21.15.122:6819/497939734","heartbeat_back_addr":"172.21.15.122:6823/497939734","heartbeat_front_addr":"172.21.15.122:6821/497939734","state":["exists","up"]},{"osd":9,"uuid":"0449b46b-87c2-4174-b0fc-623efb4f453d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6827","nonce":3157866333}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6829","nonce":3157866333}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6833","nonce":3157866333}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3157866333},{"type":"v1","addr":"172.21.15.33:6831","nonce":3157866333}]},"public_addr":"172.21.15.33:6827/3157866333","cluster_addr":"172.21.15.33:6829/3157866333","heartbeat_back_addr":"172.21.15.33:6833/3157866333","heartbeat_front_addr":"172.21.15.33:6831/3157866333","state":["exists","up"]},{"osd":10,"uuid":"7fab0d97-f3b4-484d-8f60-12b5327a1a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6825","nonce":4120078808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6827","nonce":4120078808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6831","nonce":4120078808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":4120078808},{"type":"v1","addr":"172.21.15.106:6829","nonce":4120078808}]},"public_addr":"172.21.15.106:6825/4120078808","cluster_addr":"172.21.15.106:6827/4120078808","heartbeat_back_addr":"172.21.15.106:6831/4120078808","heartbeat_front_addr":"172.21.15.106:6829/4120078808","state":["exists","up"]},{"osd":11,"uuid":"95bc8f26-44c0-429c-893b-566a272d0dbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6825","nonce":3755562411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6827","nonce":3755562411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6831","nonce":3755562411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3755562411},{"type":"v1","addr":"172.21.15.122:6829","nonce":3755562411}]},"public_addr":"172.21.15.122:6825/3755562411","cluster_addr":"172.21.15.122:6827/3755562411","heartbeat_back_addr":"172.21.15.122:6831/3755562411","heartbeat_front_addr":"172.21.15.122:6829/3755562411","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:18:58.538147+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:22.199420+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:19:45.689855+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:09.797645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:34.587493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:20:59.651045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:25.363048+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:21:50.109770+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:14.975939+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:22:41.512789+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:08.004621+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-23T18:23:34.356447+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.2","mappings":[{"from":3,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.19","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/851272344":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/4092751446":"2024-04-24T18:26:29.462803+0000","172.21.15.33:0/2434561620":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6800/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4110897883":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/2953329503":"2024-04-24T18:17:24.067933+0000","172.21.15.33:6801/2158874591":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/2988933181":"2024-04-24T18:26:29.462803+0000","172.21.15.33:6801/4245296611":"2024-04-24T18:17:24.067933+0000","172.21.15.33:0/4072285085":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/438669199":"2024-04-24T18:16:57.302656+0000","172.21.15.33:6800/1526532469":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/3363990252":"2024-04-24T18:16:57.302656+0000","172.21.15.33:0/1895522836":"2024-04-24T18:17:24.067933+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":64,"length":1},{"begin":76,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-23T19:08:05.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 msgr2=0x7f84600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:05.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 0x7f84600a6260 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8470005fd0 tx=0x7f8470005850 comp rx=0 tx=0).stop 2024-04-23T19:08:05.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 msgr2=0x7f8474198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f846402a930 tx=0x7f8464004270 comp rx=0 tx=0).stop 2024-04-23T19:08:05.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 shutdown_connections 2024-04-23T19:08:05.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f84600a3da0 0x7f84600a6260 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:05.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f847419d020 0x7f847419d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:05.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8474107280 0x7f8474198ca0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:06.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 --2- 172.21.15.33:0/1411513081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8474102240 0x7f8474198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:06.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 >> 172.21.15.33:0/1411513081 conn(0x7f84740790e0 msgr2=0x7f8474100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 shutdown_connections 2024-04-23T19:08:06.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:02.388+0000 7f847c175640 1 -- 172.21.15.33:0/1411513081 wait complete. 2024-04-23T19:08:06.301 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:02 smithi033 ceph-mon[29274]: pgmap v1360: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 13 op/s 2024-04-23T19:08:06.370 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:04 smithi122 ceph-mon[32546]: pgmap v1361: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.402 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:04 smithi106 ceph-mon[32716]: pgmap v1361: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-23T19:08:06.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:03 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1411513081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-23T19:08:06.614 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:04 smithi033 ceph-mon[29274]: pgmap v1361: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.702 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:06 smithi106 ceph-mon[32716]: pgmap v1362: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.711 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:06 smithi122 ceph-mon[32546]: pgmap v1362: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.727 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:06 smithi033 ceph-mon[29274]: pgmap v1362: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-23T19:08:06.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1815414505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a241021a0 msgr2=0x7f6a24102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:06.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1815414505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a241021a0 0x7f6a24102580 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009900 tx=0x7f6a2002a410 comp rx=0 tx=0).stop 2024-04-23T19:08:06.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1815414505 shutdown_connections 2024-04-23T19:08:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1815414505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a24107270 0x7f6a2410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:07.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1815414505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24102b50 0x7f6a24106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:07.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1815414505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a241021a0 0x7f6a24102580 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:07.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1815414505 >> 172.21.15.33:0/1815414505 conn(0x7f6a240fda20 msgr2=0x7f6a240ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1815414505 shutdown_connections 2024-04-23T19:08:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.817+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1815414505 wait complete. 2024-04-23T19:08:07.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 Processor -- start 2024-04-23T19:08:07.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 -- start start 2024-04-23T19:08:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a241021a0 0x7f6a241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:07.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 0x7f6a24198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:07.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:07.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241121b0 con 0x7f6a2419cfb0 2024-04-23T19:08:07.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24112030 con 0x7f6a241021a0 2024-04-23T19:08:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2bce1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24112330 con 0x7f6a24107270 2024-04-23T19:08:07.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2a27e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:07.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2a27e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50572/0 (socket says 172.21.15.33:50572) 2024-04-23T19:08:07.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2a27e640 1 -- 172.21.15.33:0/1361465822 learned_addr learned my addr 172.21.15.33:0/1361465822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:08:07.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a29a7d640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a241021a0 0x7f6a241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:07.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.818+0000 7f6a2927c640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 0x7f6a24198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:08.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 -- 172.21.15.33:0/1361465822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 msgr2=0x7f6a24198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:08.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 0x7f6a24198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:08.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 -- 172.21.15.33:0/1361465822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a241021a0 msgr2=0x7f6a241986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:08.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a241021a0 0x7f6a241986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 -- 172.21.15.33:0/1361465822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2419daa0 con 0x7f6a2419cfb0 2024-04-23T19:08:08.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2927c640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 0x7f6a24198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:08:08.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2a27e640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009fd0 tx=0x7f6a1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:08.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c017070 con 0x7f6a2419cfb0 2024-04-23T19:08:08.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a1c003e20 con 0x7f6a2419cfb0 2024-04-23T19:08:08.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2419dd90 con 0x7f6a2419cfb0 2024-04-23T19:08:08.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c010460 con 0x7f6a2419cfb0 2024-04-23T19:08:08.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.819+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241a5530 con 0x7f6a2419cfb0 2024-04-23T19:08:08.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.820+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a240691c0 con 0x7f6a2419cfb0 2024-04-23T19:08:08.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.821+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a1c003970 con 0x7f6a2419cfb0 2024-04-23T19:08:08.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.822+0000 7f6a1affd640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:08.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.822+0000 7f6a29a7d640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:08.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.822+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 8542+0+0 (secure 0 0 0) 0x7f6a1c0d05b0 con 0x7f6a2419cfb0 2024-04-23T19:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.822+0000 7f6a29a7d640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 0x7f6a100a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a20031440 tx=0x7f6a200095f0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:09.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:06.825+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a1c09cc40 con 0x7f6a2419cfb0 2024-04-23T19:08:09.077 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:09.087 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:09.167 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mds[62651]: -- 172.21.15.122:0/2757919012 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7fa0ffa40 con 0x55c7fa0d3000 2024-04-23T19:08:09.246 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: starting mds.l at 2024-04-23T19:08:09.334 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:09.343 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:09.352 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mds[62670]: -- 172.21.15.106:0/4108952161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55693c049a40 con 0x55693c01c800 2024-04-23T19:08:09.402 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: starting mds.h at 2024-04-23T19:08:09.495 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c[63262]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:09.601 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c[63262]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:09.739 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mds[63268]: -- 172.21.15.122:0/3081935153 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bfb9589a40 con 0x55bfb955c800 2024-04-23T19:08:09.855 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c[63262]: starting mds.c at 2024-04-23T19:08:09.919 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:10.054 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:10.078 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mds[62040]: -- 172.21.15.106:0/2413937944 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fe1818ba40 con 0x55fe1815ec00 2024-04-23T19:08:10.086 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: starting mds.k at 2024-04-23T19:08:10.093 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:10.100 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:10.107 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mds[63572]: -- 172.21.15.122:0/2085296319 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6be2efa40 con 0x55d6be2c2c00 2024-04-23T19:08:10.232 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: starting mds.i at 2024-04-23T19:08:10.278 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:10.341 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:10.395 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mds[62360]: -- 172.21.15.106:0/2457808427 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f61dc7a40 con 0x556f61d9ac00 2024-04-23T19:08:10.402 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: starting mds.e at 2024-04-23T19:08:10.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T19:08:10.536 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T19:08:10.605 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-04-23T19:08:10.693 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T19:08:10.784 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T19:08:10.837 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: osdmap e170: 12 total, 12 up, 12 in 2024-04-23T19:08:10.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:07 smithi122 ceph-mon[32546]: fsmap cephfs:0/5, 5 failed 2024-04-23T19:08:10.947 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: pgmap v1363: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T19:08:11.000 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-04-23T19:08:11.034 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6834/2537941763,v1:172.21.15.33:6835/2537941763] up:boot 2024-04-23T19:08:11.043 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6832/645036474,v1:172.21.15.122:6833/645036474] up:boot 2024-04-23T19:08:11.066 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6832/3962646188,v1:172.21.15.106:6834/3962646188] up:boot 2024-04-23T19:08:11.161 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6833/4029605020,v1:172.21.15.106:6835/4029605020] up:boot 2024-04-23T19:08:11.255 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6836/4194336011,v1:172.21.15.33:6837/4194336011] up:boot 2024-04-23T19:08:11.288 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.33:6838/1890464907,v1:172.21.15.33:6839/1890464907] up:boot 2024-04-23T19:08:11.352 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6835/3127766675,v1:172.21.15.122:6837/3127766675] up:boot 2024-04-23T19:08:11.380 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.122:6834/4246706410,v1:172.21.15.122:6836/4246706410] up:boot 2024-04-23T19:08:11.432 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6836/2147655304,v1:172.21.15.106:6838/2147655304] up:boot 2024-04-23T19:08:11.489 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: mds.? [v2:172.21.15.106:6837/1859274513,v1:172.21.15.106:6839/1859274513] up:boot 2024-04-23T19:08:11.587 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: fsmap cephfs:0/5 10 up:standby, 5 failed 2024-04-23T19:08:11.714 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T19:08:11.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T19:08:11.771 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T19:08:11.843 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T19:08:11.886 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T19:08:11.929 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T19:08:11.991 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T19:08:12.031 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T19:08:12.040 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T19:08:12.110 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:08 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T19:08:12.193 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T19:08:12.256 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T19:08:12.284 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-04-23T19:08:12.326 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T19:08:12.386 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T19:08:12.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: osdmap e170: 12 total, 12 up, 12 in 2024-04-23T19:08:12.451 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:07 smithi106 ceph-mon[32716]: fsmap cephfs:0/5, 5 failed 2024-04-23T19:08:12.498 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: pgmap v1363: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T19:08:12.540 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-04-23T19:08:12.592 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6834/2537941763,v1:172.21.15.33:6835/2537941763] up:boot 2024-04-23T19:08:12.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6832/645036474,v1:172.21.15.122:6833/645036474] up:boot 2024-04-23T19:08:12.715 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6832/3962646188,v1:172.21.15.106:6834/3962646188] up:boot 2024-04-23T19:08:12.752 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6833/4029605020,v1:172.21.15.106:6835/4029605020] up:boot 2024-04-23T19:08:12.760 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6836/4194336011,v1:172.21.15.33:6837/4194336011] up:boot 2024-04-23T19:08:12.770 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.33:6838/1890464907,v1:172.21.15.33:6839/1890464907] up:boot 2024-04-23T19:08:12.779 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6835/3127766675,v1:172.21.15.122:6837/3127766675] up:boot 2024-04-23T19:08:12.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.122:6834/4246706410,v1:172.21.15.122:6836/4246706410] up:boot 2024-04-23T19:08:12.848 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6836/2147655304,v1:172.21.15.106:6838/2147655304] up:boot 2024-04-23T19:08:12.857 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: mds.? [v2:172.21.15.106:6837/1859274513,v1:172.21.15.106:6839/1859274513] up:boot 2024-04-23T19:08:12.867 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: fsmap cephfs:0/5 10 up:standby, 5 failed 2024-04-23T19:08:12.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T19:08:13.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T19:08:13.053 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T19:08:13.199 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T19:08:13.348 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T19:08:13.406 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T19:08:13.511 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T19:08:13.549 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T19:08:13.558 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T19:08:13.567 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:08 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T19:08:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.053+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6a24199a40 con 0x7f6a2419cfb0 2024-04-23T19:08:13.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.055+0000 7f6a1affd640 1 -- 172.21.15.33:0/1361465822 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) ==== 161+0+0 (secure 0 0 0) 0x7f6a24199a40 con 0x7f6a2419cfb0 2024-04-23T19:08:13.595 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-23T19:08:13.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.057+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 msgr2=0x7f6a100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:13.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.057+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 0x7f6a100a61e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a20031440 tx=0x7f6a200095f0 comp rx=0 tx=0).stop 2024-04-23T19:08:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.057+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 msgr2=0x7f6a2419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.057+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c009fd0 tx=0x7f6a1c00b260 comp rx=0 tx=0).stop 2024-04-23T19:08:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 shutdown_connections 2024-04-23T19:08:13.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f6a100a3d20 0x7f6a100a61e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a2419cfb0 0x7f6a2419d460 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:13.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a24107270 0x7f6a24198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 --2- 172.21.15.33:0/1361465822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a241021a0 0x7f6a241986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:14.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 >> 172.21.15.33:0/1361465822 conn(0x7f6a240fda20 msgr2=0x7f6a240fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:14.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 shutdown_connections 2024-04-23T19:08:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:07.058+0000 7f6a2bce1640 1 -- 172.21.15.33:0/1361465822 wait complete. 2024-04-23T19:08:14.220 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:10 smithi122 ceph-mon[32546]: pgmap v1365: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:08:14.271 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:12 smithi122 ceph-mon[32546]: pgmap v1366: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:14.298 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:10 smithi106 ceph-mon[32716]: pgmap v1365: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:08:14.372 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:12 smithi106 ceph-mon[32716]: pgmap v1366: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:14.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-23T19:08:14.483 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a[29270]: 2024-04-23T19:08:07.054+0000 7f46d5670700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T19:08:14.537 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:14.593 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:14.642 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mds[70895]: -- 172.21.15.33:0/497485708 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569b0835a40 con 0x5569b0808c00 2024-04-23T19:08:14.715 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: starting mds.a at 2024-04-23T19:08:14.836 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:14.874 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:15.047 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mds[70531]: -- 172.21.15.33:0/3406024725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b41c66c4e0 con 0x55b41c642c00 2024-04-23T19:08:15.161 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mds[70531]: -- 172.21.15.33:0/3406024725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b41c66fa40 con 0x55b41c642c00 2024-04-23T19:08:15.206 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: starting mds.g at 2024-04-23T19:08:15.238 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: ignoring --setuser ceph since I am not root 2024-04-23T19:08:15.302 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: ignoring --setgroup ceph since I am not root 2024-04-23T19:08:15.415 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mds[102979]: -- 172.21.15.33:0/3456478747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594abcfda40 con 0x5594abcd0800 2024-04-23T19:08:15.439 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: starting mds.j at 2024-04-23T19:08:15.450 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-23T19:08:15.459 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-23T19:08:15.467 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2024-04-23T19:08:15.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-23T19:08:15.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/1361465822' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-23T19:08:15.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: osdmap e170: 12 total, 12 up, 12 in 2024-04-23T19:08:15.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:07 smithi033 ceph-mon[29274]: fsmap cephfs:0/5, 5 failed 2024-04-23T19:08:15.764 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: pgmap v1363: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 30 op/s 2024-04-23T19:08:15.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: Health check cleared: FS_WITH_FAILED_MDS (was: 1 filesystem has a failed mds daemon) 2024-04-23T19:08:15.849 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6834/2537941763,v1:172.21.15.33:6835/2537941763] up:boot 2024-04-23T19:08:15.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6832/645036474,v1:172.21.15.122:6833/645036474] up:boot 2024-04-23T19:08:16.002 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6832/3962646188,v1:172.21.15.106:6834/3962646188] up:boot 2024-04-23T19:08:16.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6833/4029605020,v1:172.21.15.106:6835/4029605020] up:boot 2024-04-23T19:08:16.159 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6836/4194336011,v1:172.21.15.33:6837/4194336011] up:boot 2024-04-23T19:08:16.333 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.33:6838/1890464907,v1:172.21.15.33:6839/1890464907] up:boot 2024-04-23T19:08:16.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6835/3127766675,v1:172.21.15.122:6837/3127766675] up:boot 2024-04-23T19:08:16.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.122:6834/4246706410,v1:172.21.15.122:6836/4246706410] up:boot 2024-04-23T19:08:16.525 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6836/2147655304,v1:172.21.15.106:6838/2147655304] up:boot 2024-04-23T19:08:16.580 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: mds.? [v2:172.21.15.106:6837/1859274513,v1:172.21.15.106:6839/1859274513] up:boot 2024-04-23T19:08:16.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: fsmap cephfs:0/5 10 up:standby, 5 failed 2024-04-23T19:08:16.768 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-23T19:08:16.797 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-23T19:08:16.846 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-23T19:08:16.887 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-23T19:08:16.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-23T19:08:16.997 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-23T19:08:17.053 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-23T19:08:17.133 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-23T19:08:17.217 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-23T19:08:17.256 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:08 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-23T19:08:17.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:10 smithi033 ceph-mon[29274]: pgmap v1365: 97 pgs: 97 active+clean; 15 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-04-23T19:08:17.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:12 smithi033 ceph-mon[29274]: pgmap v1366: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.413 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:14 smithi033 ceph-mon[29274]: pgmap v1367: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.423 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:14 smithi122 ceph-mon[32546]: pgmap v1367: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.433 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:16 smithi122 ceph-mon[32546]: pgmap v1368: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.442 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:16 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T19:08:17.467 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:14 smithi106 ceph-mon[32716]: pgmap v1367: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.528 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:16 smithi106 ceph-mon[32716]: pgmap v1368: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:17.579 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:16 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T19:08:17.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.567+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/3911709133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4102b40 msgr2=0x7fdbc4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:17.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.567+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/3911709133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4102b40 0x7fdbc4106b80 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009900 tx=0x7fdbb402a410 comp rx=0 tx=0).stop 2024-04-23T19:08:17.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/3911709133 shutdown_connections 2024-04-23T19:08:17.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/3911709133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4107260 0x7fdbc410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:17.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/3911709133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4102b40 0x7fdbc4106b80 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:18.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/3911709133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4102190 0x7fdbc4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/3911709133 >> 172.21.15.33:0/3911709133 conn(0x7fdbc40793a0 msgr2=0x7fdbc40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:18.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/3911709133 shutdown_connections 2024-04-23T19:08:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/3911709133 wait complete. 2024-04-23T19:08:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 Processor -- start 2024-04-23T19:08:18.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.568+0000 7fdbcb6cb640 1 -- start start 2024-04-23T19:08:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4102190 0x7fdbc419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:18.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4107260 0x7fdbc419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:18.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:18.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc41121a0 con 0x7fdbc4198650 2024-04-23T19:08:18.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4112020 con 0x7fdbc4102190 2024-04-23T19:08:18.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4112320 con 0x7fdbc4107260 2024-04-23T19:08:18.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:18.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9467640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4102190 0x7fdbc419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:18.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50636/0 (socket says 172.21.15.33:50636) 2024-04-23T19:08:18.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 -- 172.21.15.33:0/2663066879 learned_addr learned my addr 172.21.15.33:0/2663066879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:08:18.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc8c66640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4107260 0x7fdbc419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:19.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 -- 172.21.15.33:0/2663066879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4107260 msgr2=0x7fdbc419e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4107260 0x7fdbc419e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 -- 172.21.15.33:0/2663066879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4102190 msgr2=0x7fdbc419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:19.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4102190 0x7fdbc419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:19.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 -- 172.21.15.33:0/2663066879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc4199170 con 0x7fdbc4198650 2024-04-23T19:08:19.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbc9c68640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cad0 tx=0x7fdbbc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:19.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00ae20 con 0x7fdbc4198650 2024-04-23T19:08:19.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbbc005020 con 0x7fdbc4198650 2024-04-23T19:08:19.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc4199460 con 0x7fdbc4198650 2024-04-23T19:08:19.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc00f6b0 con 0x7fdbc4198650 2024-04-23T19:08:19.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.569+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc41a5610 con 0x7fdbc4198650 2024-04-23T19:08:19.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.571+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc40691c0 con 0x7fdbc4198650 2024-04-23T19:08:19.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.572+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdbbc0051d0 con 0x7fdbc4198650 2024-04-23T19:08:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.572+0000 7fdbba7fc640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 0x7fdbb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.572+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 8928+0+0 (secure 0 0 0) 0x7fdbbc0d0c90 con 0x7fdbc4198650 2024-04-23T19:08:19.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.572+0000 7fdbc9467640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 0x7fdbb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.573+0000 7fdbc9467640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 0x7fdbb00a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005fd0 tx=0x7fdbc000b390 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:19.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.576+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdbbc014030 con 0x7fdbc4198650 2024-04-23T19:08:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:15.801+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdbc410a490 con 0x7fdbc4198650 2024-04-23T19:08:20.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.375+0000 7fdbba7fc640 1 -- 172.21.15.33:0/2663066879 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) ==== 104+0+0 (secure 0 0 0) 0x7fdbbc09cb50 con 0x7fdbc4198650 2024-04-23T19:08:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 msgr2=0x7fdbb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:20.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 0x7fdbb00a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0005fd0 tx=0x7fdbc000b390 comp rx=0 tx=0).stop 2024-04-23T19:08:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 msgr2=0x7fdbc4198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:20.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00cad0 tx=0x7fdbbc00cf90 comp rx=0 tx=0).stop 2024-04-23T19:08:20.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 shutdown_connections 2024-04-23T19:08:20.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fdbb00a3d20 0x7fdbb00a61e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbc4198650 0x7fdbc4198b00 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbc4107260 0x7fdbc419e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:20.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 --2- 172.21.15.33:0/2663066879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbc4102190 0x7fdbc419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:20.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 >> 172.21.15.33:0/2663066879 conn(0x7fdbc40793a0 msgr2=0x7fdbc41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:20.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 shutdown_connections 2024-04-23T19:08:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:16.378+0000 7fdbcb6cb640 1 -- 172.21.15.33:0/2663066879 wait complete. 2024-04-23T19:08:20.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-23T19:08:20.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:16 smithi033 ceph-mon[29274]: pgmap v1368: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-23T19:08:21.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:16 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-23T19:08:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:17 smithi033 ceph-mon[29274]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T19:08:21.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:17 smithi033 ceph-mon[29274]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T19:08:21.323 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:17 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T19:08:21.372 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:17 smithi033 ceph-mon[29274]: fsmap 10 up:standby 2024-04-23T19:08:21.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:18 smithi033 ceph-mon[29274]: pgmap v1369: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-23T19:08:21.515 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:20 smithi033 ceph-mon[29274]: pgmap v1370: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s 2024-04-23T19:08:21.587 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:17 smithi106 ceph-mon[32716]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T19:08:21.618 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:17 smithi106 ceph-mon[32716]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T19:08:21.626 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:17 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T19:08:21.689 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:17 smithi106 ceph-mon[32716]: fsmap 10 up:standby 2024-04-23T19:08:21.760 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:18 smithi106 ceph-mon[32716]: pgmap v1369: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-23T19:08:21.805 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:20 smithi106 ceph-mon[32716]: pgmap v1370: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s 2024-04-23T19:08:21.847 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:17 smithi122 ceph-mon[32546]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-23T19:08:21.921 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:17 smithi122 ceph-mon[32546]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-23T19:08:21.987 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:17 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2663066879' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-23T19:08:21.994 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:17 smithi122 ceph-mon[32546]: fsmap 10 up:standby 2024-04-23T19:08:22.067 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:18 smithi122 ceph-mon[32546]: pgmap v1369: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-23T19:08:22.221 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:20 smithi122 ceph-mon[32546]: pgmap v1370: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 517 B/s rd, 1 op/s 2024-04-23T19:08:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 -- 172.21.15.33:0/600117509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 msgr2=0x7f2f48101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:22.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/600117509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48101480 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009900 tx=0x7f2f3802a410 comp rx=0 tx=0).stop 2024-04-23T19:08:22.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 -- 172.21.15.33:0/600117509 shutdown_connections 2024-04-23T19:08:22.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/600117509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f4810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:22.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/600117509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f48101a50 0x7f2f48105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/600117509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48101480 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:22.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.705+0000 7f2f4cc83640 1 -- 172.21.15.33:0/600117509 >> 172.21.15.33:0/600117509 conn(0x7f2f48077eb0 msgr2=0x7f2f480782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.706+0000 7f2f4cc83640 1 -- 172.21.15.33:0/600117509 shutdown_connections 2024-04-23T19:08:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.706+0000 7f2f4cc83640 1 -- 172.21.15.33:0/600117509 wait complete. 2024-04-23T19:08:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.706+0000 7f2f4cc83640 1 Processor -- start 2024-04-23T19:08:22.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.706+0000 7f2f4cc83640 1 -- start start 2024-04-23T19:08:22.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:22.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f48198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:22.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f4819cfe0 0x7f2f4819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:22.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f481110e0 con 0x7f2f481010a0 2024-04-23T19:08:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48110f60 con 0x7f2f4819cfe0 2024-04-23T19:08:23.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4cc83640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f48111260 con 0x7f2f48106170 2024-04-23T19:08:23.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58942/0 (socket says 172.21.15.33:58942) 2024-04-23T19:08:23.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 -- 172.21.15.33:0/2644873312 learned_addr learned my addr 172.21.15.33:0/2644873312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:08:23.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f46d9d640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f4819cfe0 0x7f2f4819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:23.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f45d9b640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f48198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:23.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 -- 172.21.15.33:0/2644873312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 msgr2=0x7f2f48198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f48198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:23.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 -- 172.21.15.33:0/2644873312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f4819cfe0 msgr2=0x7f2f4819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f4819cfe0 0x7f2f4819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:23.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 -- 172.21.15.33:0/2644873312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4819dad0 con 0x7f2f481010a0 2024-04-23T19:08:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f45d9b640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f48198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-23T19:08:23.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.707+0000 7f2f4659c640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005940 tx=0x7f2f38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:23.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.708+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3803f030 con 0x7f2f481010a0 2024-04-23T19:08:23.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.708+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4819dd60 con 0x7f2f481010a0 2024-04-23T19:08:23.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.708+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f38003aa0 con 0x7f2f481010a0 2024-04-23T19:08:24.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.708+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38035070 con 0x7f2f481010a0 2024-04-23T19:08:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.708+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f481a5580 con 0x7f2f481010a0 2024-04-23T19:08:24.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.709+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f480691c0 con 0x7f2f481010a0 2024-04-23T19:08:24.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.710+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2f38042030 con 0x7f2f481010a0 2024-04-23T19:08:24.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.711+0000 7f2f377fe640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 0x7f2f300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:24.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.711+0000 7f2f45d9b640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 0x7f2f300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:24.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.711+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 8928+0+0 (secure 0 0 0) 0x7f2f380f0c40 con 0x7f2f481010a0 2024-04-23T19:08:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.711+0000 7f2f45d9b640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 0x7f2f300a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f481075a0 tx=0x7f2f3c00a5c0 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:24.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.716+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2f380bd150 con 0x7f2f481010a0 2024-04-23T19:08:24.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:21.902+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f4810a3b0 con 0x7f2f481010a0 2024-04-23T19:08:24.713 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:22 smithi106 ceph-mon[32716]: pgmap v1371: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-04-23T19:08:24.723 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:22 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:24.736 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:23 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:24.745 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:23 smithi106 ceph-mon[32716]: osdmap e171: 12 total, 12 up, 12 in 2024-04-23T19:08:24.755 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:22 smithi122 ceph-mon[32546]: pgmap v1371: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-04-23T19:08:24.836 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:22 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:24.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:23 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:25.060 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:23 smithi122 ceph-mon[32546]: osdmap e171: 12 total, 12 up, 12 in 2024-04-23T19:08:25.153 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:24 smithi122 ceph-mon[32546]: pgmap v1373: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:25.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.393+0000 7f2f377fe640 1 -- 172.21.15.33:0/2644873312 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v171) ==== 181+0+0 (secure 0 0 0) 0x7f2f380bca50 con 0x7f2f481010a0 2024-04-23T19:08:25.297 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-04-23T19:08:25.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.396+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 msgr2=0x7f2f300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.396+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 0x7f2f300a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f481075a0 tx=0x7f2f3c00a5c0 comp rx=0 tx=0).stop 2024-04-23T19:08:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.396+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 msgr2=0x7f2f48198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:25.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.396+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005940 tx=0x7f2f38004270 comp rx=0 tx=0).stop 2024-04-23T19:08:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 shutdown_connections 2024-04-23T19:08:25.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7f2f300a3d20 0x7f2f300a61e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:25.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f4819cfe0 0x7f2f4819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:25.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f48106170 0x7f2f48198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 --2- 172.21.15.33:0/2644873312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f481010a0 0x7f2f48198720 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 >> 172.21.15.33:0/2644873312 conn(0x7f2f48077eb0 msgr2=0x7f2f4806a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:25.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 shutdown_connections 2024-04-23T19:08:25.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:22.397+0000 7f2f4cc83640 1 -- 172.21.15.33:0/2644873312 wait complete. 2024-04-23T19:08:25.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-23T19:08:25.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:22 smithi033 ceph-mon[29274]: pgmap v1371: 97 pgs: 97 active+clean; 15 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 0 op/s 2024-04-23T19:08:26.016 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:22 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:26.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:23 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/2644873312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:26.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:23 smithi033 ceph-mon[29274]: osdmap e171: 12 total, 12 up, 12 in 2024-04-23T19:08:26.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:24 smithi033 ceph-mon[29274]: pgmap v1373: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:26.169 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:24 smithi106 ceph-mon[32716]: pgmap v1373: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 -- 172.21.15.33:0/3932178353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe6406b1f0 msgr2=0x7fbe64106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:26.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 --2- 172.21.15.33:0/3932178353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe6406b1f0 0x7fbe64106b90 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009900 tx=0x7fbe5402a410 comp rx=0 tx=0).stop 2024-04-23T19:08:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 -- 172.21.15.33:0/3932178353 shutdown_connections 2024-04-23T19:08:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 --2- 172.21.15.33:0/3932178353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe64107270 0x7fbe6410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:26.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 --2- 172.21.15.33:0/3932178353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe6406b1f0 0x7fbe64106b90 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:26.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 --2- 172.21.15.33:0/3932178353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 0x7fbe6406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:26.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 -- 172.21.15.33:0/3932178353 >> 172.21.15.33:0/3932178353 conn(0x7fbe64077eb0 msgr2=0x7fbe640782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:26.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 -- 172.21.15.33:0/3932178353 shutdown_connections 2024-04-23T19:08:26.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.286+0000 7fbe6c536640 1 -- 172.21.15.33:0/3932178353 wait complete. 2024-04-23T19:08:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 Processor -- start 2024-04-23T19:08:26.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 -- start start 2024-04-23T19:08:26.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 0x7fbe641986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe6419cfb0 0x7fbe6419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe641121b0 con 0x7fbe64107270 2024-04-23T19:08:27.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64112030 con 0x7fbe6406a840 2024-04-23T19:08:27.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6c536640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64112330 con 0x7fbe6419cfb0 2024-04-23T19:08:27.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe69ad1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe69ad1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58964/0 (socket says 172.21.15.33:58964) 2024-04-23T19:08:27.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe69ad1640 1 -- 172.21.15.33:0/171979372 learned_addr learned my addr 172.21.15.33:0/171979372 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-23T19:08:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.287+0000 7fbe6aad3640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe6419cfb0 0x7fbe6419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:27.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe6a2d2640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 0x7fbe641986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:27.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 -- 172.21.15.33:0/171979372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe6419cfb0 msgr2=0x7fbe6419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe6419cfb0 0x7fbe6419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:27.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 -- 172.21.15.33:0/171979372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 msgr2=0x7fbe641986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:27.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 0x7fbe641986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:27.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 -- 172.21.15.33:0/171979372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6419daa0 con 0x7fbe64107270 2024-04-23T19:08:27.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe69ad1640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbe54032fd0 tx=0x7fbe54004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5403f030 con 0x7fbe64107270 2024-04-23T19:08:28.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe54004490 con 0x7fbe64107270 2024-04-23T19:08:28.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6419dd30 con 0x7fbe64107270 2024-04-23T19:08:28.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54035070 con 0x7fbe64107270 2024-04-23T19:08:28.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.288+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641a5550 con 0x7fbe64107270 2024-04-23T19:08:28.199 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:26 smithi106 ceph-mon[32716]: pgmap v1374: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:28.276 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:27 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:28.309 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:26 smithi122 ceph-mon[32546]: pgmap v1374: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:28.384 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:27 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:28.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.289+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe640691c0 con 0x7fbe64107270 2024-04-23T19:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.291+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbe54042030 con 0x7fbe64107270 2024-04-23T19:08:28.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.291+0000 7fbe5b7fe640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 0x7fbe500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-23T19:08:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.291+0000 7fbe6a2d2640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 0x7fbe500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-23T19:08:28.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.291+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 8403+0+0 (secure 0 0 0) 0x7fbe540f0be0 con 0x7fbe64107270 2024-04-23T19:08:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.292+0000 7fbe6a2d2640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 0x7fbe500a6260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005fd0 tx=0x7fbe60005850 comp rx=0 tx=0).ready entity=mgr.15357 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-23T19:08:28.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.296+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe540bd2f0 con 0x7fbe64107270 2024-04-23T19:08:28.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:26.482+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe6406b1f0 con 0x7fbe64107270 2024-04-23T19:08:28.904 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:26 smithi033 ceph-mon[29274]: pgmap v1374: 65 pgs: 65 active+clean; 577 KiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:29.005 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:28 smithi106 ceph-mon[32716]: pgmap v1375: 65 pgs: 65 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:29.039 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:28 smithi106 ceph-mon[32716]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:29.140 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:28 smithi106 ceph-mon[32716]: osdmap e172: 12 total, 12 up, 12 in 2024-04-23T19:08:29.228 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:28 smithi122 ceph-mon[32546]: pgmap v1375: 65 pgs: 65 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:29.284 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:28 smithi122 ceph-mon[32546]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:29.308 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:28 smithi122 ceph-mon[32546]: osdmap e172: 12 total, 12 up, 12 in 2024-04-23T19:08:29.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.418+0000 7fbe5b7fe640 1 -- 172.21.15.33:0/171979372 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v172) ==== 169+0+0 (secure 0 0 0) 0x7fbe540bcbf0 con 0x7fbe64107270 2024-04-23T19:08:29.390 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-04-23T19:08:29.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 msgr2=0x7fbe500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 0x7fbe500a6260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbe60005fd0 tx=0x7fbe60005850 comp rx=0 tx=0).stop 2024-04-23T19:08:29.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 msgr2=0x7fbe64198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-23T19:08:29.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fbe54032fd0 tx=0x7fbe54004270 comp rx=0 tx=0).stop 2024-04-23T19:08:29.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 shutdown_connections 2024-04-23T19:08:29.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:6800/3751231116,v1:172.21.15.33:6801/3751231116] conn(0x7fbe500a3da0 0x7fbe500a6260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:29.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe6419cfb0 0x7fbe6419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:29.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe64107270 0x7fbe64198c30 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:29.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 --2- 172.21.15.33:0/171979372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe6406a840 0x7fbe641986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-23T19:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 >> 172.21.15.33:0/171979372 conn(0x7fbe64077eb0 msgr2=0x7fbe64108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-23T19:08:30.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.421+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 shutdown_connections 2024-04-23T19:08:30.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-23T19:08:27.422+0000 7fbe6c536640 1 -- 172.21.15.33:0/171979372 wait complete. 2024-04-23T19:08:30.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:08:30.331 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:27 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-23T19:08:30.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:28 smithi033 ceph-mon[29274]: pgmap v1375: 65 pgs: 65 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:30.449 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:28 smithi033 ceph-mon[29274]: from='client.? 172.21.15.33:0/171979372' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-23T19:08:30.500 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:28 smithi033 ceph-mon[29274]: osdmap e172: 12 total, 12 up, 12 in 2024-04-23T19:08:30.577 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:29 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:08:30.627 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:29 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:08:30.636 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:29 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-23T19:08:30.652 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:08:30.706 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: pgmap v1377: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:30.769 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:30.847 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:30.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:08:31.008 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:08:31.097 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:30 smithi122 ceph-mon[32546]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.149 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: pgmap v1377: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:31.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:08:31.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:08:31.332 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:30 smithi033 ceph-mon[29274]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.375 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: pgmap v1377: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:31.457 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.527 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-23T19:08:31.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-23T19:08:31.654 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:30 smithi106 ceph-mon[32716]: from='mgr.15357 172.21.15.33:0/2895484382' entity='mgr.x' 2024-04-23T19:08:31.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:08:31.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:31 smithi122 ceph-mon[32546]: pgmap v1378: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:31.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:31 smithi122 ceph-mon[32546]: pgmap v1379: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:32.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:31 smithi033 ceph-mon[29274]: pgmap v1378: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:32.055 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:31 smithi033 ceph-mon[29274]: pgmap v1379: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:32.113 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:31 smithi106 ceph-mon[32716]: pgmap v1378: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:32.155 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:31 smithi106 ceph-mon[32716]: pgmap v1379: 1 pgs: 1 active+clean; 577 KiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-23T19:08:32.210 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-23T19:08:32.267 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T19:08:32.341 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T19:08:32.397 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-23T19:08:32.456 INFO:tasks.cephadm:Stopping all daemons... 2024-04-23T19:08:32.539 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-23T19:08:32.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a 2024-04-23T19:08:32.963 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:32 smithi033 systemd[1]: Stopping Ceph mon.a for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:33.238 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a[29270]: 2024-04-23T19:08:32.962+0000 7f46dae7b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:08:33.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a[29270]: 2024-04-23T19:08:32.962+0000 7f46dae7b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-23T19:08:33.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:33 smithi033 podman[106436]: 2024-04-23 19:08:33.156485675 +0000 UTC m=+0.310333481 container died 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:08:33.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:33 smithi033 podman[106436]: 2024-04-23 19:08:33.546690879 +0000 UTC m=+0.700538689 container cleanup 5bf33b2d8c0ebbf974d27c54e8a7ee48631d9e2599ae08835feb00bfd8431f94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-23T19:08:33.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Apr 23 19:08:33 smithi033 bash[106436]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-a 2024-04-23T19:08:33.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.a.service' 2024-04-23T19:08:34.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:34.126 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-23T19:08:34.134 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-23T19:08:34.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.b 2024-04-23T19:08:34.545 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:34 smithi106 systemd[1]: Stopping Ceph mon.b for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:34.878 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-b[32712]: 2024-04-23T19:08:34.544+0000 7f818d4d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:08:34.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-b[32712]: 2024-04-23T19:08:34.544+0000 7f818d4d0700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-23T19:08:35.006 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:34 smithi106 podman[75968]: 2024-04-23 19:08:34.606444235 +0000 UTC m=+0.264419972 container died 7c4223f6e9fe32246b7250b330d13c5c874e692d58d769185867249243982842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-23T19:08:35.378 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:35 smithi106 podman[75968]: 2024-04-23 19:08:35.155092216 +0000 UTC m=+0.813068005 container cleanup 7c4223f6e9fe32246b7250b330d13c5c874e692d58d769185867249243982842 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T19:08:35.474 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 23 19:08:35 smithi106 bash[75968]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-b 2024-04-23T19:08:35.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.b.service' 2024-04-23T19:08:35.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:35.758 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-23T19:08:35.797 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-23T19:08:35.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.c 2024-04-23T19:08:36.456 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:36 smithi122 systemd[1]: Stopping Ceph mon.c for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:36.956 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c[32534]: 2024-04-23T19:08:36.650+0000 7ff5afa38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:08:36.989 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c[32534]: 2024-04-23T19:08:36.650+0000 7ff5afa38700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-23T19:08:37.049 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:36 smithi122 podman[78417]: 2024-04-23 19:08:36.825263064 +0000 UTC m=+0.557571443 container died 3aafede04df2fd57a19b4fbb2aa3c07b0982dd2b9fbb502a00e6a4690a747917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T19:08:37.456 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 podman[78417]: 2024-04-23 19:08:37.351524757 +0000 UTC m=+1.083833137 container cleanup 3aafede04df2fd57a19b4fbb2aa3c07b0982dd2b9fbb502a00e6a4690a747917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:08:37.520 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 bash[78417]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c 2024-04-23T19:08:37.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.c.service' 2024-04-23T19:08:37.883 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 podman[78431]: 2024-04-23 19:08:37.595304028 +0000 UTC m=+0.762360103 container remove 3aafede04df2fd57a19b4fbb2aa3c07b0982dd2b9fbb502a00e6a4690a747917 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-23T19:08:37.891 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-23T19:08:37.905 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 systemd[1]: Stopped Ceph mon.c for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:08:37.944 INFO:journalctl@ceph.mon.c.smithi122.stdout:Apr 23 19:08:37 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mon.c.service: Consumed 26.972s CPU time. 2024-04-23T19:08:38.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:38.162 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-23T19:08:38.238 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-23T19:08:38.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x 2024-04-23T19:08:38.622 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:38 smithi033 systemd[1]: Stopping Ceph mgr.x for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:38.911 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:38 smithi033 podman[106557]: 2024-04-23 19:08:38.624184908 +0000 UTC m=+0.160063702 container died d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:08:39.238 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 podman[106557]: 2024-04-23 19:08:39.010571965 +0000 UTC m=+0.546450758 container cleanup d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-23T19:08:39.263 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 bash[106557]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x 2024-04-23T19:08:39.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-23T19:08:39.543 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 podman[106569]: 2024-04-23 19:08:39.271117916 +0000 UTC m=+0.642527242 container remove d7656f9fb00470ff7e2de2b7dc3cdbbfdb17cbe994d80ac7996770092e20657f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T19:08:39.589 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service: Deactivated successfully. 2024-04-23T19:08:39.612 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 systemd[1]: Stopped Ceph mgr.x for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:08:39.620 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Apr 23 19:08:39 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.x.service: Consumed 2min 32.328s CPU time. 2024-04-23T19:08:39.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:39.728 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-23T19:08:39.771 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-23T19:08:39.899 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.y 2024-04-23T19:08:40.378 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 19:08:40 smithi106 systemd[1]: Stopping Ceph mgr.y for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:40.694 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 19:08:40 smithi106 podman[76090]: 2024-04-23 19:08:40.380967217 +0000 UTC m=+0.221513888 container died 8f55751ed7ec65ce34aa2a713e523b27ab9c413a11bce0f844edf755966b2687 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T19:08:41.037 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 19:08:40 smithi106 podman[76090]: 2024-04-23 19:08:40.794326099 +0000 UTC m=+0.634872770 container cleanup 8f55751ed7ec65ce34aa2a713e523b27ab9c413a11bce0f844edf755966b2687 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-23T19:08:41.051 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 23 19:08:40 smithi106 bash[76090]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-y 2024-04-23T19:08:41.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-23T19:08:41.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:41.499 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-23T19:08:41.532 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-23T19:08:41.580 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z 2024-04-23T19:08:41.956 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:41 smithi122 systemd[1]: Stopping Ceph mgr.z for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:42.291 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:41 smithi122 podman[78538]: 2024-04-23 19:08:41.985984427 +0000 UTC m=+0.154860396 container died 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:08:42.659 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 podman[78538]: 2024-04-23 19:08:42.38251156 +0000 UTC m=+0.551387535 container cleanup 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T19:08:42.685 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 bash[78538]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z 2024-04-23T19:08:42.815 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-23T19:08:42.939 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 podman[78552]: 2024-04-23 19:08:42.659170591 +0000 UTC m=+0.665117386 container remove 313fc5dbb8cf6889a199eaf04cd68239147db50eea7e61e369099ba00ef0b913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-23T19:08:43.025 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-23T19:08:43.157 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-23T19:08:43.264 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 systemd[1]: Stopped Ceph mgr.z for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:08:43.306 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Apr 23 19:08:42 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mgr.z.service: Consumed 27.894s CPU time. 2024-04-23T19:08:43.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:43.347 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-23T19:08:43.356 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-23T19:08:43.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.0 2024-04-23T19:08:43.849 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:43 smithi033 systemd[1]: Stopping Ceph osd.0 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:44.238 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0[43690]: 2024-04-23T19:08:43.848+0000 7f685c54f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:08:44.287 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0[43690]: 2024-04-23T19:08:43.848+0000 7f685c54f700 -1 osd.0 172 *** Got signal Terminated *** 2024-04-23T19:08:44.356 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0[43690]: 2024-04-23T19:08:43.848+0000 7f685c54f700 -1 osd.0 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:08:49.484 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:49 smithi033 podman[106676]: 2024-04-23 19:08:49.17926475 +0000 UTC m=+5.451630827 container died 90820a1016aa81f7e4a9947185384475d8aeddb9cf63ad1eb35d8edc2c489417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-23T19:08:49.738 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:49 smithi033 podman[106676]: 2024-04-23 19:08:49.574592376 +0000 UTC m=+5.846958452 container cleanup 90820a1016aa81f7e4a9947185384475d8aeddb9cf63ad1eb35d8edc2c489417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T19:08:49.798 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:49 smithi033 bash[106676]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0 2024-04-23T19:08:50.145 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:49 smithi033 podman[106696]: 2024-04-23 19:08:49.809278656 +0000 UTC m=+0.628118404 container remove 90820a1016aa81f7e4a9947185384475d8aeddb9cf63ad1eb35d8edc2c489417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:08:50.480 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.04889293 +0000 UTC m=+0.018960781 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:08:50.581 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.160613805 +0000 UTC m=+0.130681676 container create 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-23T19:08:50.642 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.380142732 +0000 UTC m=+0.350210561 container init 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:08:50.682 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.383408817 +0000 UTC m=+0.353476647 container start 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-23T19:08:50.691 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.480137179 +0000 UTC m=+0.450205020 container attach 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-23T19:08:50.738 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:50 smithi033 podman[106789]: 2024-04-23 19:08:50.692467471 +0000 UTC m=+0.662535301 container died 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:08:51.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.0.service' 2024-04-23T19:08:51.438 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:51 smithi033 podman[106815]: 2024-04-23 19:08:51.351684319 +0000 UTC m=+0.653170607 container remove 255d0a2695fb0575c90ce869012d2471bf88cd29405061580d62be77ccbbed54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T19:08:51.448 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:51 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-23T19:08:51.456 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:51 smithi033 systemd[1]: Stopped Ceph osd.0 for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:08:51.464 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 23 19:08:51 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.0.service: Consumed 24.950s CPU time. 2024-04-23T19:08:51.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:51.501 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-23T19:08:51.548 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-23T19:08:51.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.1 2024-04-23T19:08:52.057 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:51 smithi106 systemd[1]: Stopping Ceph osd.1 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:08:52.378 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-23T19:08:52.056+0000 7fd8377cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:08:52.416 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-23T19:08:52.056+0000 7fd8377cf700 -1 osd.1 172 *** Got signal Terminated *** 2024-04-23T19:08:52.495 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1[37313]: 2024-04-23T19:08:52.056+0000 7fd8377cf700 -1 osd.1 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:08:57.708 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:57 smithi106 podman[76218]: 2024-04-23 19:08:57.405468409 +0000 UTC m=+5.460967936 container died 872ac92dbdf414b9bb57dd41391cde4077b406b491f88b10e0fb03eb192f00d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T19:08:58.034 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:57 smithi106 podman[76218]: 2024-04-23 19:08:57.816132534 +0000 UTC m=+5.871632037 container cleanup 872ac92dbdf414b9bb57dd41391cde4077b406b491f88b10e0fb03eb192f00d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-23T19:08:58.094 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:57 smithi106 bash[76218]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1 2024-04-23T19:08:58.332 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76235]: 2024-04-23 19:08:58.034179482 +0000 UTC m=+0.625307770 container remove 872ac92dbdf414b9bb57dd41391cde4077b406b491f88b10e0fb03eb192f00d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-23T19:08:58.595 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.235525809 +0000 UTC m=+0.017859263 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:08:58.666 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.343206486 +0000 UTC m=+0.125539922 container create 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:08:58.878 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.596132696 +0000 UTC m=+0.378466134 container init 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:08:58.893 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.599401963 +0000 UTC m=+0.381735409 container start 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-23T19:08:58.903 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.679382809 +0000 UTC m=+0.461716261 container attach 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:08:59.199 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:58 smithi106 podman[76324]: 2024-04-23 19:08:58.896287377 +0000 UTC m=+0.678620825 container died 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T19:08:59.455 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 23 19:08:59 smithi106 podman[76351]: 2024-04-23 19:08:59.442658076 +0000 UTC m=+0.540772007 container remove 26305fc91c366d7e9484ff6a30405d3cafdf5baf9521bf3526f3a9c7cc290f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-23T19:08:59.533 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.1.service' 2024-04-23T19:08:59.614 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:08:59.623 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-23T19:08:59.639 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-23T19:08:59.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.2 2024-04-23T19:08:59.956 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:08:59 smithi122 systemd[1]: Stopping Ceph osd.2 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:09:00.456 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:08:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T19:08:59.977+0000 7f9cce5a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:09:00.500 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:08:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T19:08:59.977+0000 7f9cce5a5700 -1 osd.2 172 *** Got signal Terminated *** 2024-04-23T19:09:00.544 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:08:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2[37706]: 2024-04-23T19:08:59.977+0000 7f9cce5a5700 -1 osd.2 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:09:05.622 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:05 smithi122 podman[78664]: 2024-04-23 19:09:05.320593352 +0000 UTC m=+5.470368567 container died c96e93307e4e85c73d8436a122ddd162dcd0beed5a6e1abe21cf1793bd446949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-23T19:09:05.956 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:05 smithi122 podman[78664]: 2024-04-23 19:09:05.72070355 +0000 UTC m=+5.870478749 container cleanup c96e93307e4e85c73d8436a122ddd162dcd0beed5a6e1abe21cf1793bd446949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T19:09:06.036 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:05 smithi122 bash[78664]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2 2024-04-23T19:09:06.343 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78693]: 2024-04-23 19:09:06.005581675 +0000 UTC m=+0.684593696 container remove c96e93307e4e85c73d8436a122ddd162dcd0beed5a6e1abe21cf1793bd446949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-23T19:09:06.667 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.246725903 +0000 UTC m=+0.017927642 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:09:06.817 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.356463576 +0000 UTC m=+0.127665290 container create 17a668bbb0a3be541de79c0c4d69320a210eb3ac63bf91f73f122b4415385ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T19:09:06.852 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.576019472 +0000 UTC m=+0.347221198 container init 17a668bbb0a3be541de79c0c4d69320a210eb3ac63bf91f73f122b4415385ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T19:09:06.879 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.579257906 +0000 UTC m=+0.350459624 container start 17a668bbb0a3be541de79c0c4d69320a210eb3ac63bf91f73f122b4415385ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-23T19:09:06.889 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.667774295 +0000 UTC m=+0.438976020 container attach 17a668bbb0a3be541de79c0c4d69320a210eb3ac63bf91f73f122b4415385ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:09:06.956 INFO:journalctl@ceph.osd.2.smithi122.stdout:Apr 23 19:09:06 smithi122 podman[78788]: 2024-04-23 19:09:06.873768058 +0000 UTC m=+0.644969776 container died 17a668bbb0a3be541de79c0c4d69320a210eb3ac63bf91f73f122b4415385ccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-23T19:09:07.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.2.service' 2024-04-23T19:09:07.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:09:07.687 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-23T19:09:07.838 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-23T19:09:07.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.3 2024-04-23T19:09:08.237 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:07 smithi033 systemd[1]: Stopping Ceph osd.3 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:09:08.738 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T19:09:08.245+0000 7f6a296c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:09:08.777 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T19:09:08.245+0000 7f6a296c2700 -1 osd.3 172 *** Got signal Terminated *** 2024-04-23T19:09:08.819 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T19:09:08.245+0000 7f6a296c2700 -1 osd.3 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:09:11.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:10.826+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:11.956 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:11.461+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:11.996 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T19:09:11.663+0000 7f6a21cae700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:45.050513+0000 front 2024-04-23T19:08:45.050486+0000 (oldest deadline 2024-04-23T19:09:10.950645+0000) 2024-04-23T19:09:12.021 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:11.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:12.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:11.862+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:12.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:12.434+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:12.988 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3[50551]: 2024-04-23T19:09:12.615+0000 7f6a21cae700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:45.050513+0000 front 2024-04-23T19:08:45.050486+0000 (oldest deadline 2024-04-23T19:09:10.950645+0000) 2024-04-23T19:09:13.027 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:12.685+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:13.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:12.894+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:13.670 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:13.403+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:13.956 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:13.459+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:13.989 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:13.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:14.036 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:13 smithi033 podman[106907]: 2024-04-23 19:09:13.750233592 +0000 UTC m=+5.627008468 container died 64d95280fadcda07e1326f4b31605d38be4aefa3976bb41ac9f5e8f3edd5c2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-23T19:09:14.078 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:13.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:14.363 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[106907]: 2024-04-23 19:09:14.120878649 +0000 UTC m=+5.997653514 container cleanup 64d95280fadcda07e1326f4b31605d38be4aefa3976bb41ac9f5e8f3edd5c2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T19:09:14.503 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 bash[106907]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3 2024-04-23T19:09:14.515 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:13.933+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:14.696 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[106924]: 2024-04-23 19:09:14.363736264 +0000 UTC m=+0.607298500 container remove 64d95280fadcda07e1326f4b31605d38be4aefa3976bb41ac9f5e8f3edd5c2d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:09:14.786 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:14.382+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:14.956 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:14.496+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:14.994 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:14.471+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:15.026 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:14.695+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:15.071 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[107017]: 2024-04-23 19:09:14.606605065 +0000 UTC m=+0.018230875 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:09:15.098 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[107017]: 2024-04-23 19:09:14.706403354 +0000 UTC m=+0.118029206 container create fdb7742c1d511ab3493c84e753c7289031d4a09e89383e94f57f8afe256ef395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-23T19:09:15.216 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[107017]: 2024-04-23 19:09:14.892501297 +0000 UTC m=+0.304127110 container init fdb7742c1d511ab3493c84e753c7289031d4a09e89383e94f57f8afe256ef395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T19:09:15.242 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[107017]: 2024-04-23 19:09:14.895645523 +0000 UTC m=+0.307271335 container start fdb7742c1d511ab3493c84e753c7289031d4a09e89383e94f57f8afe256ef395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-23T19:09:15.379 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:14.941+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:15.625 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:14 smithi033 podman[107017]: 2024-04-23 19:09:14.992706692 +0000 UTC m=+0.404332520 container attach fdb7742c1d511ab3493c84e753c7289031d4a09e89383e94f57f8afe256ef395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-23T19:09:15.772 INFO:journalctl@ceph.osd.3.smithi033.stdout:Apr 23 19:09:15 smithi033 podman[107017]: 2024-04-23 19:09:15.208215934 +0000 UTC m=+0.619841738 container died fdb7742c1d511ab3493c84e753c7289031d4a09e89383e94f57f8afe256ef395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T19:09:15.799 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:15.395+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:15.829 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.3.service' 2024-04-23T19:09:15.930 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:15.667+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:15.969 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:15.455+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:15.984 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:15.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:15.995 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:09:16.048 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-23T19:09:16.152 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-23T19:09:16.188 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.4 2024-04-23T19:09:16.316 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:15.960+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:16.596 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:16 smithi106 systemd[1]: Stopping Ceph osd.4 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:09:16.684 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:16.414+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:16.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:16.433+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:16.719 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:16.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:16.878 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:16.595+0000 7f55ca340700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:09:16.961 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:16.595+0000 7f55ca340700 -1 osd.4 172 *** Got signal Terminated *** 2024-04-23T19:09:17.029 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:16.595+0000 7f55ca340700 -1 osd.4 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:09:17.118 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:16.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:17.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:16.928+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:17.660 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:17.393+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:17.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:17.387+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:17.745 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:17.461+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:17.988 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:17.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:18.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:17.918+0000 7f55c6934700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:52.839670+0000 front 2024-04-23T19:08:52.839694+0000 (oldest deadline 2024-04-23T19:09:17.539812+0000) 2024-04-23T19:09:18.480 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:17.954+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:18.529 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:18.089+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:18.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:18.397+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:18.753 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:18.447+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:18.779 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:18.447+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:18.813 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:18.413+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:18.820 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:18.631+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:19.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:18.888+0000 7f55c6934700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:52.839670+0000 front 2024-04-23T19:08:52.839694+0000 (oldest deadline 2024-04-23T19:09:17.539812+0000) 2024-04-23T19:09:19.461 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:19.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:19.523 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:18.942+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:19.596 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:18.942+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:19.662 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:19.370+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:19.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:19.446+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:19.808 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:19.446+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:19.817 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:19.493+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:19.883 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:19.493+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:19.988 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:19.661+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:20.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:19.893+0000 7f55c6934700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:52.839670+0000 front 2024-04-23T19:08:52.839694+0000 (oldest deadline 2024-04-23T19:09:17.539812+0000) 2024-04-23T19:09:20.414 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:20.118+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:20.476 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:19.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:20.521 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:19.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:20.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:20.412+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:20.750 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:20.412+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:20.828 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:20.518+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:20.837 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:20.518+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:20.964 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:20.370+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:21.030 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:20.370+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:21.083 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:20.619+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:21.180 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:20.619+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:21.209 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4[44278]: 2024-04-23T19:09:20.917+0000 7f55c6934700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:52.839670+0000 front 2024-04-23T19:08:52.839694+0000 (oldest deadline 2024-04-23T19:09:17.539812+0000) 2024-04-23T19:09:21.353 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:20.912+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:21.438 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:20.912+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:21.628 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:21.165+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:21.692 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:21.406+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:21.846 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:21.406+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:21.964 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:21.313+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:21.973 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:21.374+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:22.007 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:21.374+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:22.077 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:21.540+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:22.166 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:21.540+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:22.240 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:21.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:22.274 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:21.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:22.378 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:22 smithi106 podman[76442]: 2024-04-23 19:09:22.112877081 +0000 UTC m=+5.627738559 container died 75319b60c8f65528f0296b578fd7cb6f14653c3bc5edee12721d13c423858ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T19:09:22.415 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:22.143+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:22.465 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:21.933+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:22.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:21.933+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:22.706 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:22.335+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:22.784 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:22.402+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:22.836 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:22.402+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:22.930 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:22.575+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:22.986 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:22.575+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:23.164 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:22.367+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:23.220 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:22.367+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:23.229 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:22 smithi106 podman[76442]: 2024-04-23 19:09:22.513683198 +0000 UTC m=+6.028544684 container cleanup 75319b60c8f65528f0296b578fd7cb6f14653c3bc5edee12721d13c423858ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-23T19:09:23.238 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:22 smithi106 bash[76442]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4 2024-04-23T19:09:23.414 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:22 smithi106 podman[76459]: 2024-04-23 19:09:22.723740562 +0000 UTC m=+0.613707345 container remove 75319b60c8f65528f0296b578fd7cb6f14653c3bc5edee12721d13c423858ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:09:23.453 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:22.712+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:23.517 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:22.712+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:23.576 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:22.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:23.639 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:22.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:23.704 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:23.125+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:23.758 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:22.925509022 +0000 UTC m=+0.017743188 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:09:23.798 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:23.058107399 +0000 UTC m=+0.150341610 container create dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T19:09:23.867 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:23.319403304 +0000 UTC m=+0.411637469 container init dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-23T19:09:23.927 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:23.322570717 +0000 UTC m=+0.414804882 container start dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:09:23.951 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:23.427937207 +0000 UTC m=+0.520171377 container attach dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:09:24.007 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:23.294+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:24.088 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:23 smithi106 podman[76550]: 2024-04-23 19:09:23.613055192 +0000 UTC m=+0.705289362 container died dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:09:24.255 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:23.391+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:24.313 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:23.391+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:24.340 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:23.438+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:24.361 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:23.438+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:24.409 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:23.855+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:24.466 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:23.855+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:24.497 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:23.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:24.550 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:23.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:24.587 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:23.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:24.663 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:23.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:24.683 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:24.076+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:24.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.4.service' 2024-04-23T19:09:24.702 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:24 smithi106 podman[76576]: 2024-04-23 19:09:24.215182377 +0000 UTC m=+0.596291692 container remove dcc4edd522d0063e207c2a89f47dd8d5989f57a2e466d00efd2a34ee5daefa98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:09:24.774 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:24 smithi106 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-23T19:09:24.825 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:24 smithi106 systemd[1]: Stopped Ceph osd.4 for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:09:24.865 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 23 19:09:24 smithi106 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.4.service: Consumed 25.540s CPU time. 2024-04-23T19:09:24.950 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:24.277+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:25.028 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:24.353+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:25.053 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:24.353+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:25.062 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:24.463+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:25.072 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:24.463+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:25.081 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:24.523+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:25.090 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:24.523+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:25.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:09:25.123 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-23T19:09:25.177 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-23T19:09:25.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.5 2024-04-23T19:09:25.317 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:24.755+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:25.356 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:24.755+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:25.403 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:25.078+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:25.494 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:24.856+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:25.621 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:24.856+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:25.702 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:25.279+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:25.780 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 systemd[1]: Stopping Ceph osd.5 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:09:25.842 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:25.459+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:25.851 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:25.459+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:25.860 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:25.353+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:25.868 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:25.353+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:25.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:25.549+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:25.977 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:25.549+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:26.049 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:25.641+0000 7ff2e3d07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:09:26.105 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:25.641+0000 7ff2e3d07700 -1 osd.5 172 *** Got signal Terminated *** 2024-04-23T19:09:26.155 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:25.641+0000 7ff2e3d07700 -1 osd.5 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:09:26.222 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:26.036+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:26.230 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:25.861+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:26.236 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:25.861+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:26.270 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:25.785+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:26.327 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:25.785+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:26.576 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:26.263+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:26.625 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:26.465+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:26.665 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:26.465+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:26.776 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:26.465+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.903357+0000 front 2024-04-23T19:09:02.903405+0000 (oldest deadline 2024-04-23T19:09:25.803541+0000) 2024-04-23T19:09:26.811 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:26.387+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:26.908 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:26.387+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:26.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:26.575+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:27.010 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:26.575+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:27.128 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:27.019+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:27.166 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:26.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:27.198 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:26.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:27.262 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:26.796+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:27.314 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:26.796+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:27.545 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:27.262+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:27.614 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:27.262+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:27.706 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:27.422+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:27.801 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:27.422+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:27.921 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:27.422+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.903357+0000 front 2024-04-23T19:09:02.903405+0000 (oldest deadline 2024-04-23T19:09:25.803541+0000) 2024-04-23T19:09:27.967 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:27.420+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:28.026 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:27.420+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:28.038 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:27.545+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:28.193 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:27.545+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:28.243 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:28.068+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:28.281 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:27.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:28.347 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:27.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:28.439 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:27.825+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:28.476 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:27.825+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:28.571 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:28.220+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:28.637 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:28.220+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:28.702 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:28.434+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:28.751 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:28.434+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:28.772 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:28.434+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.903357+0000 front 2024-04-23T19:09:02.903405+0000 (oldest deadline 2024-04-23T19:09:25.803541+0000) 2024-04-23T19:09:28.781 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:28.420+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:28.835 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:28.420+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:28.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:28.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:29.025 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:28.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:29.119 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:28.781+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:29.209 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:28.781+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:29.264 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:28.781+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:29.330 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:28.825+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:29.434 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:28.825+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:29.506 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:29.086+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:29.515 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:29.086+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:29.526 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:29.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:29.534 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:29.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:29.543 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:29.431+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:29.593 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:29.431+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:29.670 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:29.431+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.903357+0000 front 2024-04-23T19:09:02.903405+0000 (oldest deadline 2024-04-23T19:09:25.803541+0000) 2024-04-23T19:09:29.779 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:29.447+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:29.819 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:29.447+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:29.920 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:29.447+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:29.952 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:29.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:30.123 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:29.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:30.189 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:29.821+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:30.216 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:29.821+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:30.267 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:29.821+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:30.352 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:29.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:30.380 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:29.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:30.434 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:29.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:30.491 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:30.077+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:30.621 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:30.077+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:30.649 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:30.393+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.100462+0000 front 2024-04-23T19:08:46.100504+0000 (oldest deadline 2024-04-23T19:09:11.400496+0000) 2024-04-23T19:09:30.702 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:30.393+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.502616+0000 front 2024-04-23T19:08:56.502738+0000 (oldest deadline 2024-04-23T19:09:19.402642+0000) 2024-04-23T19:09:30.799 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5[44635]: 2024-04-23T19:09:30.393+0000 7ff2e02fb700 -1 osd.5 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.903357+0000 front 2024-04-23T19:09:02.903405+0000 (oldest deadline 2024-04-23T19:09:25.803541+0000) 2024-04-23T19:09:30.939 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:30.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:30.996 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:30.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:31.019 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:30.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:31.115 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:30.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:31.147 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:30.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:31.155 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:30.796+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:31.162 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:30.796+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:31.271 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:30.796+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:31.302 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:30.477+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:31.449 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:30.477+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:31.477 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:30.477+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:31.518 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:31 smithi122 podman[78893]: 2024-04-23 19:09:31.118228424 +0000 UTC m=+5.613624777 container died ff11a459760eb96fdbd33fa01e0f781f53e9df72fed5ea21e115acdc3f87b14a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-23T19:09:31.540 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:31.066+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:31.568 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:31.066+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:31.635 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:30.738+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:31.690 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:30.738+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:31.748 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:30.738+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:31.826 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:31.196+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:31.834 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:31.196+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:31.841 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:31 smithi122 podman[78893]: 2024-04-23 19:09:31.53405711 +0000 UTC m=+6.029453444 container cleanup ff11a459760eb96fdbd33fa01e0f781f53e9df72fed5ea21e115acdc3f87b14a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:09:31.883 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:31 smithi122 bash[78893]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5 2024-04-23T19:09:31.890 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:31.438+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:31.897 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:31.438+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:31.966 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:31.438+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:32.029 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:31.514+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:32.146 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:31.514+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:32.196 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:31.514+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:32.246 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:31.807+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:32.331 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:31.807+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:32.376 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:31.807+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:32.577 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:31 smithi122 podman[78922]: 2024-04-23 19:09:31.760434315 +0000 UTC m=+0.636192378 container remove ff11a459760eb96fdbd33fa01e0f781f53e9df72fed5ea21e115acdc3f87b14a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:09:32.627 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:31.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:32.665 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:31.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:32.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:31.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:32.876 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:32.094+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:32.901 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:32.094+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:32.910 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:32.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:32.976 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:32.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:32.986 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:32.397+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:33.007 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:32.397+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:33.072 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:32.397+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:33.132 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:31.994216856 +0000 UTC m=+0.017785774 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:09:33.208 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:32.09473395 +0000 UTC m=+0.118302853 container create 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-23T19:09:33.294 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:32.306326912 +0000 UTC m=+0.329895826 container init 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T19:09:33.302 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:32.309553127 +0000 UTC m=+0.333122034 container start 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-23T19:09:33.312 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:32.397450886 +0000 UTC m=+0.421019790 container attach 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-23T19:09:33.346 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:32 smithi122 podman[79011]: 2024-04-23 19:09:32.595424947 +0000 UTC m=+0.618993843 container died 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:09:33.380 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:32.558+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:33.447 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:32.558+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:33.518 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:32.558+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:33.566 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.5.service' 2024-04-23T19:09:33.651 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:33.198+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:33.690 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:33.198+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:33.749 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:33 smithi122 podman[79037]: 2024-04-23 19:09:33.209584369 +0000 UTC m=+0.608608103 container remove 2d439fd3d399950316204d9769de7819d5f55893a121df65f5e586a9d5a0eafb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T19:09:33.816 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:33 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-23T19:09:33.899 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:33 smithi122 systemd[1]: Stopped Ceph osd.5 for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:09:33.945 INFO:journalctl@ceph.osd.5.smithi122.stdout:Apr 23 19:09:33 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.5.service: Consumed 31.945s CPU time. 2024-04-23T19:09:34.008 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:32.769+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:34.066 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:32.769+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:34.195 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:32.769+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:34.250 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:33.538+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:34.319 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:33.538+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:34.494 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:33.538+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:34.553 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:32.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:34.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:32.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:34.654 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:32.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:34.741 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:33.126+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:34.822 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:33.126+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:34.940 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:33.413+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:35.007 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:33.413+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:35.037 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:33.413+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:35.045 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:33.723+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:35.052 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:33.723+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:35.059 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:33.723+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:35.130 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:33.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:35.231 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:33.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:35.350 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:33.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:35.420 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:34.140+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:35.451 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:34.140+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:35.494 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:34.415+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:35.536 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:34.415+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:35.619 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:34.415+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:35.712 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:34.726+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:35.719 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:34.726+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:35.780 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:34.726+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:35.874 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:34.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:35.923 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:34.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:35.945 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:34.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:36.045 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:35.163+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:36.054 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:35.163+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:36.063 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:35.163+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:36.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:35.680+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:36.230 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:35.680+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:36.381 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:35.680+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:36.459 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:35.680+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:36.533 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:35.380+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:36.593 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:35.380+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:36.657 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:35.380+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:36.708 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:35.380+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:36.765 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:35.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:36.790 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:35.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:36.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:35.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:37.075 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:36.361+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:37.114 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:36.361+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:37.122 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:36.361+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:37.237 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:36.361+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:37.285 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:36.702+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:37.326 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:36.702+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:37.397 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:36.702+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:37.450 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:36.702+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:37.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:09:37.515 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-23T19:09:37.564 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-23T19:09:37.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.6 2024-04-23T19:09:37.676 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:34.198+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:37.733 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:34.198+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:37.805 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:34.198+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:37.872 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:34.565+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:37.945 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:34.565+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:38.011 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:34.565+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:38.070 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:35.221+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:38.094 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:35.221+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:38.209 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:35.221+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:38.260 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:36.167+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:38.322 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:36.167+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:38.400 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:36.167+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:38.459 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:37.350+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:38.506 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:37.350+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:38.513 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:37.350+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:38.521 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:37.350+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:38.530 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:35.528+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:38.616 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:35.528+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:35.528+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:38.757 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:36.674+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:38.811 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:36.674+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:38.864 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:36.674+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:38.940 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:36.674+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:38.999 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:37.695+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:39.049 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:37.695+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:39.115 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:37.695+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:39.122 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:37.695+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:39.130 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:36.191+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:39.188 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:36.191+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:39.196 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:36.191+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:39.239 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:37.212+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:39.319 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:37.212+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:39.352 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:37.212+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:39.397 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 systemd[1]: Stopping Ceph osd.6 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:09:39.445 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.378+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:39.525 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.378+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:39.534 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.378+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:39.542 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.378+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:39.612 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.381+0000 7fbe07348700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:09:39.704 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.381+0000 7fbe07348700 -1 osd.6 172 *** Got signal Terminated *** 2024-04-23T19:09:39.766 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:38.381+0000 7fbe07348700 -1 osd.6 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:09:39.800 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:36.578+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:39.916 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:36.578+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:39.947 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:36.578+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:39.987 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:36.578+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:40.082 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:37.675+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:40.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:37.675+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:40.256 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:37.675+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:40.324 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:37.675+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:40.397 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:38.650+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:40.451 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:38.650+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:40.476 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:38.650+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:40.502 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:38.650+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:40.511 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:37.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:40.519 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:37.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:40.585 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:37.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:40.708 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:38.238+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:40.751 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:38.238+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:40.800 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:38.238+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:40.881 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:39.330+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:40.890 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:39.330+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:40.900 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:39.330+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:40.968 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:39.330+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:41.051 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:37.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:41.157 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:37.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:41.228 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:37.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:41.288 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:37.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:41.334 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:38.695+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:41.363 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:38.695+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:41.412 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:38.695+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:41.464 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:38.695+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:41.495 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:39.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:41.597 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:39.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:41.698 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:39.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:41.742 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:39.683+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:41.800 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:38.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:41.901 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:38.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:41.908 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:38.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:41.916 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:39.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:42.047 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:39.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:42.084 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:39.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:42.266 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:40.314+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:42.309 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:40.314+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:42.358 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:40.314+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:42.413 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:40.314+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:42.469 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:38.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:42.537 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:38.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:42.584 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:38.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:42.659 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:38.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:42.733 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:39.709+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:42.811 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:39.709+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:42.857 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:39.709+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:42.913 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:39.709+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:42.960 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:40.680+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:42.994 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:40.680+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:43.033 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:40.680+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:43.109 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:40.680+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:43.157 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:39.185+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:43.263 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:39.185+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:43.309 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:39.185+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:43.364 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:40.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:43.420 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:40.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:43.505 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:40.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:43.556 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:41.344+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:43.597 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:41.344+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:43.606 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:41.344+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:43.660 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:41.344+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:43.746 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:39.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:43.821 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:39.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:43.865 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:39.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:43.918 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:39.559+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:43.947 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:40.662+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:43.956 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:40.662+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:43.964 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:40.662+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:44.049 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:40.662+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:44.144 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:41.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:44.212 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:41.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:44.298 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:41.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:44.355 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:41.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:44.402 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:40.162+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:44.437 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:40.162+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:44.475 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:40.162+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:44.561 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:41.189+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:44.623 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:41.189+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:44.709 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:41.189+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:44.769 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:42.369+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:44.824 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:42.369+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:44.894 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:42.369+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:44.922 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:42.369+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:44.996 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:40.526+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:45.093 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:40.526+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:45.153 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:40.526+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:45.251 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:40.526+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:45.285 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:41.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:45.341 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:41.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:45.357 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:41.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:45.366 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:41.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:45.375 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:42.706+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:45.383 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:42.706+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:45.394 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:42.706+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:45.490 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:42.706+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:45.575 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:41.196+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:45.625 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:41.196+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:45.679 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:41.196+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:45.769 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:42.159+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:45.799 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:42.159+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:45.844 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:42.159+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:45.921 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:41.564+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:45.965 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:41.564+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:46.046 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:41.564+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:46.116 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:41.564+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:46.188 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:42.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:46.222 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:42.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:46.302 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:42.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:46.345 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:42.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:46.389 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:43.352+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:46.584483+0000 front 2024-04-23T19:08:46.584516+0000 (oldest deadline 2024-04-23T19:09:12.985557+0000) 2024-04-23T19:09:46.424 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:43.352+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.486719+0000 front 2024-04-23T19:08:53.486666+0000 (oldest deadline 2024-04-23T19:09:19.386776+0000) 2024-04-23T19:09:46.480 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:43.352+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.389453+0000 front 2024-04-23T19:09:04.389379+0000 (oldest deadline 2024-04-23T19:09:29.089444+0000) 2024-04-23T19:09:46.557 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:43.352+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.190265+0000 front 2024-04-23T19:09:10.190201+0000 (oldest deadline 2024-04-23T19:09:34.890405+0000) 2024-04-23T19:09:46.607 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6[57425]: 2024-04-23T19:09:43.352+0000 7fbdff934700 -1 osd.6 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.591734+0000 front 2024-04-23T19:09:19.591779+0000 (oldest deadline 2024-04-23T19:09:43.091829+0000) 2024-04-23T19:09:46.696 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:43 smithi033 podman[107135]: 2024-04-23 19:09:43.7970308 +0000 UTC m=+5.517481488 container died 3a3df1f940d8ba0dd9808d0c1cb6c3a8261e3e19ae50644543dfecfa08623efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:09:46.784 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:43.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:46.792 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:43.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:46.799 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:43.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:46.806 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:43.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:46.871 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:42.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:46.996 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:42.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:47.050 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:42.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:47.122 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:43.196+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:47.185 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:43.196+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:47.237 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:43.196+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:47.266 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:42.532+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:47.322 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:42.532+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:47.353 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:42.532+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:47.420 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:42.532+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:47.494 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:43.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:47.546 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:43.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:47.565 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:43.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:47.643 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:43.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:47.679 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:44 smithi033 podman[107135]: 2024-04-23 19:09:44.13208783 +0000 UTC m=+5.852538515 container cleanup 3a3df1f940d8ba0dd9808d0c1cb6c3a8261e3e19ae50644543dfecfa08623efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-23T19:09:47.781 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:44 smithi033 bash[107135]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6 2024-04-23T19:09:47.850 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:44 smithi033 podman[107152]: 2024-04-23 19:09:44.333222929 +0000 UTC m=+0.533117778 container remove 3a3df1f940d8ba0dd9808d0c1cb6c3a8261e3e19ae50644543dfecfa08623efc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-23T19:09:47.889 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:44 smithi033 podman[107246]: 2024-04-23 19:09:44.665559622 +0000 UTC m=+0.018898580 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:09:47.955 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:44 smithi033 podman[107246]: 2024-04-23 19:09:44.784226792 +0000 UTC m=+0.137565739 container create aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:09:48.014 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:43.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:48.068 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:43.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:48.170 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:43.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:48.222 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:43.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:48.231 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:44.201+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:48.240 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:44.201+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:48.248 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:44.201+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:48.320 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:44.201+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:48.399 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:44.644+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:48.455 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:44.644+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:48.505 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:44.644+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:48.585 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:44.644+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:48.593 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:43.494+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:48.602 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:43.494+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:48.611 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:43.494+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:48.682 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:43.494+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:48.797 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:44.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:48.859 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:44.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:48.941 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:44.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:49.020 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:44.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:49.070 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:44.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:49.123 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:44.183+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:49.186 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:44.183+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:49.271 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:44.183+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:49.318 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:44.183+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:49.397 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:45.229+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:49.484 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:45.229+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:49.572 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:45.229+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:49.654 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:45.229+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:49.663 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:45 smithi033 podman[107246]: 2024-04-23 19:09:45.055737758 +0000 UTC m=+0.409076698 container init aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T19:09:49.672 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:45 smithi033 podman[107246]: 2024-04-23 19:09:45.058983657 +0000 UTC m=+0.412322604 container start aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:09:49.702 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:45 smithi033 podman[107246]: 2024-04-23 19:09:45.138995807 +0000 UTC m=+0.492334775 container attach aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:09:49.726 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:45 smithi033 podman[107246]: 2024-04-23 19:09:45.361537011 +0000 UTC m=+0.714875944 container died aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:09:49.850 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:44.459+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:49.926 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:44.459+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:49.975 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:44.459+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:50.028 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:44.459+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:50.070 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:45.638+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:50.108 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:45.638+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:50.195 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:45.638+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:50.299 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:45.638+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:50.447 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:45.638+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:50.519 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:45.663+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:50.580 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:45.663+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:50.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:45.663+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:50.864 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:45.663+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:50.888 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:45.134+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:50.927 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:45.134+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:50.986 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:45.134+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:51.031 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:45.134+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:51.164 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:46.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:51.213 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:46.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:51.271 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:46.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:51.343 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:46.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:51.396 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:45.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:51.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:45.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:51.510 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:45.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:51.554 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:45.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:51.621 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:45.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:09:51.732 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:46.609+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:51.810 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:46.609+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:51.845 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:46.609+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:51.906 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:46.609+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:52.022 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:46.609+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:52.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.6.service' 2024-04-23T19:09:52.129 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:46 smithi033 podman[107272]: 2024-04-23 19:09:46.010676183 +0000 UTC m=+0.643553864 container remove aaae51c3bac784b206212b422ab3fe2f1b8ba51ab06b3cf1721a583d6200ca91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T19:09:52.138 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:46 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.6.service: Deactivated successfully. 2024-04-23T19:09:52.194 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:46 smithi033 systemd[1]: Stopped Ceph osd.6 for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:09:52.203 INFO:journalctl@ceph.osd.6.smithi033.stdout:Apr 23 19:09:46 smithi033 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.6.service: Consumed 25.719s CPU time. 2024-04-23T19:09:52.213 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:46.143+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:52.221 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:46.143+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:52.232 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:46.143+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:52.240 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:46.143+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:52.249 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:47.222+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:52.257 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:47.222+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:52.326 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:47.222+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:52.387 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:47.222+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:52.474 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:46.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:52.518 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:46.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:52.599 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:46.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:52.662 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:46.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:52.696 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:46.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:09:52.718 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:47.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:52.727 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:47.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:52.806 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:47.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:52.878 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:47.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:52.972 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:47.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:53.068 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:47.158+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:53.113 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:47.158+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:53.182 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:47.158+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:53.236 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:47.158+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:53.278 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:48.214+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:53.344 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:48.214+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:53.410 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:48.214+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:53.419 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:48.214+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:53.497 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:47.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:53.568 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:47.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:53.642 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:47.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:53.700 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:47.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:53.709 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:47.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:09:53.776 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:48.595+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:53.892 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:48.595+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:53.939 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:48.595+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:54.007 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:48.595+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:54.044 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:48.595+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:54.117 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:46.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:54.168 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:46.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:54.224 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:46.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:54.282 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:46.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:54.363 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:46.709+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:54.511 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:47.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:54.605 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:47.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:54.649 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:47.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:54.730 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:47.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:54.769 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:47.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:54.820 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:48.662+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:54.868 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:48.662+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:54.928 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:48.662+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:55.062 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:48.662+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:55.105 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:48.662+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:55.208 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:49.617+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:55.269 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:49.617+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:55.350 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:49.617+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:55.411 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:49.617+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:55.433 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:49.617+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:55.516 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:50.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:55.667 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:50.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:55.729 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:50.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:55.760 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:50.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:55.772 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:50.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:55.827 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:51.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:55.839 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:51.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:55.863 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:51.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:55.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:51.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:56.015 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:51.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:56.156 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:52.613+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:56.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:52.613+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:56.258 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:52.613+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:56.399 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:52.613+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:56.441 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:52.613+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:56.500 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:53.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:09:56.550 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:53.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:09:56.562 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:53.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:09:56.641 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:53.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:09:56.775 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:53.630+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:09:56.836 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:48.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:56.909 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:48.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:57.028 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:48.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:57.097 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:48.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:57.163 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:49.241+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:57.243 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:49.241+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:57.251 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:49.241+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:57.309 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:49.241+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:57.317 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:48.452+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:57.328 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:48.452+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:57.389 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:48.452+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:57.455 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:48.452+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:57.522 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:48.452+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:09:57.623 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:49.547+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:57.630 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:49.547+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:57.686 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:49.547+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:57.775 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:49.547+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:57.853 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:49.547+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:57.896 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:49.153+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:58.015 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:49.153+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:58.099 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:49.153+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:58.161 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:49.153+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:58.233 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:50.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:58.240 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:50.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:58.317 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:50.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:58.423 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:50.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:58.541 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:49.470+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:58.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:49.470+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:58.723 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:49.470+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:09:58.752 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:49.470+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:09:58.817 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:49.470+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:09:58.938 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:50.559+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:09:59.003 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:50.559+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:09:59.110 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:50.559+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:09:59.148 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:50.559+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:09:59.208 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:50.559+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:09:59.263 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:50.147+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:09:59.297 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:50.147+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:09:59.357 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:50.147+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:09:59.424 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:50.147+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:09:59.473 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:51.227+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:09:59.600 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:51.227+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:09:59.639 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:51.227+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:09:59.772 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:51.227+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:09:59.820 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:50.436+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:09:59.899 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:50.436+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:09:59.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:50.436+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:00.030 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:50.436+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:00.095 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:50.437+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:00.162 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:00.196 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:00.255 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:00.328 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:00.480 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:00.492 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:51.573+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:00.505 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:51.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:00.552 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:51.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:00.655 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:51.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:00.785 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:51.159+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:00.847 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:52.257+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:00.934 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:52.257+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:00.988 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:52.257+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:01.097 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:52.257+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:01.161 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:51.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:01.220 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:51.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:01.235 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:51.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:01.328 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:51.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:01.367 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:51.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:01.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:10:01.489 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:01.576 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:01.609 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:01.732 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:01.780 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:01.789 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:52.549+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:01.879 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-23T19:10:01.936 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-23T19:10:02.042 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.7 2024-04-23T19:10:02.115 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:52.119+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:02.153 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:52.119+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:02.203 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:52.119+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:02.236 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:52.119+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:02.273 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:53.282+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:02.354 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:53.282+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:02.421 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:53.282+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:02.440 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:53.282+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:02.450 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:52.412+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:02.500 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:52.412+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:02.547 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:52.412+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:02.610 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:52.412+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:02.685 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:52.412+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:02.755 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:02.865 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:02.944 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:03.008 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:03.086 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:03.139 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:53.546+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:03.180 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:53.123+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:03.234 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:53.123+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:03.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:53.123+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:03.355 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:53.123+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:03.412 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:54.275+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:03.469 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:54.275+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:03.515 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:54.275+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:03.590 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:54.275+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:03.641 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:53.458+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:03.731 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:53.458+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:03.758 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:53.458+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:03.768 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:53.458+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:03.862 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:53.458+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:03.925 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:04.036 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:04.084 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:04.150 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:04.208 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:04.314 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:54.504+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:04.365 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:04.436 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:04.506 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:04.590 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:04.738 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:04.812 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:54.606+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:04.846 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:04.910 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:04.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:05.036 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:05.118 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:05.158 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:55.645+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:05.169 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:05.199 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:05.256 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:05.323 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:05.367 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:05.398 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:56.595+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:05.532 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:05.544 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:05.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:05.570 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:05.641 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:05.743 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:57.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:05.807 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:05.887 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:05.939 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:06.025 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:06.113 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:06.187 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:58.637+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:06.248 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:06.260 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:06.314 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:06.407 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:06.461 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:06.520 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:09:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:09:59.603+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:06.531 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:06.542 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:06.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:06.630 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:06.799 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:06.855 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:00.578+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:06.924 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:06.965 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:07.094 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:07.130 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:07.227 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:07.307 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:01.598+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:07.379 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:07.406 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:07.480 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:07.557 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:07.633 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:07.689 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:02.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:07.708 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:54.097+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:07.718 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:54.097+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:07.823 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:54.097+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:07.892 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:54.097+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:07.937 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:54.097+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:07.992 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:55.284+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:08.001 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:55.284+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:08.104 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:55.284+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:08.174 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:55.284+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:08.233 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:55.284+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:08.329 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:08.356 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:08.366 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:08.376 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:08.385 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:08.460 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:54.486+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:08.516 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:08.547 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:08.642 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:08.718 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:08.778 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:08.791 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:55.517+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:08.848 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:55.115+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:08.909 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:55.115+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:09.015 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:55.115+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:09.071 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:55.115+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:09.114 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:55.115+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:09.199 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:56.278+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:09.280 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:56.278+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:09.346 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:56.278+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:09.385 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:56.278+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:09.427 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:56.278+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:09.516 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:09.581 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:09.638 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:09.661 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:09.757 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:09.800 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:55.438+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:09.871 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:09.982 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:10.144 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:10.209 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:10.217 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:10.227 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:56.553+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:10.258 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:56.139+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:10.282 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:56.139+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:10.460 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:56.139+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:10.481 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:56.139+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:10.491 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:56.139+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:10.546 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:57.251+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:10.586 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:57.251+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:10.662 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:57.251+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:10.757 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:57.251+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:10.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:57.251+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:10.850 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:10.860 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:10.921 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:11.002 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:11.109 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:11.180 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:56.401+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:11.247 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:11.321 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:11.403 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:11.461 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:11.581 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:11.590 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:57.586+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:11.684 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:57.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:11.772 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:57.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:11.823 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:57.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:11.879 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:57.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:11.972 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:57.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:12.083 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:58.293+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:12.116 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:58.293+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:12.175 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:58.293+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:12.185 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:58.293+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:12.299 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:58.293+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:12.365 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:12.424 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:12.486 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:12.520 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:12.593 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:12.635 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:57.419+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:12.725 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:12.814 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:12.866 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:12.911 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:12.977 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:13.052 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:58.582+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:13.121 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:58.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:13.131 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:58.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:13.204 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:58.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:13.257 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:58.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:13.412 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:58.140+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:13.481 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:59.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:13.516 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:59.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:13.568 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:59.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:13.578 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:59.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:13.590 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:09:59.266+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:13.667 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:13.713 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:13.775 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:13.834 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:13.849 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:13.941 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:58.426+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:14.021 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:14.074 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:14.124 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:14.191 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:14.203 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:14.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:09:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:09:59.579+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:14.413 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:14.497 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:14.546 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:14.633 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:14.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:14.790 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:03.505+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:14.831 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:14.882 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:14.899 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:14.911 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:14.923 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:14.971 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:04.494+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:14.983 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:14.991 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:15.058 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:15.126 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:15.357 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:15.385 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:05.457+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:15.395 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:15.416 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:15.429 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:15.442 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:15.454 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:15.460 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:06.444+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:15.497 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:15.505 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:15.515 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:15.523 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:15.624 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:15.708 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:15.787 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:07.476+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:15.888 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:15.905 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:16.014 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:16.105 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:16.171 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:16.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:16.193 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:08.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:16.202 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:16.213 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:16.265 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:16.333 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:16.402 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:16.470 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:16.511 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:09.530+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:16.594 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:16.640 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:16.694 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:16.744 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:16.788 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:16.973 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:17.118 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:10.483+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:17.175 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.481+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:17.201 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:17.229 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:17.307 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:17.327 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:17.364 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:17.445 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:11.482+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:17.488 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:17.545 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:17.577 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:17.650 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:17.711 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:17.756 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:17.827 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:12.514+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:17.873 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:59.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:17.981 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:59.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:18.085 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:59.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:18.128 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:59.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:18.179 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:09:59.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:18.242 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:00.250+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:18.252 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:00.250+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:18.264 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:00.250+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:18.283 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:00.250+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:18.353 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:00.250+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:18.392 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:18.435 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:18.500 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:18.563 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:18.573 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:18.586 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:00.618+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:18.680 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:18.756 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:18.889 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:18.947 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:18.998 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:19.046 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:09:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:09:59.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:19.188 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:19.219 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:19.231 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:19.244 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:19.319 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:19.411 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:00.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:19.490 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:00.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:19.545 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:00.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:19.591 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:00.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:19.646 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:00.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:19.767 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:00.208+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:19.895 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:01.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:19.908 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:01.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:19.989 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:01.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:20.066 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:01.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:20.129 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:01.236+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:20.266 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:21.247 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:21.776 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:21.825 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:21.872 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:21.932 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:01.622+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:21.940 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:21.949 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:21.958 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:22.047 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:22.091 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:22.215 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:01.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:22.224 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:01.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:22.305 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:01.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:22.456 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:01.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:22.532 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:01.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:22.614 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:01.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:22.657 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:02.224+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:22.722 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:02.224+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:22.763 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:02.224+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:22.868 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:02.224+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:22.903 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:02.224+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:22.975 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:23.035 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:23.081 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:23.203 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:23.278 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:23.331 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:02.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:23.399 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:23.461 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:23.537 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:23.669 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:23.690 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:23.778 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:02.371+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:23.855 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:02.197+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:23.909 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:02.197+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:23.972 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:02.197+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:23.982 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:02.197+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:23.991 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:02.197+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:24.102 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:03.243+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:24.161 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:03.243+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:24.242 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:03.243+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:24.315 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:03.243+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:24.390 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:03.243+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:24.399 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:03.202+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:24.410 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:03.202+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:24.481 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:03.202+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:24.534 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:03.202+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:24.689 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:03.202+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:24.783 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:24.824 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:24.874 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:24.921 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:24.963 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:25.021 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:03.323+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:25.075 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:25.172 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:25.184 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:25.208 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:25.273 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:25.397 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:03.653+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:25.439 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:04.224+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:25.481 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:04.224+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:25.535 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:04.224+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:25.588 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:04.224+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:25.709 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:04.224+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:25.778 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:04.198+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:25.815 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:04.198+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:25.848 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:04.198+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:25.874 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:04.198+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:25.940 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:04.198+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:25.988 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:26.033 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:26.141 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:26.173 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:26.227 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:26.237 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:04.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:26.247 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:26.258 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:26.322 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:26.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:26.471 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:26.570 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:26.609 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:04.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:26.620 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:26.629 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:26.683 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:26.766 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:26.840 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:26.882 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:05.249+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:26.951 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:27.004 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:27.060 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:27.149 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:27.226 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:27.257 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:05.223+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:27.268 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:27.323 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:27.430 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:27.499 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:27.592 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:27.601 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:05.329+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:27.611 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:27.619 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:27.627 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:27.635 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:27.643 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:27.721 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:27.764 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:05.699+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:27.844 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:27.917 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:27.988 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:28.032 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:28.097 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:28.158 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:28.209 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:13.513+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:28.310 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:28.351 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:28.446 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:28.511 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:28.616 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:28.630 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:28.639 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:14.553+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:28.649 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:28.726 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:28.756 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:28.836 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:28.907 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:29.001 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:29.013 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:15.556+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:29.024 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:29.108 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:29.145 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:29.234 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:29.335 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:29.414 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:29.432 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:16.541+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:29.488 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:29.580 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:29.684 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:29.711 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:29.811 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:29.854 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:29.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:17.555+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:30.015 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:30.103 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:30.184 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:30.308 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:30.373 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:30.393 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:30.521 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:18.604+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:30.566 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:30.639 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:30.702 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:30.716 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:30.728 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:30.739 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:30.787 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:19.620+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:30.849 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:30.872 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:30.910 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:30.952 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:31.003 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:31.055 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:31.079 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:20.593+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:31.182 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:31.276 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:31.418 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:31.445 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:31.521 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:31.601 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:31.652 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:21.640+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:31.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:31.748 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:31.758 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:31.835 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:31.877 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:31.934 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:32.027 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:22.659+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:32.113 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:32.125 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:32.161 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:32.192 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:32.201 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:32.282 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:32.290 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:23.671+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:32.299 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:32.310 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:32.321 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:32.332 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:32.344 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:32.404 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:32.518 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:24.658+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:32.548 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:32.614 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:32.624 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:32.691 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:32.740 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:32.785 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:32.895 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:25.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:32.906 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:33.021 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:33.050 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:33.148 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:33.209 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:33.255 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:33.277 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:26.687+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:33.286 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:33.295 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:33.362 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:33.429 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:33.473 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:33.523 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:06.259+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:33.594 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:33.601 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:33.609 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:33.678 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:33.710 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:33.853 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:06.179+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:33.910 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:33.990 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:34.035 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:34.120 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:34.189 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:34.264 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:06.322+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:34.329 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:34.392 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:34.477 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:34.543 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:34.605 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:34.685 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:34.718 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:06.654+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:34.728 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:34.789 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:34.864 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:34.929 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:35.013 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:35.076 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:07.219+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:35.103 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:35.113 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:35.123 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:35.133 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:35.142 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:35.228 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:07.177+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:35.292 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:35.375 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:35.456 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:35.511 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:35.563 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:35.646 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:07.324+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:35.741 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:35.751 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:35.858 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:35.887 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:35.943 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:36.005 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:36.112 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:07.626+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:36.172 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:36.210 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:36.257 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:36.328 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:36.425 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:36.434 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:08.188+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:36.558 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:36.621 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:36.696 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:36.740 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:36.787 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:36.800 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:08.186+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:36.920 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:36.980 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:37.050 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:37.127 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:37.184 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:37.266 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:08.306+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:37.383 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:37.456 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:37.518 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:37.549 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:37.591 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:37.713 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:37.796 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:08.665+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:37.832 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:37.843 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:37.943 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:38.024 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:38.137 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:38.174 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:09.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:38.221 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:38.278 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:38.304 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:38.374 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:38.487 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:38.521 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:09.150+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:38.575 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:38.682 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:38.726 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:38.871 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:38.938 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:39.005 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:09.260+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:39.142 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:39.204 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:39.214 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:39.286 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:39.334 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:39.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:39.443 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:09.673+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:39.510 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:39.576 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:39.657 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:39.733 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:39.859 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:39.911 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:10.141+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:39.935 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:40.014 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:40.082 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:40.141 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:40.196 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:40.262 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:10.181+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:40.273 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:40.398 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:40.478 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:40.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:40.592 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:40.634 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:10.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:40.673 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:40.734 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:40.811 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:40.926 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:40.981 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:41.005 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:41.053 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:10.642+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:41.127 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:41.185 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:41.213 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:41.288 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:41.354 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:41.479 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:41.490 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:27.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:41.554 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:41.586 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:41.617 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:41.673 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:41.736 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:41.803 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:41.898 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:28.727+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:41.971 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:42.018 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:42.113 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:42.171 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:42.236 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:42.318 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:42.376 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:29.767+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:42.385 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:42.467 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:42.553 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:42.599 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:42.683 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:42.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:42.721 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:30.780+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:42.730 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:42.738 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:42.789 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:42.842 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:42.899 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:42.984 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:43.052 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:31.770+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:43.110 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:43.118 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:43.186 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:43.264 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:43.333 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:43.416 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:43.570 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:32 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:32.776+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:43.591 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:43.661 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:43.739 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:43.806 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:43.877 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:43.926 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:43.934 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:33 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:33.777+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:43.941 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:43.987 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:44.122 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:44.204 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:44.244 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:44.301 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:44.310 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:34 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:34.732+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:44.413 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:44.488 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:44.580 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:44.669 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:44.701 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:44.741 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:44.820 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:35 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:35.694+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:44.937 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:44.997 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:45.006 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:45.016 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:45.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:45.254 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:45.320 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:36 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:36.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:45.350 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:45.360 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:45.468 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:45.557 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:45.607 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:45.643 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:45.709 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:37 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:37.669+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:45.739 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:45.755 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:45.763 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:45.774 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:45.847 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:45.907 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:45.975 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:38 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:38.714+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:46.060 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:46.104 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:46.114 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:46.123 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:46.162 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:46.171 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:46.259 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:39 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:39.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:46.312 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:46.376 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:46.430 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:46.470 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:46.531 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:46.584 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:46.593 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:40 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:40.725+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:46.690 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:46.716 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:46.780 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:46.865 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:46.935 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:46.943 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:11.172+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:46.951 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:46.959 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:46.996 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:47.011 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:47.018 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:47.026 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:11.134+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:47.034 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.691+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:47.048 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.691+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:47.055 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.691+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:47.065 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.691+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:47.117 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.691+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:47.185 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.692+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:47.247 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:11.692+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:47.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:47.372 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:47.420 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:47.429 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:47.436 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:47.466 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:11.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:47.474 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:47.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:47.630 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:47.733 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:47.780 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:47.787 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:12.303+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:47.795 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:47.802 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:47.809 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:47.861 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:47.868 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:47.878 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:12.173+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:47.928 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:47.996 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:48.089 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:48.210 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:48.299 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:48.428 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:12.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:48.510 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:48.588 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:48.638 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:48.719 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:48.745 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:48.782 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:48.811 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:12.657+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:48.887 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:49.056 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:49.065 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:49.083 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:49.196 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:49.350 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:13.330+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:49.373 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:49.440 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:49.520 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:49.603 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:49.669 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:49.760 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:13.206+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:49.815 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:49.906 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:49.947 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:49.974 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:49.982 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:49.991 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:13.142+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:50.060 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:50.091 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:50.134 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:50.216 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:50.269 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:50.307 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:50.316 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:13.684+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:50.377 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:50.425 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:50.557 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:50.611 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:50.696 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:50.738 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:14.292+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:50.759 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:50.822 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:50.942 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:50.997 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:51.092 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:51.201 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:14.252+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:51.304 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:51.404 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:51.505 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:51.542 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.213+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:51.604 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.214+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:51.670 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:15.214+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:51.728 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.145+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:51.791 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.145+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:51.842 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.145+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:51.924 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.146+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:51.973 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.146+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:52.044 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:14.146+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:52.077 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:52.208 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:52.277 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:52.422 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:52.510 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:52.569 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:15.273+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:52.681 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:52.741 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:52.823 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:52.887 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:52.944 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:52.952 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:53.035 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:14.688+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:53.102 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:53.147 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:53.183 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:53.213 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:53.265 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:53.323 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:16.207+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:53.332 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:53.410 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:53.418 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:53.426 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:53.503 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:53.600 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:15.124+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:53.651 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:53.659 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:53.726 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:53.838 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:53.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:53.995 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:16.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:54.018 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:54.065 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:54.108 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:54.151 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:54.181 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:54.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:54.266 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:41 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:41.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:54.331 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:54.373 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:54.477 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:54.571 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:54.614 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:54.668 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:54.735 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:42 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:42.715+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:54.745 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:54.839 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:54.904 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:54.999 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:55.092 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:55.215 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:55.282 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:43 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:43.711+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:55.349 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:55.417 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:55.476 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:55.564 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:55.613 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:55.721 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:55.829 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:44 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:44.730+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:55.901 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:55.964 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.036 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.060 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.135 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.196 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.266 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:45 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:45.699+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.347 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.448 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.505 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.522 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.534 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.544 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.557 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:46 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:46.676+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.565 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.574 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.582 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.590 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.599 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.608 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.617 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:47 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:47.639+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.625 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.634 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.641 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.650 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.658 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.730 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.739 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:48 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:48.646+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.748 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.756 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.764 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.773 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.781 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.789 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.798 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.806 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:49 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:49.621+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:56.814 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.823 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.831 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.839 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.847 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.855 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.863 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.894 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:50.616+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:56.902 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.910 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.918 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.926 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.934 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:56.942 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:56.949 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:56.956 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:51 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:51.592+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:56.964 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:56.971 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:56.978 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:56.985 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:56.993 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:57.000 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:57.007 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:57.014 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:52 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:52.602+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:57.051 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:57.058 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:57.065 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:57.073 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:57.080 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:57.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:57.095 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:15.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:57.103 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:57.110 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:57.117 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:57.125 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:57.132 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:57.138 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:17.209+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:57.146 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:57.153 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:57.160 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:57.168 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:57.214 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:57.223 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:16.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:57.231 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:57.238 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:57.245 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:57.252 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:57.259 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:57.266 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:57.276 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:16.729+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:57.283 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:57.290 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:57.297 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:57.304 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:57.312 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:57.320 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:17.360+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:57.330 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:57.338 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:57.386 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:57.395 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:57.404 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:57.413 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:57.423 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:57.431 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:57.440 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:57.449 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:57.459 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:57.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:18.205+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:57.477 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.102+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:57.486 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.102+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:57.495 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.102+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:57.504 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:57.513 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:57.521 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:17.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:57.531 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:57.604 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:57.643 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:57.652 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:57.661 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:57.670 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:57.679 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:17.743+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:57.688 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:57.698 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:57.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:57.715 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:57.724 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:57.733 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:19.393+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:57.743 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:57.751 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:57.760 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:57.769 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:57.778 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:57.944 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:19.180+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:57.958 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:57.967 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:57.975 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:57.984 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:57.993 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:58.002 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:18.103+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:58.011 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:58.019 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:58.028 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:58.036 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:58.045 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:58.054 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:58.063 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:18.728+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:58.072 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:58.081 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:58.090 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:58.130 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:58.138 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:58.147 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:20.442+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:58.156 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:58.165 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:58.174 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:58.182 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:58.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:58.200 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:58.209 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:58.217 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:53 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:53.622+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:58.226 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:58.235 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:58.243 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:58.252 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:58.261 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:58.270 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:58.326 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:58.335 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:54 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:54.666+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:58.342 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:58.351 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:58.358 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:58.365 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:58.372 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:58.379 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:58.386 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:58.394 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:55 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:55.693+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:58.401 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:58.408 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:58.415 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:58.423 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:58.430 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:58.437 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:58.491 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:58.561 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:56.686+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:58.568 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:58.575 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:58.582 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:58.589 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:58.596 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:58.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:58.612 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:58.619 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:57 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:57.697+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:58.626 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:58.633 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:58.641 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:58.648 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:58.655 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:58.670 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:20.211+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:58.677 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:58.684 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:58.768 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:58.775 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:58.781 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:58.788 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:21.210+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:58.795 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:58.802 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:58.808 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:58.815 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:58.822 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:58.829 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:19.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:58.837 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:58.845 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:58.854 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:58.863 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:58.871 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:58.880 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:21.448+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:58.889 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:58.955 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:58.963 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:58.972 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:58.980 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:58.989 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:58.997 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:19.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:59.007 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.233+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:59.016 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.233+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:59.024 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.233+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:59.033 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.233+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:59.041 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.233+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:59.050 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:22.234+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:59.059 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:59.068 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:59.077 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:59.085 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:59.094 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:59.151 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:20.132+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:59.202 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:59.210 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:59.219 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:59.227 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:59.237 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:59.245 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:22.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:59.254 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:59.262 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:59.270 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:59.279 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:59.288 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:59.297 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:59.306 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:20.686+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:59.315 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:10:59.324 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:10:59.332 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:10:59.367 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:10:59.390 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:10:59.399 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:10:59.407 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:10:59.416 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:58 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:58.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:10:59.425 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:59.433 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:59.442 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:59.450 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:59.459 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:59.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:23.190+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:59.476 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:59.485 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:59.494 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:59.510 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:59.518 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:59.526 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:21.109+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:59.568 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:59.612 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:59.621 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:59.629 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:59.639 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:59.647 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:23.449+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:59.657 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:59.666 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:59.675 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:10:59.683 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:10:59.692 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:10:59.701 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:10:59.709 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:21.702+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:10:59.718 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:10:59.728 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:10:59.737 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:10:59.746 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:10:59.785 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:10:59.793 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:24.217+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:10:59.801 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:21 smithi106 systemd[1]: Stopping Ceph osd.7 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:10:59.808 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:10:59.815 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:10:59.823 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:10:59.830 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:10:59.837 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:10:59.844 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.100+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:10:59.851 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.232+0000 7f2d824a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:10:59.859 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.232+0000 7f2d824a5700 -1 osd.7 172 *** Got signal Terminated *** 2024-04-23T19:10:59.866 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:22.232+0000 7f2d824a5700 -1 osd.7 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:10:59.873 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:10:59.881 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:10:59.889 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:10:59.897 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:10:59.904 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:10:59.970 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:24.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:10:59.978 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:10:59.985 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:10:59.993 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:00.000 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:00.008 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:00.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:00.055 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:22.718+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:00.065 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:00.072 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:00.081 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:00.089 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:00.097 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:00.105 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:25.187+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:00.114 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:11:00.122 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:11:00.131 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:11:00.200 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:11:00.220 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:11:00.229 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:23.149+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:11:00.238 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:00.246 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:00.255 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:00.263 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:00.270 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:00.280 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:00.288 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:00.296 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:10:59 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:10:59.651+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:00.305 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:00.314 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:00.322 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:00.331 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:00.340 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:00.350 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:25.450+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:00.410 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:00.441 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:00.452 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:00.461 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:00.469 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:00.478 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:00.486 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:23.689+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:00.495 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:00.504 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:00.513 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:00.521 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:00.530 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:00.539 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:26.169+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:00.549 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:11:00.562 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:11:00.571 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:11:00.579 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:11:00.662 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:11:00.685 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:24.106+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:11:00.695 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:00.703 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:00.712 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:00.721 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:00.730 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:00.738 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:26.498+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:00.747 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:00.755 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:00.765 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:00.776 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:00.785 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:00.793 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:00.802 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:24.651+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:00.812 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:00.821 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:00.883 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:00.919 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:00.927 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:00.936 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:27.133+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:00.945 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:11:00.953 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:11:00.962 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:11:00.970 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:11:00.978 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:11:00.987 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:25.067+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:11:00.998 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:01.007 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:01.017 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:01.025 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:01.034 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:01.042 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:27.508+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:01.050 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:01.112 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:01.121 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:01.130 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:01.138 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:01.151 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:01.159 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:25.698+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:01.168 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:01.176 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:01.185 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:01.194 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:01.202 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:01.211 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:28.174+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:01.221 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:11:01.230 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:11:01.239 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:11:01.247 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:11:01.256 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:11:01.313 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:26.075+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:11:01.322 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:01.329 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:01.337 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:01.344 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:01.352 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:01.359 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:28.536+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:01.366 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:01.373 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:01.379 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:01.386 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:01.393 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:01.401 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:01.443 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:26.745+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:02.898 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:02.920 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:02.932 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:03.057 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:03.067 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:03.076 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:03.084 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:03.093 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:00.690+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:03.102 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:03.113 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:03.121 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:03.129 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:03.138 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:03.147 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:29.148+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:03.156 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:03.171 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:03.180 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:03.189 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:03.197 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:03.228 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:29.496+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:03.284 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.186176+0000 front 2024-04-23T19:08:55.186199+0000 (oldest deadline 2024-04-23T19:09:17.486449+0000) 2024-04-23T19:11:03.294 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:03.288331+0000 front 2024-04-23T19:09:03.288241+0000 (oldest deadline 2024-04-23T19:09:28.488411+0000) 2024-04-23T19:11:03.302 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.488914+0000 front 2024-04-23T19:09:08.488978+0000 (oldest deadline 2024-04-23T19:09:34.389082+0000) 2024-04-23T19:11:03.311 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.689905+0000 front 2024-04-23T19:09:19.689939+0000 (oldest deadline 2024-04-23T19:09:43.790099+0000) 2024-04-23T19:11:03.319 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:28.392213+0000 front 2024-04-23T19:09:28.392058+0000 (oldest deadline 2024-04-23T19:09:54.292000+0000) 2024-04-23T19:11:03.328 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7[50966]: 2024-04-23T19:10:27.088+0000 7f2d7aa91700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:41.694396+0000 front 2024-04-23T19:09:41.694385+0000 (oldest deadline 2024-04-23T19:10:04.594339+0000) 2024-04-23T19:11:03.337 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:27 smithi106 podman[76681]: 2024-04-23 19:10:27.73543017 +0000 UTC m=+5.604987809 container died eeace3722f09ce0af6d45cb67cae279b95b31cc5835e66177611d82fb13ea053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T19:11:03.346 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:03.355 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:03.364 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:03.372 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:03.380 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:03.389 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:03.397 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:27.783+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:03.406 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:03.415 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:03.471 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:03.567 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:03.575 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:03.582 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:30.120+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:03.590 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:03.597 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:03.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:03.612 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:03.619 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:03.626 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:30.543+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:03.634 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76681]: 2024-04-23 19:10:28.032312586 +0000 UTC m=+5.901870232 container cleanup eeace3722f09ce0af6d45cb67cae279b95b31cc5835e66177611d82fb13ea053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:11:03.641 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 bash[76681]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7 2024-04-23T19:11:03.650 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76699]: 2024-04-23 19:10:28.300306687 +0000 UTC m=+0.557577981 container remove eeace3722f09ce0af6d45cb67cae279b95b31cc5835e66177611d82fb13ea053 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-23T19:11:03.657 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76792]: 2024-04-23 19:10:28.601100586 +0000 UTC m=+0.114722709 container create 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-23T19:11:03.665 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76792]: 2024-04-23 19:10:28.504149626 +0000 UTC m=+0.017771706 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:11:03.672 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76792]: 2024-04-23 19:10:28.836879577 +0000 UTC m=+0.350501654 container init 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-23T19:11:03.684 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76792]: 2024-04-23 19:10:28.839927449 +0000 UTC m=+0.353549535 container start 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T19:11:03.717 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:03.774 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:03.817 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:03.823 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:03.830 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:03.837 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:31.105+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:03.845 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:03.855 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:03.864 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:03.872 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:03.880 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:03.890 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:03.898 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:28.742+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:03.907 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:03.915 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:03.923 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:03.932 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:03.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:04.058 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:31.579+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:04.068 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:28 smithi106 podman[76792]: 2024-04-23 19:10:28.937001226 +0000 UTC m=+0.450623328 container attach 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:11:04.077 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:29 smithi106 podman[76792]: 2024-04-23 19:10:29.126115947 +0000 UTC m=+0.639738027 container died 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T19:11:04.085 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:04.094 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:04.103 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:04.112 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:04.120 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:04.129 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:32.137+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:04.139 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:04.148 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:04.156 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:04.165 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:04.173 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:04.181 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:32.541+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:04.190 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:04.249 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:04.341 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:04.355 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:04.364 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:04.373 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:04.381 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:04.390 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:01 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:01.679+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:04.398 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:04.407 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:04.415 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:04.424 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:04.434 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:04.443 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:04.452 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:04.461 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:02 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:02.634+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:04.469 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:04.478 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:04.515 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:04.577 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:04.587 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:04.596 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:04.605 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:04.613 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:03 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:03.615+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:04.623 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:04.632 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:04.641 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:04.650 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:04.660 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:04.668 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:33.152+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:04.678 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:04.687 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:04.696 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:04.705 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:04.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:04.762 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:33.521+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:04.818 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.7.service' 2024-04-23T19:11:04.831 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:29 smithi106 podman[76819]: 2024-04-23 19:10:29.690754244 +0000 UTC m=+0.554372040 container remove 29c8603454e4deef56025139e9a3caf625e473ffba4c11c13bfeddd771060bfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-23T19:11:04.840 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:29 smithi106 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-23T19:11:04.849 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:29 smithi106 systemd[1]: Stopped Ceph osd.7 for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:11:04.858 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 23 19:10:29 smithi106 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.7.service: Consumed 28.005s CPU time. 2024-04-23T19:11:04.869 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:04.878 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:04.887 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:04.896 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:04.908 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:04.917 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:34.118+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:04.926 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:04.935 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:04.945 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:04.954 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:04.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:04.994 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:34.474+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.047 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.056 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.065 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.073 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.081 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.090 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:35.109+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.100 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:05.110 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:05.119 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:05.127 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:05.136 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:05.145 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:35.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.155 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.164 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.171 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.178 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.205 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.254 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:36.072+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.262 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:05.269 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:05.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:05.283 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:05.290 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:05.297 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:36.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.306 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.314 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.321 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.328 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.336 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.343 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:37.024+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.350 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.431+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:05.358 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:05.365 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:05.409 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:05.450 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:05.479 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:37.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.489 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:05.500 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:05.509 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:05.517 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:05.526 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:05.534 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:05.543 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:05.551 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:04 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:04.648+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:05.560 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.569 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.577 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.584 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.594 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.601 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:38.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.643 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:05.694 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:05.705 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:05.714 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:05.724 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:05.734 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:38.410+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.743 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.752 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.760 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.769 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.777 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.049+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.785 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:39.049+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.794 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:05.803 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:05.811 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:05.820 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:05.829 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:05.867 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:39.435+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:05.937 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:05.945 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:05.956 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:05.965 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:05.973 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:05.982 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:40.000+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:05.993 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:06.003 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:06.024 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:06.039 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:06.049 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:06.058 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:40.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:06.068 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:06.077 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:06.086 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:06.094 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:06.159 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:06.219 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.010+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:06.229 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:06.239 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:06.248 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:06.257 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:06.266 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:06.275 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:06.283 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:06.292 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:05 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:05.668+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:06.302 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:06.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:06.319 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:06.329 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:06.338 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:06.346 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:41.429+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:06.357 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:06.413 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:06.472 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:06.481 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:06.489 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:06.498 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:41.997+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:06.507 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:06.516 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:06.525 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:06.534 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:06.543 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:06.552 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:42.380+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:06.563 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.573 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.582 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:06.591 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:06.611 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:06.620 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:06.657 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:29.772+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:06.704 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.712 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.722 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:06.731 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:06.739 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:06.748 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:06.758 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:30.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:06.766 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.774 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.783 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:06.791 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:06.801 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:06.808 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:06.817 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:31.820+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:06.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.832 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.859 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:06.895 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:06.903 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:06.912 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:06.922 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:32.795+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:06.929 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.937 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.944 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:06.951 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:06.958 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:06.965 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:06.974 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:33.770+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:06.981 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:06.988 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:06.996 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.003 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.010 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.050 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.109 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:34.759+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.116 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.125 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.133 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.141 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.148 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.155 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.163 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:35.788+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.170 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.177 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.185 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.192 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.199 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.208 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.215 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:36.761+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.222 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.285 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.326 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.336 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.344 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.353 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.362 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:37.712+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.371 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.379 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.388 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.405 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.413 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.422 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:38.753+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.431 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.438 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.448 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.456 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.500 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.552 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.559 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:39.751+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.568 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.748+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.576 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.593 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.602 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.610 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.619 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:40 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:40.749+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.628 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.636 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.645 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.654 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.692 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.701 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.709 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:41 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:41.727+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.750 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.813 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.828 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.837 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.846 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.855 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.865 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:42 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:42.687+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.873 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.882 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.891 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:07.900 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:07.909 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:07.919 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:07.931 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:43 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:43.734+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:07.940 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:07.951 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:07.959 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.000 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.093 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.102 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.110 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:44 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:44.735+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.119 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.137 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.145 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.154 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.163 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.171 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:45 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:45.779+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.180 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.188 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.201 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.209 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.218 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.226 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.268 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:46 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:46.810+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.346 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.357 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.366 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.383 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.391 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.399 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:47 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:47.844+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.408 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.416 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.425 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.434 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.442 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.451 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.459 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:48 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:48.834+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.468 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.476 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.516 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.568 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.577 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.583 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.589 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:49 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:49.879+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.597 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.604 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.611 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.617 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.625 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.633 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.640 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:50 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:50.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.647 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.655 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.662 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.668 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.675 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.716 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.773 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:51 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:51.846+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.781 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.788 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.795 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.803 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.809 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.817 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.824 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:52 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:52.873+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.833 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.841 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:08.848 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:08.861 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:08.868 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:08.876 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:08.883 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:53 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:53.898+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:08.892 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:08.933 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.026 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.035 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.043 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.055 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.064 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:54 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:54.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.073 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.083 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.092 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.101 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.110 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.119 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:55 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:55.920+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.137 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.157 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.167 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.208 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.284 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.299 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:56 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:56.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.308 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.318 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.326 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.334 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.343 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.352 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.361 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:57 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:57.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.371 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.379 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.388 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.405 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.413 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.421 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:58 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:58.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.466 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.523 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.533 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.550 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.560 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.568 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.577 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:10:59 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:10:59.880+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.594 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.602 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.611 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.620 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.629 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.638 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:00 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:00.832+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.646 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.656 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.665 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.717 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.778 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.790 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.799 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:01 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:01.838+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.808 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.817 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.826 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.835 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.845 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.853 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.862 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:02 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:02.817+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:09.870 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:09.879 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:09.888 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:09.896 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:09.905 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:09.915 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:09.948 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:03 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:03.798+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:10.017 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:10.027 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:10.035 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:10.044 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:10.053 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:10.062 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:10.071 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:04.829+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:10.081 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:10.091 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:10.105 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:10.114 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:10.123 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:10.132 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:10.141 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:05 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:05.860+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:10.151 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:10.160 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:10.198 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:10.253 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:10.264 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:10.274 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:42.954+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:10.284 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:10.292 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:10.301 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:10.310 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:10.319 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:10.327 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:43.377+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:10.336 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:10.345 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:10.354 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:10.362 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:10.370 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:10.378 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:43.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:10.386 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:10.423 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:10.467 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:10.475 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:10.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:10.491 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:44.398+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:10.500 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:10.508 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:10.516 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:10.525 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:10.533 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:10.541 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:10.548 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:10.555 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:06 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:06.698+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:10.562 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:10.569 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:10.577 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:10.584 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:10.658 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:10.726 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:10.734 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:10.744 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:07 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:07.707+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:10.753 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:10.762 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:10.771 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:10.780 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:10.790 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:10.801 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:10.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:10.820 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:08 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:08.733+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:10.829 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.757+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:10.838 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.757+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:10.847 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.757+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:10.856 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.757+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:10.866 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.757+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:10.906 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.758+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:10.967 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.758+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:11.015 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:09 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:09.758+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:11.025 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:11.035 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:11.045 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:11.054 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:11.065 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:11.075 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:44.933+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:11.084 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:11.097 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:11.106 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:11.115 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:11.124 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:11.134 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:45.378+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:11.143 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:11.152 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-23T19:11:11.192 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-23T19:11:11.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.8 2024-04-23T19:11:11.244 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:11.253 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:11.263 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:11.271 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:11.280 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:11.289 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:45.966+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:11.298 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:11.307 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:11.316 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:11.325 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:11.337 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:11.346 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:46.367+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:11.355 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:11.363 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:11.372 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:11.406 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:11.485 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:11.495 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:11.504 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:06 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:06.830+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:11.513 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:11.522 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:11.531 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:11.540 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:11.548 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:11.557 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:11.566 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:07 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:07.791+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:11.574 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:11.585 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:11.593 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:11.602 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:11.610 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:11.618 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:11.683 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:08.823+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:11.744 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:11.753 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:11.761 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:11.771 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:11.779 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:11.788 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:11.797 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:09 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:09.858+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:11.805 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:11.812 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:11.821 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:11.830 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:11.839 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:11.848 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:11.857 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:11.865 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:10 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:10.716+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:11.874 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:11.942 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:11.989 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.006 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:12.014 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:12.021 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:46.949+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:12.033 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:12.044 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:12.055 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:12.065 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:12.074 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:12.083 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:47.332+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:12.092 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:12.101 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:12.110 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.119 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:12.128 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:12.136 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:47.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:12.219 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:12.270 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:12.280 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:12.289 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:12.299 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:12.308 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:48.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:12.317 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:12.326 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:12.335 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.344 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:12.353 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:12.362 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:48 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:48.977+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:12.371 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:12.381 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:12.389 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:12.399 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:12.408 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:12.475 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:12.539 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:49.313+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:12.549 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:12.558 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:12.568 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.577 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:12.586 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:12.595 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:12.604 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:49 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:49.961+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:12.613 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:12.624 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:12.633 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:12.642 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:12.651 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:12.661 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:12.670 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:50.356+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:12.679 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:12.750 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:12.822 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.832 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:12.840 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:12.849 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:12.858 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:50 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:50.939+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:12.867 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:12.877 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:12.886 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:12.895 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:12.904 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:12.913 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:12.922 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:51.314+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:12.931 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:12.940 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:12.949 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:12.958 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:13.017 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:13.056 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:13.082 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:51 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:51.982+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:13.092 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:13.101 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:13.110 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:13.118 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:13.127 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:13.136 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:13.145 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:52.268+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:13.159 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:13.167 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:13.176 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:13.185 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:13.194 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:13.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:13.211 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:13.300 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:11 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:11.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:13.348 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:13.356 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:13.364 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:13.372 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:13.380 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:13.389 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:13.397 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:10 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:10.904+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:13.406 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:13.414 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:13.422 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:13.430 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:13.439 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:13.447 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:13.455 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:11 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:11.944+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:13.465 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:13.474 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:13.533 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:13.587 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:13.595 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:13.602 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:13.610 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:52 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:52.987+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:13.618 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:13.625 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:13.632 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:13.640 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:13.649 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:13.658 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:13.669 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:53 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:53.284+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:13.680 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:13.689 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:13.698 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:13.708 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:13.717 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:13.775 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:13.814 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:54.009+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:13.822 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:13.831 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:13.838 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:13.847 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:13.856 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:13.865 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:13.875 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:54 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:54.291+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:13.884 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:13.895 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:13.904 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:13.913 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:13.922 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:13.933 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:13.942 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.039+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:13.952 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:14.016 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:14.122 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:14.132 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:14.141 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:14.149 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:14.158 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:55.305+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:14.167 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:14.176 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:14.185 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:14.194 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:14.203 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:14.213 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:14.222 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:55 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:55.992+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:14.231 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:14.241 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:14.250 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:14.260 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:14.317 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:14.363 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:14.375 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:56 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:56.272+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:14.386 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:14.395 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:14.404 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:14.414 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:14.423 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:14.433 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:14.442 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:57.016+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:14.452 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:14.461 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:14.470 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:14.478 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:14.491 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:14.500 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:14.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:57 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:57.267+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:14.583 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:14.638 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:14.647 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:14.657 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:14.666 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:14.675 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:14.685 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:58.226+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:14.694 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:14.704 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:14.713 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:14.724 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:14.733 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:14.742 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:14.751 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:58 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:58.048+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:14.763 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:14.772 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:14.781 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:14.850 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:14.908 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:14.917 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:14.926 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:10:59.232+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:14.935 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:14.944 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:14.953 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:14.961 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:14.970 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:14.979 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:14.988 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:12.915+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:14.996 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:15.005 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:15.014 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:15.024 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:15.033 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:15.042 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:15.149 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:13 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:13.886+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:15.197 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:15.206 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:15.215 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:15.223 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:15.231 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:15.239 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:15.251 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:15.259 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:12 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:12.771+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:15.266 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:15.274 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:15.281 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:15.289 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:15.296 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:15.304 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:15.313 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:15.321 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:13 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:13.790+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:15.392 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.052+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:15.455 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:15.464 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:15.473 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:15.482 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:15.491 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:15.500 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:10:59 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:10:59.053+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:15.509 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:15.518 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:15.527 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:15.536 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:15.545 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:15.554 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:15.563 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:00.051+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:15.573 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:15.582 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:15.590 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:15.683 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:15.750 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:15.761 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:15.770 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:00 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:00.262+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:15.779 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:15.788 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:15.797 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:15.806 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:15.815 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:15.824 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:15.833 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.020+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:15.842 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:15.851 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:15.860 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:15.869 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:15.891 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:15.900 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:15.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:01.297+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:16.030 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:16.040 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:16.049 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:16.058 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:16.066 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:16.075 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:16.084 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:01 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:01.979+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:16.093 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:16.102 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:16.111 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:16.120 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:16.129 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:16.141 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:16.149 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:02.298+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:16.158 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:16.167 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:16.235 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:16.299 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:16.308 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:16.317 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:16.327 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:02 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:02.940+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:16.336 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:16.345 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:16.355 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:16.364 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:16.373 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:16.382 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:16.391 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:03.285+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:16.401 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:16.410 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:16.419 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:16.428 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:16.437 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:16.507 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:16.563 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:03 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:03.928+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:16.573 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:16.582 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:16.600 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:16.609 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:16.618 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:16.627 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:16.636 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:04.253+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:16.645 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:16.655 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:16.664 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:16.672 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:16.681 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:16.690 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:16.699 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:04 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:04.917+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:16.708 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:16.774 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:16.809 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:16.837 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:16.847 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:16.856 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:16.863 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:14 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:14.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:16.872 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:16.881 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:16.888 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:16.896 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:16.906 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:16.915 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:16.923 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:15 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:15.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:16.931 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:16.939 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:16.951 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:16.959 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:17.032 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:17.082 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:17.090 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:17.097 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:14 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:14.834+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:17.105 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:17.112 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:17.120 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:17.129 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:17.139 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:17.148 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:17.159 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:17.168 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:15 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:15.849+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:17.178 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:17.187 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:17.197 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:17.206 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:17.215 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:17.299 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:17.332 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:05.296+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:17.342 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:17.351 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:17.361 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:17.370 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:17.380 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:17.389 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:17.398 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:05 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:05.938+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:17.408 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:17.417 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:17.430 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:17.439 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:17.448 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:17.457 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:17.466 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:06.325+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:17.476 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:17.523 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:17.567 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:17.575 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:17.584 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:17.593 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:17.602 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:06 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:06.958+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:17.611 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:17.620 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:17.629 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:17.639 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:17.647 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:17.656 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:17.665 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:07.370+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:17.674 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:17.683 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:17.692 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:17.701 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:17.754 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:17.795 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:17.804 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:07 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:07.956+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:17.813 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:17.829 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:17.838 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:17.848 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:17.857 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:17.866 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:17.875 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:08.347+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:17.884 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:17.893 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:17.902 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:17.910 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:17.919 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:17.928 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:17.937 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:08 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:08.934+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:17.999 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:18.053 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:18.062 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:18.071 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:18.080 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:18.089 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:18.098 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:09.311+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:18.107 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:18.116 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:18.125 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:18.134 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:18.142 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:18.152 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:18.160 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:09 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:09.887+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:18.170 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:18.179 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:18.188 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:18.248 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:18.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:18.285 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:18.294 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:10.344+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:18.303 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:18.313 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:18.322 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:18.331 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:18.340 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:18.348 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:18.356 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:16.889+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:18.370 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:18.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:18.387 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:18.396 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:18.404 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:18.413 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:18.464 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:18.498 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:16 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:16.808+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:18.507 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:18.515 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:18.523 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:18.531 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:18.542 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:18.550 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:18.559 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:10 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:10.903+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:18.569 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:18.578 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:18.586 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:18.594 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:18.602 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:18.609 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:18.616 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:11.387+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:18.624 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:18.680 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:18.744 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:18.762 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:18.792 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:18.800 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:18.807 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:11 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:11.889+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:18.818 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:18.842 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:18.854 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:18.875 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:18.884 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:18.893 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:18.902 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:12.413+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:18.914 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:18.922 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:18.939 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:18.948 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:19.010 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:19.099 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:19.123 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:12 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:12.841+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:19.141 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:19.150 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:19.159 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:19.167 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:19.177 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:19.185 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:19.194 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:13.440+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:19.203 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:19.212 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:19.220 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:19.229 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:19.237 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:19.246 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:19.254 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:13 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:13.843+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:19.311 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:19.375 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:19.384 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:19.393 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:19.404 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:19.414 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:19.423 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:14.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:19.432 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:19.443 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:19.452 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:19.461 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:19.469 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:19.479 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:19.489 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:14 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:14.880+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:19.498 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:19.507 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:19.516 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:19.580 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:19.660 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:19.669 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:19.678 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:15.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:19.689 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.829+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:19.698 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.829+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:19.707 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.829+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:19.716 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.830+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:19.725 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.830+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:19.735 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.830+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:19.748 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:15 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:15.830+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:19.758 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:19.768 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:19.779 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:19.788 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:19.798 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:19.810 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:19.907 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:16.454+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:19.974 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:19.983 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:19.992 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:20.001 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:20.012 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:20.022 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:20.031 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:17 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:17.895+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:20.040 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:20.050 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:20.058 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:20.069 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:20.081 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:20.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:20.097 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:18 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:18.921+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:20.106 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:20.114 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:20.191 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:20.277 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:20.286 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:20.295 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:20.304 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:20.312 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:17 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:17.783+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:20.322 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:20.333 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:20.342 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:20.350 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:20.358 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:20.367 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:20.378 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:20.387 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:18 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:18.743+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:20.395 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:20.403 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:20.418 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:20.489 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:20.533 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:20.542 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:20.550 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:16 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:16.855+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:20.558 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:20.566 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:20.573 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:20.580 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:20.588 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:20.595 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:17.433+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:20.614 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:20.623 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:20.632 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:20.641 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:20.650 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:20.659 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:20.705 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:17 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:17.871+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:20.778 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 systemd[1]: Stopping Ceph osd.8 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:20.788 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:20.798 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:20.807 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:20.818 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:20.829 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:20.841 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:20.851 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:18.405+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:20.863 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:20.874 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:20.886 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:20.897 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:20.906 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:20.919 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:20.928 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.828+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:20.939 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.891+0000 7f978f9c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:20.985 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.891+0000 7f978f9c3700 -1 osd.8 172 *** Got signal Terminated *** 2024-04-23T19:11:21.030 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:18 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:18.891+0000 7f978f9c3700 -1 osd.8 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:11:21.039 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:21.051 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:21.059 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:21.067 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:21.076 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:21.085 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:21.093 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:19.455+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:21.103 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:21.111 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:21.122 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:21.131 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:21.140 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:21.148 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:21.157 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:19 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:19.865+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:21.167 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:21.219 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:21.299 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:21.308 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:21.321 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:21.331 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:21.341 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:20.465+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:21.350 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:21.358 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:21.367 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:21.375 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:21.384 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:21.393 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:21.401 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:19 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:19.963+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:21.410 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:21.420 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:21.430 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:21.439 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:21.480 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:21.525 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:21.534 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:21.543 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:19 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:19.752+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:21.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:21.561 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:21.572 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:21.583 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:21.592 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:21.600 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:21.609 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:21.617 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:20 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:20.799+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:21.627 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:21.637 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:21.645 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:21.654 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:21.664 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:21.722 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:21.789 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:20.852+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:21.799 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:21.808 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:21.818 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:21.827 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:21.836 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:21.845 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:21.854 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:20 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:20.918+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:21.865 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:21.874 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:21.882 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:21.893 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:21.902 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:21.910 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:21.918 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:21.416+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:22.207 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:22.252 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:22.314 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:22.323 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:22.331 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:22.340 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:22.349 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:21 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:21.824+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:22.359 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:22.367 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:22.375 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:22.385 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:22.394 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:22.402 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:22.410 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:22.421 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:21 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:21.818+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:22.430 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:22.439 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:22.448 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:22.556 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:22.592 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:22.605 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:22.614 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:21 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:21.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:22.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:22.713 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:22.722 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:22.731 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:22.750 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:22.760 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:22.769 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:22.382+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:23.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:23.138 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:23.148 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:23.157 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:23.168 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:23.180 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:23.292 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:22 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:22.854+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:23.352 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:23.361 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:23.372 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:23.381 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:23.390 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:23.403 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:23.411 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:23.420 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:22 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:22.815+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:23.431 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:23.439 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:23.448 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:23.458 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:23.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:23.475 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:23.487 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:22 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:22.849+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:23.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:23.753 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:23.812 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:23.821 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:23.830 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:23.839 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:23.846 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:23.432+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:24.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:24.135 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:24.144 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:24.153 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:24.162 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:24.168 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:24.178 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:23 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:23.842+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:24.207 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:55.523974+0000 front 2024-04-23T19:08:55.523842+0000 (oldest deadline 2024-04-23T19:09:20.823948+0000) 2024-04-23T19:11:24.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:02.425297+0000 front 2024-04-23T19:09:02.425238+0000 (oldest deadline 2024-04-23T19:09:26.525263+0000) 2024-04-23T19:11:24.255 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:08.826240+0000 front 2024-04-23T19:09:08.826309+0000 (oldest deadline 2024-04-23T19:09:34.126212+0000) 2024-04-23T19:11:24.263 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:19.328306+0000 front 2024-04-23T19:09:19.328150+0000 (oldest deadline 2024-04-23T19:09:42.828054+0000) 2024-04-23T19:11:24.338 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.829418+0000 front 2024-04-23T19:09:29.829435+0000 (oldest deadline 2024-04-23T19:09:53.929428+0000) 2024-04-23T19:11:24.381 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.531177+0000 front 2024-04-23T19:09:38.531189+0000 (oldest deadline 2024-04-23T19:10:04.430994+0000) 2024-04-23T19:11:24.389 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:23 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8[51163]: 2024-04-23T19:11:23.810+0000 7f978bfb7700 -1 osd.8 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.037309+0000 front 2024-04-23T19:10:27.037456+0000 (oldest deadline 2024-04-23T19:10:49.337181+0000) 2024-04-23T19:11:24.398 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:24.407 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:24.415 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:24.423 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:24.432 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:24.441 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:24.450 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:24.458 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:23 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:23.814+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:24.610 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:24 smithi122 podman[79134]: 2024-04-23 19:11:24.299572731 +0000 UTC m=+5.535825537 container died 4dd55a98d46aff7d77e5a8e0a55ea2a99488a451a4af4424302f8960e74c346d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-23T19:11:24.619 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:24.628 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:24.637 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:24.646 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:24.654 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:24.702 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:24.773 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:24.446+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:24.953 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:24 smithi122 podman[79134]: 2024-04-23 19:11:24.717822636 +0000 UTC m=+5.954075453 container cleanup 4dd55a98d46aff7d77e5a8e0a55ea2a99488a451a4af4424302f8960e74c346d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-23T19:11:24.962 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:24 smithi122 bash[79134]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8 2024-04-23T19:11:25.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:25.137 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:25.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:25.155 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:25.164 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:25.173 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:25.181 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:24 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:24.822+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:25.206 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:24 smithi122 podman[79151]: 2024-04-23 19:11:24.95228599 +0000 UTC m=+0.653784980 container remove 4dd55a98d46aff7d77e5a8e0a55ea2a99488a451a4af4424302f8960e74c346d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-23T19:11:25.238 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:25.248 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:25.257 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:25.265 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:25.305 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:25.342 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:25.371 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:25.405 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:24 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:24.804+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:25.647 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:25.656 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:25.665 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:25.673 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:25.682 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:25.690 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:25.698 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:25 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:25.417+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:25.707 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.288868062 +0000 UTC m=+0.017906911 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:11:25.716 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.394875287 +0000 UTC m=+0.123914121 container create bbbf5200f86fa1441441a04bf0cc434475bd1ecfe15f10eab6a026ab6bc4d81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-23T19:11:25.724 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.572047286 +0000 UTC m=+0.301086133 container init bbbf5200f86fa1441441a04bf0cc434475bd1ecfe15f10eab6a026ab6bc4d81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:11:25.733 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.575078396 +0000 UTC m=+0.304117231 container start bbbf5200f86fa1441441a04bf0cc434475bd1ecfe15f10eab6a026ab6bc4d81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:11:25.741 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.647028799 +0000 UTC m=+0.376067640 container attach bbbf5200f86fa1441441a04bf0cc434475bd1ecfe15f10eab6a026ab6bc4d81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-23T19:11:25.956 INFO:journalctl@ceph.osd.8.smithi122.stdout:Apr 23 19:11:25 smithi122 podman[79240]: 2024-04-23 19:11:25.871889034 +0000 UTC m=+0.600927878 container died bbbf5200f86fa1441441a04bf0cc434475bd1ecfe15f10eab6a026ab6bc4d81f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T19:11:26.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:26.152 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:26.210 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:26.219 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:26.228 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:26.236 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:26.245 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:25 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:25.792+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:26.254 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:26.263 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:26.271 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:26.279 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:26.287 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:26.296 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:26.305 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:26.314 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:25 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:25.756+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:26.462 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.8.service' 2024-04-23T19:11:26.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:26.528 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-23T19:11:26.579 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-23T19:11:26.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.9 2024-04-23T19:11:26.707 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:26.714 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:26.721 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:26.728 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:26.735 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:26.744 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:26.751 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:26 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:26.463+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:26.929 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 systemd[1]: Stopping Ceph osd.9 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:26.936 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:26.943 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:26.950 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:26.957 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:26.964 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:26.970 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:27.010 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:27.054 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.739+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:27.101 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.927+0000 7f4808053700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:27.109 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.927+0000 7f4808053700 -1 osd.9 172 *** Got signal Terminated *** 2024-04-23T19:11:27.116 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:26 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:26.927+0000 7f4808053700 -1 osd.9 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:11:27.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:27.136 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:27.143 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:27.150 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:27.158 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:27.165 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:27.172 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:26 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:26.803+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:27.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:27.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:27.725 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:27.733 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:27.743 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:27.752 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:27.837 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:27 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:27.418+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:27.988 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:27.997 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:28.005 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:28.014 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:28.025 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:28.034 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:28.043 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:28.052 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:27 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:27.719+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:28.128 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:28.137 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:28.146 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:28.155 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:28.163 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:28.172 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:28.181 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:27 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:27.801+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:28.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:28.804 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:28.812 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:28.821 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:28.830 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:28.839 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:28.451+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:29.129 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:29.135 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:29.144 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:29.153 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:29.162 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:29.172 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:29.181 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:28 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:28.843+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:29.238 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:29.248 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:29.257 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:29.317 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:29.395 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:29.466 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:29.475 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:29.484 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:28 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:28.744+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:29.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:29.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:29.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:29.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:29.991 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:30.003 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:30.012 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:29 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:29.456+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:30.239 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:30.246 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:30.255 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:30.264 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:30.273 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:30.373 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:30.448 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:30.585 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:29 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:29.788+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:30.594 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:30.603 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:30.611 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:30.620 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:30.628 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:30.662 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:30.671 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:29 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:29.882+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:30.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:30.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:30.972 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:30.980 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:30.987 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:30.994 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:31.001 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:30 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:30.478+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:31.238 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:31.278 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:31.363 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:31.370 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:31.377 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:31.384 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:31.392 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:31.399 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:30 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:30.781+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:31.411 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:31.418 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:31.427 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:31.435 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:31.443 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:31.452 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:31.460 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:30 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:30.896+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:31.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:31.963 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:31.971 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:32.023 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:32.088 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:32.097 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:32.106 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:31 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:31.511+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:32.140 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.781063+0000 front 2024-04-23T19:08:48.781132+0000 (oldest deadline 2024-04-23T19:09:11.081274+0000) 2024-04-23T19:11:32.149 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:56.283395+0000 front 2024-04-23T19:08:56.283138+0000 (oldest deadline 2024-04-23T19:09:19.783281+0000) 2024-04-23T19:11:32.158 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.385138+0000 front 2024-04-23T19:09:04.385214+0000 (oldest deadline 2024-04-23T19:09:29.085091+0000) 2024-04-23T19:11:32.167 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.686844+0000 front 2024-04-23T19:09:10.686775+0000 (oldest deadline 2024-04-23T19:09:35.986947+0000) 2024-04-23T19:11:32.176 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.088020+0000 front 2024-04-23T19:09:20.088055+0000 (oldest deadline 2024-04-23T19:09:45.988045+0000) 2024-04-23T19:11:32.185 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:30.490591+0000 front 2024-04-23T19:09:30.490614+0000 (oldest deadline 2024-04-23T19:09:54.590665+0000) 2024-04-23T19:11:32.194 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:40.992150+0000 front 2024-04-23T19:09:40.992174+0000 (oldest deadline 2024-04-23T19:10:06.892311+0000) 2024-04-23T19:11:32.203 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:31 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9[64048]: 2024-04-23T19:11:31.763+0000 7f480063f700 -1 osd.9 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:27.099441+0000 front 2024-04-23T19:10:27.099404+0000 (oldest deadline 2024-04-23T19:10:48.700037+0000) 2024-04-23T19:11:32.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:32.413 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:33.712 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:33.751 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:33.825 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:33.910 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:33.971 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:31 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:31.908+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:33.981 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:33.990 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:33.998 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:34.007 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:34.017 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:34.025 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:34.034 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:32 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:32.907+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:34.043 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:34.052 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:34.061 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:34.070 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:34.078 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:34.087 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:34.097 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:32 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:32.464+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:34.105 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:34.175 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:34.212 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:34.221 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:34.230 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:34.238 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:34.247 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:33 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:33.468+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:34.256 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:32 smithi033 podman[107378]: 2024-04-23 19:11:32.335302954 +0000 UTC m=+5.508332701 container died a33438cddff875d15e99066cafdb23bd92819b0235506694f8c78ce6e86f799c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-23T19:11:34.265 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:32 smithi033 podman[107378]: 2024-04-23 19:11:32.819199453 +0000 UTC m=+5.992229190 container cleanup a33438cddff875d15e99066cafdb23bd92819b0235506694f8c78ce6e86f799c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-23T19:11:34.274 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:32 smithi033 bash[107378]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9 2024-04-23T19:11:34.283 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107396]: 2024-04-23 19:11:33.095689286 +0000 UTC m=+0.753922930 container remove a33438cddff875d15e99066cafdb23bd92819b0235506694f8c78ce6e86f799c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-23T19:11:34.290 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.463057794 +0000 UTC m=+0.114845605 container create 1499817b0b01d11cf04849d6eda0c7c3d88371f860f17b32e108339de5ee0e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-23T19:11:34.299 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.366865914 +0000 UTC m=+0.018653735 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:11:34.308 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.682001425 +0000 UTC m=+0.333789237 container init 1499817b0b01d11cf04849d6eda0c7c3d88371f860f17b32e108339de5ee0e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T19:11:34.316 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.685368978 +0000 UTC m=+0.337156798 container start 1499817b0b01d11cf04849d6eda0c7c3d88371f860f17b32e108339de5ee0e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-23T19:11:34.325 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.756995222 +0000 UTC m=+0.408783040 container attach 1499817b0b01d11cf04849d6eda0c7c3d88371f860f17b32e108339de5ee0e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-04-23T19:11:34.349 INFO:journalctl@ceph.osd.9.smithi033.stdout:Apr 23 19:11:33 smithi033 podman[107484]: 2024-04-23 19:11:33.98988498 +0000 UTC m=+0.641672794 container died 1499817b0b01d11cf04849d6eda0c7c3d88371f860f17b32e108339de5ee0e38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-23T19:11:34.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:34.442 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:34.479 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:34.490 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:34.498 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:34.507 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:34.517 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:33 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:33.897+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:34.659 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.9.service' 2024-04-23T19:11:34.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:34.718 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-23T19:11:34.726 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-23T19:11:34.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.10 2024-04-23T19:11:34.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:34.965 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:34.973 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:34.982 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:34.990 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:35.046 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:35.124 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:34 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:34.502+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:35.199 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 systemd[1]: Stopping Ceph osd.10 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:35.207 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:35.215 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:35.223 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:35.230 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:35.238 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:35.247 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:35.255 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:34 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:34.866+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:35.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.049+0000 7f04bbccd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:35.387 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.049+0000 7f04bbccd700 -1 osd.10 172 *** Got signal Terminated *** 2024-04-23T19:11:35.395 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.049+0000 7f04bbccd700 -1 osd.10 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:11:35.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:35.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:35.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:35.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:36.047 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:36.106 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:36.173 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:35 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:35.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:36.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:36.385 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:36.394 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:36.403 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:36.411 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:36.422 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:36.430 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:35 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:35.877+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:36.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:36.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:36.973 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:36.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:36.992 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:37.001 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:37.011 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:36 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:36.475+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:37.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:37.400 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:37.497 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:37.506 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:37.515 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:37.524 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:37.533 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:36 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:36.925+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:37.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:37.713 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:37.722 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:37.731 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:37.741 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:37.750 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:37.758 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:37 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:37.439+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:38.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:38.387 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:38.396 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:38.405 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:38.498 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:38.556 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:38.564 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:37 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:37.890+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:38.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:38.965 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:38.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:38.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:38.992 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:39.002 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:39.010 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:38 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:38.488+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:39.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:39.386 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:39.394 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:39.405 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:39.413 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:39.450 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:39.512 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:38 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:38.926+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:39.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:39.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:39.991 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:39.999 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:40.008 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:40.016 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:40.025 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:39 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:39.520+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:40.280 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:48.222572+0000 front 2024-04-23T19:08:48.222644+0000 (oldest deadline 2024-04-23T19:09:10.522475+0000) 2024-04-23T19:11:40.287 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.423667+0000 front 2024-04-23T19:08:53.423648+0000 (oldest deadline 2024-04-23T19:09:18.123757+0000) 2024-04-23T19:11:40.297 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.525422+0000 front 2024-04-23T19:09:04.525413+0000 (oldest deadline 2024-04-23T19:09:28.625401+0000) 2024-04-23T19:11:40.306 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:11.426822+0000 front 2024-04-23T19:09:11.426916+0000 (oldest deadline 2024-04-23T19:09:35.526757+0000) 2024-04-23T19:11:40.315 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.827610+0000 front 2024-04-23T19:09:20.827633+0000 (oldest deadline 2024-04-23T19:09:44.327809+0000) 2024-04-23T19:11:40.324 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:29.529316+0000 front 2024-04-23T19:09:29.529357+0000 (oldest deadline 2024-04-23T19:09:51.229427+0000) 2024-04-23T19:11:40.332 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:39 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10[57574]: 2024-04-23T19:11:39.976+0000 7f04b42b9700 -1 osd.10 172 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-04-23T19:09:38.131060+0000 front 2024-04-23T19:09:38.130913+0000 (oldest deadline 2024-04-23T19:10:04.030836+0000) 2024-04-23T19:11:40.628 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:40 smithi106 podman[76911]: 2024-04-23 19:11:40.463412822 +0000 UTC m=+5.531151135 container died 563b95a28b39d76881fd9d8849af0a7d14c7ffb1e16470dcd3a03d3c31479e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:11:40.905 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:40 smithi106 podman[76911]: 2024-04-23 19:11:40.857432818 +0000 UTC m=+5.925171130 container cleanup 563b95a28b39d76881fd9d8849af0a7d14c7ffb1e16470dcd3a03d3c31479e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-23T19:11:40.913 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:40 smithi106 bash[76911]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10 2024-04-23T19:11:41.029 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:41.094 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:41.102 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:41.110 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:41.118 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:41.127 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:41.134 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:40 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:40.525+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:41.378 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[76929]: 2024-04-23 19:11:41.083605628 +0000 UTC m=+0.615568851 container remove 563b95a28b39d76881fd9d8849af0a7d14c7ffb1e16470dcd3a03d3c31479e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:11:41.728 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.32191815 +0000 UTC m=+0.023320584 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:11:41.735 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.434425429 +0000 UTC m=+0.135827844 container create cab4c159d05e0a2b4a086484e9e824f543d7258d179967b92b00f14bdbc7eed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:11:41.742 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.645465431 +0000 UTC m=+0.346867878 container init cab4c159d05e0a2b4a086484e9e824f543d7258d179967b92b00f14bdbc7eed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-23T19:11:41.749 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.649059385 +0000 UTC m=+0.350461803 container start cab4c159d05e0a2b4a086484e9e824f543d7258d179967b92b00f14bdbc7eed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-23T19:11:41.757 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.728138085 +0000 UTC m=+0.429540519 container attach cab4c159d05e0a2b4a086484e9e824f543d7258d179967b92b00f14bdbc7eed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-23T19:11:41.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:41.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:41.988 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:42.071 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:42.125 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:42.198 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:42.206 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:41 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:41.510+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:42.217 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 23 19:11:41 smithi106 podman[77026]: 2024-04-23 19:11:41.950247316 +0000 UTC m=+0.651649740 container died cab4c159d05e0a2b4a086484e9e824f543d7258d179967b92b00f14bdbc7eed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-23T19:11:42.513 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.10.service' 2024-04-23T19:11:42.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:42.566 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-23T19:11:42.574 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-23T19:11:42.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.11 2024-04-23T19:11:42.945 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:42.954 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:42.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:42.984 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:42.993 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:43.002 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:43.011 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.539+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:43.078 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 systemd[1]: Stopping Ceph osd.11 for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:43.184 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.944+0000 7f532af0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:43.249 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.944+0000 7f532af0e700 -1 osd.11 172 *** Got signal Terminated *** 2024-04-23T19:11:43.258 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:42 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:42.944+0000 7f532af0e700 -1 osd.11 172 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-23T19:11:43.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:43.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:43.973 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:43.982 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:43.990 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:43.999 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:44.008 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:43 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:43.507+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:44.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:44.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:44.978 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:44.987 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:44.995 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:45.006 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:45.014 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:44 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:44.503+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:45.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:45.984 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:45.993 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:46.002 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:46.011 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:46.020 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:46.028 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:46.039 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:45 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:45.548+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-23T19:11:20.089712+0000 front 2024-04-23T19:11:20.089751+0000 (oldest deadline 2024-04-23T19:11:44.789862+0000) 2024-04-23T19:11:46.957 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:46.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:46.974 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:46.983 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:46.992 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:47.000 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:47.010 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:47.019 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:46 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:46.563+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-23T19:11:20.089712+0000 front 2024-04-23T19:11:20.089751+0000 (oldest deadline 2024-04-23T19:11:44.789862+0000) 2024-04-23T19:11:47.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-23T19:08:44.268171+0000 front 2024-04-23T19:08:44.268058+0000 (oldest deadline 2024-04-23T19:09:13.068522+0000) 2024-04-23T19:11:48.018 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-23T19:08:53.069022+0000 front 2024-04-23T19:08:53.069085+0000 (oldest deadline 2024-04-23T19:09:17.769061+0000) 2024-04-23T19:11:48.128 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-04-23T19:09:04.170493+0000 front 2024-04-23T19:09:04.170666+0000 (oldest deadline 2024-04-23T19:09:30.070573+0000) 2024-04-23T19:11:48.138 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-04-23T19:09:10.071211+0000 front 2024-04-23T19:09:10.071232+0000 (oldest deadline 2024-04-23T19:09:35.970962+0000) 2024-04-23T19:11:48.147 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-23T19:09:20.572891+0000 front 2024-04-23T19:09:20.572950+0000 (oldest deadline 2024-04-23T19:09:45.173417+0000) 2024-04-23T19:11:48.155 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-04-23T19:09:25.173809+0000 front 2024-04-23T19:09:25.173868+0000 (oldest deadline 2024-04-23T19:09:53.974413+0000) 2024-04-23T19:11:48.164 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-23T19:10:25.182538+0000 front 2024-04-23T19:10:25.182561+0000 (oldest deadline 2024-04-23T19:10:49.282546+0000) 2024-04-23T19:11:48.172 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:47 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11[58202]: 2024-04-23T19:11:47.583+0000 7f53234fa700 -1 osd.11 172 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-04-23T19:11:20.089712+0000 front 2024-04-23T19:11:20.089751+0000 (oldest deadline 2024-04-23T19:11:44.789862+0000) 2024-04-23T19:11:48.706 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:48 smithi122 podman[79359]: 2024-04-23 19:11:48.455230194 +0000 UTC m=+5.667371498 container died 5d9fca4a1c4c14938b31e9482070e4ddc50cc2e449656e7c2becd26bdbb95908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:11:49.207 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:48 smithi122 podman[79359]: 2024-04-23 19:11:48.803357434 +0000 UTC m=+6.015498770 container cleanup 5d9fca4a1c4c14938b31e9482070e4ddc50cc2e449656e7c2becd26bdbb95908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-23T19:11:49.214 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:48 smithi122 bash[79359]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11 2024-04-23T19:11:49.222 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:48 smithi122 podman[79376]: 2024-04-23 19:11:48.996123896 +0000 UTC m=+0.533942401 container remove 5d9fca4a1c4c14938b31e9482070e4ddc50cc2e449656e7c2becd26bdbb95908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-23T19:11:49.670 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.389003845 +0000 UTC m=+0.107418815 container create 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-04-23T19:11:49.677 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.299350553 +0000 UTC m=+0.017765486 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-23T19:11:49.684 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.591164981 +0000 UTC m=+0.309579906 container init 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-23T19:11:49.692 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.594244407 +0000 UTC m=+0.312659335 container start 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-04-23T19:11:49.698 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.669963388 +0000 UTC m=+0.388378323 container attach 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-23T19:11:49.956 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:49 smithi122 podman[79470]: 2024-04-23 19:11:49.881844714 +0000 UTC m=+0.600259632 container died 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-23T19:11:50.421 INFO:journalctl@ceph.osd.11.smithi122.stdout:Apr 23 19:11:50 smithi122 podman[79497]: 2024-04-23 19:11:50.419236243 +0000 UTC m=+0.532214841 container remove 4507611b9be947cda90b21d35a737d11bf3fb2c219de82648b9dcfa5f24e2c57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-23T19:11:50.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@osd.11.service' 2024-04-23T19:11:50.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:50.492 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-23T19:11:50.499 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-23T19:11:50.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.a 2024-04-23T19:11:50.846 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:50 smithi033 systemd[1]: Stopping Ceph mds.a for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:51.238 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: 2024-04-23T19:11:50.845+0000 7f87872a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:51.245 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:50 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a[70891]: 2024-04-23T19:11:50.845+0000 7f87872a0700 -1 mds.a *** got signal Terminated *** 2024-04-23T19:11:55.129 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:54 smithi033 podman[107601]: 2024-04-23 19:11:54.851183982 +0000 UTC m=+4.130583798 container died 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-23T19:11:55.462 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:55 smithi033 podman[107601]: 2024-04-23 19:11:55.219570836 +0000 UTC m=+4.498970653 container cleanup 8db0770a5d17e8a2344c1883099722f8024d05c894dbf794ee233e51e1bc7852 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-23T19:11:55.471 INFO:journalctl@ceph.mds.a.smithi033.stdout:Apr 23 19:11:55 smithi033 bash[107601]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-a 2024-04-23T19:11:55.614 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.a.service' 2024-04-23T19:11:55.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:55.667 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-23T19:11:55.676 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-23T19:11:55.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d 2024-04-23T19:11:55.725 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.d.service' 2024-04-23T19:11:55.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:11:55.806 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-23T19:11:55.816 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-23T19:11:55.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.g 2024-04-23T19:11:55.988 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:55 smithi033 systemd[1]: Stopping Ceph mds.g for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:11:56.488 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: 2024-04-23T19:11:56.122+0000 7fd4c7292700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:11:56.534 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:56 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g[70527]: 2024-04-23T19:11:56.122+0000 7fd4c7292700 -1 mds.g *** got signal Terminated *** 2024-04-23T19:11:59.399 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:59 smithi033 podman[107760]: 2024-04-23 19:11:59.121433395 +0000 UTC m=+3.095426446 container died 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-23T19:11:59.649 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:59 smithi033 podman[107760]: 2024-04-23 19:11:59.473028011 +0000 UTC m=+3.447021052 container cleanup 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-23T19:11:59.655 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:59 smithi033 bash[107760]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g 2024-04-23T19:11:59.905 INFO:journalctl@ceph.mds.g.smithi033.stdout:Apr 23 19:11:59 smithi033 podman[107774]: 2024-04-23 19:11:59.649050936 +0000 UTC m=+0.521247824 container remove 0a775758747474d45752ce5788e60d5d562087d27c219f6edac7d616fc33be38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:11:59.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.g.service' 2024-04-23T19:12:00.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:00.047 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-23T19:12:00.054 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-23T19:12:00.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j 2024-04-23T19:12:00.238 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:00 smithi033 systemd[1]: Stopping Ceph mds.j for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:00.738 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: 2024-04-23T19:12:00.384+0000 7f6e6ab57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:00.746 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:00 smithi033 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j[102975]: 2024-04-23T19:12:00.384+0000 7f6e6ab57700 -1 mds.j *** got signal Terminated *** 2024-04-23T19:12:03.441 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:03 smithi033 podman[107881]: 2024-04-23 19:12:03.135273466 +0000 UTC m=+2.878373800 container died 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True) 2024-04-23T19:12:03.736 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:03 smithi033 podman[107881]: 2024-04-23 19:12:03.551533264 +0000 UTC m=+3.294633603 container cleanup 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-23T19:12:03.743 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:03 smithi033 bash[107881]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j 2024-04-23T19:12:03.988 INFO:journalctl@ceph.mds.j.smithi033.stdout:Apr 23 19:12:03 smithi033 podman[107895]: 2024-04-23 19:12:03.735676961 +0000 UTC m=+0.599711577 container remove 99f60a6e18f7036e4ef064bb6ea9ba7580a98626c2fb0ec711d7594b90cf5efd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-23T19:12:03.994 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.j.service' 2024-04-23T19:12:04.035 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:04.042 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-23T19:12:04.050 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-23T19:12:04.059 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.b 2024-04-23T19:12:04.366 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:04 smithi106 systemd[1]: Stopping Ceph mds.b for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:04.628 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-b[62971]: 2024-04-23T19:12:04.365+0000 7f92b2a5c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:04.635 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:04 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-b[62971]: 2024-04-23T19:12:04.365+0000 7f92b2a5c700 -1 mds.b *** got signal Terminated *** 2024-04-23T19:12:07.408 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:07 smithi106 podman[77135]: 2024-04-23 19:12:07.13188549 +0000 UTC m=+2.871365135 container died 7bea2d1eab25c7c77e04fed701c00a09706e8a24bb29808978834601a42c31b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:12:07.774 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:07 smithi106 podman[77135]: 2024-04-23 19:12:07.490154463 +0000 UTC m=+3.229634108 container cleanup 7bea2d1eab25c7c77e04fed701c00a09706e8a24bb29808978834601a42c31b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:12:07.784 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 23 19:12:07 smithi106 bash[77135]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-b 2024-04-23T19:12:07.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.b.service' 2024-04-23T19:12:07.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:08.012 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-23T19:12:08.023 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-23T19:12:08.033 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.e 2024-04-23T19:12:08.378 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:08 smithi106 systemd[1]: Stopping Ceph mds.e for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:08.467 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: 2024-04-23T19:12:08.309+0000 7f21b91cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:08.476 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:08 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e[62356]: 2024-04-23T19:12:08.309+0000 7f21b91cb700 -1 mds.e *** got signal Terminated *** 2024-04-23T19:12:11.403 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:11 smithi106 podman[77255]: 2024-04-23 19:12:11.136282188 +0000 UTC m=+2.921398157 container died be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-23T19:12:11.762 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:11 smithi106 podman[77255]: 2024-04-23 19:12:11.494185509 +0000 UTC m=+3.279301531 container cleanup be4874427eaa48c9c92c92dfcb8cbb803f0a55b7833f520c5a57745dc76ad7ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-23T19:12:11.770 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 23 19:12:11 smithi106 bash[77255]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-e 2024-04-23T19:12:11.917 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.e.service' 2024-04-23T19:12:11.956 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:12.045 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-23T19:12:12.052 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-23T19:12:12.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.h 2024-04-23T19:12:12.378 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:12 smithi106 systemd[1]: Stopping Ceph mds.h for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:12.386 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: 2024-04-23T19:12:12.330+0000 7ff41134b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:12.393 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:12 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h[62666]: 2024-04-23T19:12:12.330+0000 7ff41134b700 -1 mds.h *** got signal Terminated *** 2024-04-23T19:12:15.399 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:15 smithi106 podman[77374]: 2024-04-23 19:12:15.129871345 +0000 UTC m=+2.888052048 container died a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-23T19:12:15.757 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:15 smithi106 podman[77374]: 2024-04-23 19:12:15.497384634 +0000 UTC m=+3.255565337 container cleanup a20af9def49ad64d26b15f12c9402d4c23819c1510fb97183edd8671c14ec182 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-23T19:12:15.765 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 23 19:12:15 smithi106 bash[77374]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-h 2024-04-23T19:12:15.918 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.h.service' 2024-04-23T19:12:15.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:15.967 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-23T19:12:15.976 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-23T19:12:15.984 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.k 2024-04-23T19:12:16.277 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:16 smithi106 systemd[1]: Stopping Ceph mds.k for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:16.628 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: 2024-04-23T19:12:16.275+0000 7f5d86d97700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:16.636 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:16 smithi106 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k[62036]: 2024-04-23T19:12:16.275+0000 7f5d86d97700 -1 mds.k *** got signal Terminated *** 2024-04-23T19:12:19.410 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:19 smithi106 podman[77508]: 2024-04-23 19:12:19.132184924 +0000 UTC m=+2.950931016 container died 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-23T19:12:19.870 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:19 smithi106 podman[77508]: 2024-04-23 19:12:19.467594254 +0000 UTC m=+3.286340351 container cleanup 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-23T19:12:19.880 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:19 smithi106 bash[77508]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k 2024-04-23T19:12:19.889 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 23 19:12:19 smithi106 podman[77522]: 2024-04-23 19:12:19.635119608 +0000 UTC m=+0.505725165 container remove 00784d7ce4811dbb7234978e98822fc884ce039244e39c1e0256ea2caa2c0f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:12:19.900 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.k.service' 2024-04-23T19:12:19.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:19.948 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-23T19:12:19.957 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-23T19:12:19.966 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.c 2024-04-23T19:12:20.270 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:20 smithi122 systemd[1]: Stopping Ceph mds.c for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:20.528 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c[63262]: 2024-04-23T19:12:20.269+0000 7f775a1a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:20.586 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:20 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c[63262]: 2024-04-23T19:12:20.269+0000 7f775a1a0700 -1 mds.c *** got signal Terminated *** 2024-04-23T19:12:23.439 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 podman[79575]: 2024-04-23 19:12:23.136151473 +0000 UTC m=+2.973287895 container died 3af0d6340c4062d3f803079c9ff6630779514f114c3aa50ae4c2d7d625ce3601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-23T19:12:23.706 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 podman[79575]: 2024-04-23 19:12:23.537743639 +0000 UTC m=+3.374880060 container cleanup 3af0d6340c4062d3f803079c9ff6630779514f114c3aa50ae4c2d7d625ce3601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-23T19:12:23.715 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 bash[79575]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c 2024-04-23T19:12:23.884 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.c.service' 2024-04-23T19:12:24.223 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 podman[79601]: 2024-04-23 19:12:23.755562997 +0000 UTC m=+0.610521771 container remove 3af0d6340c4062d3f803079c9ff6630779514f114c3aa50ae4c2d7d625ce3601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-23T19:12:24.250 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-23T19:12:24.259 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 systemd[1]: Stopped Ceph mds.c for 766ecab6-019d-11ef-bc93-c7b262605968. 2024-04-23T19:12:24.267 INFO:journalctl@ceph.mds.c.smithi122.stdout:Apr 23 19:12:23 smithi122 systemd[1]: ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.c.service: Consumed 4.010s CPU time. 2024-04-23T19:12:24.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:24.321 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-23T19:12:24.330 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-23T19:12:24.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f 2024-04-23T19:12:24.381 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.f.service' 2024-04-23T19:12:24.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:24.461 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-23T19:12:24.470 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-23T19:12:24.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.i 2024-04-23T19:12:24.706 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:24 smithi122 systemd[1]: Stopping Ceph mds.i for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:25.206 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: 2024-04-23T19:12:24.758+0000 7f707ba06700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:25.216 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:24 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i[63568]: 2024-04-23T19:12:24.758+0000 7f707ba06700 -1 mds.i *** got signal Terminated *** 2024-04-23T19:12:27.401 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:27 smithi122 podman[79744]: 2024-04-23 19:12:27.138163168 +0000 UTC m=+2.474695095 container died 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-23T19:12:27.706 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:27 smithi122 podman[79744]: 2024-04-23 19:12:27.491686605 +0000 UTC m=+2.828218528 container cleanup 0dfa79d04a530a864d432b6bf8515b7424149f5323550767cfc448eba7d8c98f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-23T19:12:27.719 INFO:journalctl@ceph.mds.i.smithi122.stdout:Apr 23 19:12:27 smithi122 bash[79744]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-i 2024-04-23T19:12:27.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.i.service' 2024-04-23T19:12:27.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:27.972 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-23T19:12:28.054 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-23T19:12:28.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.l 2024-04-23T19:12:28.354 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:28 smithi122 systemd[1]: Stopping Ceph mds.l for 766ecab6-019d-11ef-bc93-c7b262605968... 2024-04-23T19:12:28.706 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: 2024-04-23T19:12:28.353+0000 7fa52a3f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-23T19:12:28.716 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:28 smithi122 ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l[62647]: 2024-04-23T19:12:28.353+0000 7fa52a3f2700 -1 mds.l *** got signal Terminated *** 2024-04-23T19:12:31.380 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:31 smithi122 podman[79866]: 2024-04-23 19:12:31.124782178 +0000 UTC m=+2.877322888 container died 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-23T19:12:31.707 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:31 smithi122 podman[79866]: 2024-04-23 19:12:31.478924441 +0000 UTC m=+3.231465139 container cleanup 1b6ba1fc8d8f5c6abeff9c383eca7949715f10f633ab5b45be6527992197ae9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-23T19:12:31.718 INFO:journalctl@ceph.mds.l.smithi122.stdout:Apr 23 19:12:31 smithi122 bash[79866]: ceph-766ecab6-019d-11ef-bc93-c7b262605968-mds-l 2024-04-23T19:12:31.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-766ecab6-019d-11ef-bc93-c7b262605968@mds.l.service' 2024-04-23T19:12:31.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-23T19:12:31.942 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-23T19:12:31.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T19:12:32.158 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:34.540 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T19:12:34.745 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:37.278 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-23T19:12:37.530 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:39.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:12:39.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:12:39.878 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-23T19:12:39.914 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-23T19:12:39.926 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi033/crash 2024-04-23T19:12:39.935 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T19:12:39.988 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T19:12:39.998 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-04-23T19:12:40.008 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi106/crash 2024-04-23T19:12:40.018 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T19:12:40.063 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T19:12:40.073 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-04-23T19:12:40.083 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi122/crash 2024-04-23T19:12:40.094 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash -- . 2024-04-23T19:12:40.147 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/766ecab6-019d-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-23T19:12:40.155 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-04-23T19:12:40.224 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-23T19:12:40.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-23T19:12:40.389 INFO:teuthology.orchestra.run.smithi033.stdout:2024-04-23T18:46:56.574873+0000 mon.a (mon.0) 1487 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-23T19:12:40.399 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-23T19:12:40.408 INFO:tasks.cephadm:Compressing logs... 2024-04-23T19:12:40.417 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T19:12:40.432 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T19:12:40.444 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-23T19:12:40.457 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T19:12:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:: No such file or directory 2024-04-23T19:12:40.491 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T19:12:40.500 INFO:teuthology.orchestra.run.smithi106.stderr:: No such file or directory 2024-04-23T19:12:40.509 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T19:12:40.518 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-23T19:12:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-23T19:12:40.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-23T19:12:40.544 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-23T19:12:40.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-23T19:12:40.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-23T19:12:40.671 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-23T19:12:40.681 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.7% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T19:12:40.690 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T19:12:40.700 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T19:12:40.709 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-23T19:12:40.718 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T19:12:40.728 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T19:12:40.737 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.e.log: 93.2% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-23T19:12:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-23T19:12:40.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-23T19:12:40.772 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T19:12:40.781 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T19:12:40.790 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T19:12:40.797 INFO:teuthology.orchestra.run.smithi106.stderr: 93.6% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-23T19:12:40.806 INFO:teuthology.orchestra.run.smithi106.stderr: 93.7% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-23T19:12:40.872 INFO:teuthology.orchestra.run.smithi106.stderr: 94.9% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-23T19:12:40.912 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T19:12:40.944 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-23T19:12:40.952 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-23T19:12:40.960 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-23T19:12:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-23T19:12:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-23T19:12:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-23T19:12:41.031 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-23T19:12:41.039 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.6% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T19:12:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T19:12:41.055 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.6.log: 88.5% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T19:12:41.063 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-23T19:12:41.070 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T19:12:41.078 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T19:12:41.085 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-23T19:12:41.093 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-23T19:12:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T19:12:41.164 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 91.2% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T19:12:41.191 INFO:teuthology.orchestra.run.smithi033.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T19:12:41.201 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-23T19:12:41.209 INFO:teuthology.orchestra.run.smithi033.stderr: 94.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-23T19:12:41.218 INFO:teuthology.orchestra.run.smithi033.stderr: 92.3% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-23T19:12:41.227 INFO:teuthology.orchestra.run.smithi033.stderr: 94.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-23T19:12:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-23T19:12:41.248 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log 2024-04-23T19:12:41.257 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-23T19:12:41.266 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-23T19:12:41.275 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-23T19:12:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-23T19:12:41.293 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-23T19:12:41.302 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-23T19:12:41.311 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-23T19:12:41.320 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-23T19:12:41.351 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-23T19:12:41.379 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-23T19:12:41.406 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-23T19:12:41.416 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-23T19:12:41.425 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-23T19:12:41.434 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-23T19:12:41.444 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.11.log: 93.2% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-23T19:12:41.454 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-23T19:12:41.463 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-23T19:12:41.472 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-23T19:12:41.480 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-23T19:12:41.490 INFO:teuthology.orchestra.run.smithi122.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-23T19:12:41.499 INFO:teuthology.orchestra.run.smithi122.stderr: 94.0% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-23T19:12:41.508 INFO:teuthology.orchestra.run.smithi122.stderr: 91.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-23T19:12:41.517 INFO:teuthology.orchestra.run.smithi122.stderr: 95.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-23T19:12:41.526 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-23T19:12:41.535 INFO:teuthology.orchestra.run.smithi033.stderr: 90.4% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-23T19:12:41.571 INFO:teuthology.orchestra.run.smithi106.stderr: 92.9% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-23T19:12:41.601 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-23T19:12:42.174 INFO:teuthology.orchestra.run.smithi122.stderr: 93.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-23T19:12:42.854 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-23T19:12:42.887 INFO:teuthology.orchestra.run.smithi122.stderr: 93.2% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-23T19:12:42.969 INFO:teuthology.orchestra.run.smithi033.stderr: 92.5% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-23T19:12:42.993 INFO:teuthology.orchestra.run.smithi033.stderr: 92.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-23T19:12:43.092 INFO:teuthology.orchestra.run.smithi106.stderr: 91.8% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-23T19:12:43.114 INFO:teuthology.orchestra.run.smithi122.stderr: 92.4% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-23T19:12:43.193 INFO:teuthology.orchestra.run.smithi033.stderr: 90.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-23T19:12:43.308 INFO:teuthology.orchestra.run.smithi106.stderr: 92.4% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-23T19:12:43.459 INFO:teuthology.orchestra.run.smithi106.stderr: 92.4% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-23T19:12:43.472 INFO:teuthology.orchestra.run.smithi033.stderr: 92.6% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-23T19:12:43.918 INFO:teuthology.orchestra.run.smithi122.stderr: 92.1% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-23T19:12:43.929 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-23T19:12:43.938 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m3.453s 2024-04-23T19:12:43.946 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m13.379s 2024-04-23T19:12:44.012 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.446s 2024-04-23T19:12:44.332 INFO:teuthology.orchestra.run.smithi106.stderr: 92.4% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-23T19:12:44.351 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-23T19:12:44.361 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m3.876s 2024-04-23T19:12:44.369 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m14.212s 2024-04-23T19:12:44.377 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m0.524s 2024-04-23T19:12:44.387 INFO:teuthology.orchestra.run.smithi033.stderr: 92.3% -- replaced with /var/log/ceph/766ecab6-019d-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-23T19:12:44.395 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-23T19:12:44.403 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m3.908s 2024-04-23T19:12:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m15.521s 2024-04-23T19:12:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.490s 2024-04-23T19:12:44.429 INFO:tasks.cephadm:Archiving logs... 2024-04-23T19:12:44.437 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi033/log 2024-04-23T19:12:44.445 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T19:12:46.685 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi106/log 2024-04-23T19:12:46.695 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T19:12:48.995 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi122/log 2024-04-23T19:12:49.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-23T19:12:51.113 INFO:tasks.cephadm:Removing cluster... 2024-04-23T19:12:51.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force 2024-04-23T19:12:51.374 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:51.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force 2024-04-23T19:12:51.850 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:52.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 766ecab6-019d-11ef-bc93-c7b262605968 --force 2024-04-23T19:12:52.324 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 766ecab6-019d-11ef-bc93-c7b262605968 2024-04-23T19:12:52.613 INFO:tasks.cephadm:Removing cephadm ... 2024-04-23T19:12:52.622 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T19:12:52.649 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T19:12:52.674 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-23T19:12:52.697 INFO:tasks.cephadm:Teardown complete 2024-04-23T19:12:52.705 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-23T19:12:52.724 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-23T19:12:52.732 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T19:12:52.742 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T19:12:52.752 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-23T19:12:52.855 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-23T19:12:52.916 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-23T19:12:52.925 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:12:52.935 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:12:52.946 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-23T19:12:52.965 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:12:52.973 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:12:52.984 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:12:52.994 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-23T19:12:53.003 INFO:teuthology.orchestra.run.smithi033.stdout:^- hv01.front.sepia.ceph.com 2 7 377 67 +192us[ +199us] +/- 593ms 2024-04-23T19:12:53.013 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -310us[ -303us] +/- 57ms 2024-04-23T19:12:53.023 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 +446us[ +446us] +/- 86ms 2024-04-23T19:12:53.032 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:12:53.041 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:12:53.052 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-23T19:12:53.061 INFO:teuthology.orchestra.run.smithi106.stdout:^- hv01.front.sepia.ceph.com 2 7 377 0 +148us[ +148us] +/- 594ms 2024-04-23T19:12:53.070 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 7 377 72 -288us[ -311us] +/- 56ms 2024-04-23T19:12:53.080 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +427us[ +427us] +/- 86ms 2024-04-23T19:12:53.118 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:12:53.128 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-23T19:12:53.138 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-23T19:12:53.147 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-23T19:12:53.156 INFO:teuthology.orchestra.run.smithi122.stdout:^- hv01.front.sepia.ceph.com 2 7 377 65 +154us[ +140us] +/- 593ms 2024-04-23T19:12:53.165 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 7 377 63 -281us[ -295us] +/- 56ms 2024-04-23T19:12:53.175 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +448us[ +448us] +/- 85ms 2024-04-23T19:12:53.184 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-23T19:12:53.194 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-23T19:12:53.217 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-23T19:12:53.227 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-23T19:12:53.265 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T19:12:53.360 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T19:12:53.442 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-23T19:12:53.520 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:12:53.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:53.570 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:12:53.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:53.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-23T19:12:53.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:53.693 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-23T19:12:53.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-23T19:12:53.749 INFO:teuthology.task.internal:Duration was 3813.966194 seconds 2024-04-23T19:12:53.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-23T19:12:53.781 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-23T19:12:53.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T19:12:53.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T19:12:53.815 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-23T19:12:53.842 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:12:53.855 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:12:53.865 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-23T19:12:54.279 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-23T19:12:54.289 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-23T19:12:54.298 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T19:12:54.374 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-23T19:12:54.384 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T19:12:54.421 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-23T19:12:54.430 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-23T19:12:54.466 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-23T19:12:54.476 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T19:12:54.488 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T19:12:54.499 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-23T19:12:54.534 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-23T19:12:54.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T19:12:54.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T19:12:54.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-23T19:12:55.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-23T19:12:55.025 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-23T19:12:55.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T19:12:55.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T19:12:55.101 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-23T19:12:55.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-23T19:12:55.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.191 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.201 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.211 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-23T19:12:55.220 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-23T19:12:55.232 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-23T19:12:55.251 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:55.294 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:55.328 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-23T19:12:55.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-23T19:12:55.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-23T19:12:55.380 INFO:teuthology.task.internal:Transferring archived files... 2024-04-23T19:12:55.390 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi033 2024-04-23T19:12:55.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T19:12:55.495 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi106 2024-04-23T19:12:55.505 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T19:12:55.549 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114/remote/smithi122 2024-04-23T19:12:55.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-23T19:12:55.608 INFO:teuthology.task.internal:Removing archive directory... 2024-04-23T19:12:55.617 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T19:12:55.627 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T19:12:55.638 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-23T19:12:55.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-23T19:12:55.683 INFO:teuthology.task.internal:Not uploading archives. 2024-04-23T19:12:55.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-23T19:12:55.725 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-23T19:12:55.734 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T19:12:55.745 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T19:12:55.755 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-23T19:12:55.765 INFO:teuthology.orchestra.run.smithi033.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 19:12 /home/ubuntu/cephtest 2024-04-23T19:12:55.776 INFO:teuthology.orchestra.run.smithi106.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 19:12 /home/ubuntu/cephtest 2024-04-23T19:12:55.843 INFO:teuthology.orchestra.run.smithi122.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 23 19:12 /home/ubuntu/cephtest 2024-04-23T19:12:55.852 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-23T19:12:55.873 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-23T19:12:55.934 INFO:teuthology.nuke:Checking targets against current locks 2024-04-23T19:12:55.969 DEBUG:teuthology.nuke:shortname: smithi033 2024-04-23T19:12:55.978 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T19:12:56.015 DEBUG:teuthology.nuke:shortname: smithi106 2024-04-23T19:12:56.024 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T19:12:56.037 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.331214', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAdzgjUz4UKsk+xC7gwWrQTH52WMXwv0EtInqpZV35h/C2SQZcujHFi6AoM7hEcBBKUK91DbDitIJJ7hTtCVy+w='} 2024-04-23T19:12:56.059 DEBUG:teuthology.nuke:shortname: smithi122 2024-04-23T19:12:56.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-23T19:12:56.082 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.333189', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFu92g+Tux/vRXy/r03xV1Fu2Nbs1a+keFd+ch+/1ANXVh+Eja4TlcJq1YNqkn39KjFOAK+h5wdvxNa4fy9XZXo='} 2024-04-23T19:12:56.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-23_17:44:29-fs-wip-lusov-quiescer-distro-default-smithi/7670114', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-23 18:03:39.329517', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAbICBQ+WuqZTjhay4PcBhL2mlwnjuW0IQoPWBv/rwJU+LuTDd1CBalq5aHbgLBTxwYr5F4VWwc14EA0EtEcQaY='} 2024-04-23T19:12:56.132 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-23T19:12:56.140 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T19:12:56.172 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-23T19:12:56.180 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:12:56.259 INFO:teuthology.orchestra.console.smithi106:Power off 2024-04-23T19:12:56.266 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T19:12:56.289 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-23T19:12:56.298 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2024-04-23T19:12:56.306 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:12:56.328 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-23T19:12:56.336 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-23T19:12:56.353 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-23T19:12:56.368 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-23T19:12:56.376 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:12:56.408 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-23T19:13:00.298 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:00.361 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:00.418 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:00.438 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-23T19:13:00.484 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2024-04-23T19:13:00.539 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-23T19:13:04.453 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:04.494 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:04.550 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-23T19:13:04.572 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-23T19:13:04.614 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2024-04-23T19:13:04.669 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-23T19:13:04.683 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-23T19:13:04.724 INFO:teuthology.orchestra.console.smithi106:Power off completed 2024-04-23T19:13:04.780 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-23T19:13:04.894 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 3813.966193675995 failure_reason: 'error during quiesce thrashing: Error releasing set ''6bf9e7c1'': 1 (EPERM)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-23T19:13:04.904 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-23T19:13:04.987 INFO:teuthology.run:FAIL